OrientationObservationIn-depth interviewsDocument analysis and semiologyConversation and discourse analysisSecondary Data
SurveysExperimentsEthicsResearch outcomes
Conclusion
Geoff Payne and Malcolm Williams explored the issue of generalisationwhen undertaking qualitative rsearch and provided some examples one of which, referring to focus groups, is reproduced here.
The Structure of Argument in Qualitative Sociology
The second example of generalizing practice (Wray, 2003) is a persuasive and
stimulating study of gender, age and health which also addresses national
policy. It is based on 'in-depth semi-structured interviews and focus groups'
with 170 women in several minority ethnic groups. Its strengths, however, have
nothing to do with the larger number of interviewees, but rather with the comparatively
greater caution in the conclusions it draws. The final section in the
article combines summary statements about what respondents have said, with
more general evaluative sentences about older women as a whole. The author
uses the format of examples from fieldwork—such as 'most women noted how
despite changes to the agility of their bodies....Being in good health was linked
to...' (Wray, 2003: 525)—as a basis for making immediately juxtaposed
general claims:
...the needs of older women should be recognized and prioritized by local councils
and other funding agencies. If older women are to have a good quality of life they
need to feel in control of their present and future. This meant making their own
decisions, being included in the wider social and political arena and continuing to
have a role in society. (p. 525, emphasis added)
This latter quotation shows first how the specificities of the study are reinterpreted
as policy imperatives of general applicability, and then in the last
italicized sentence, pulled back by use of the past tense to how the author
understands the empirical statements that have been elicited from informants.
The juxtaposition makes it clear from which part of the evidence the generalization
is coming. On the other hand, the quotation also demonstrates how
closely empirical specificities are linked to generalities, and how easy it is to
move from one to the other without explicit discussion of whether generalization
is justified on the basis of the data actually collected.
To be more precise, the author has collected data by interviews and focus
groups, from nine different self-defined ethnic groups (p. 513). If we assume
(because we are not told) that all of the nine ethnic groups were included in
some numbers, it seems a reasonable moderatum position to take that the experiences
of aging that the author discusses are not confined to only a few ethnic
groups, but are widely encountered. To the extent that the experiences of physical
activity and sense of self highlighted by the article were systematically
reported by very many of the women (and this again is not reported), it is a reasonable
moderatum position to suggest that many other (un-interviewed) older
women (elsewhere in Britain) might also share the experience. The kinds of
empowering physical activities reported may plausibly offer the benefits
suggested, but the article would be stronger if the grounds for claiming plausibility
were spelled out.
However, to claim that these particular activities have the same benefits for all older women in every part of Britain would be a riskier generalization. We
might also ask whether the activities might benefit younger women as well, or
indeed, older men. How much counter-evidence (older women exercising but
not reporting benefits) would it take to impugn the researcher's conclusions?
This second example is by no means atypical in not addressing these points.
Sociological conventions in research papers do not currently require such justifications.
If the intention is to generalize from the situation of a small number
of people (in this case 170) to local authority policy throughout Britain, then
we would suggest that it is incumbent on researchers to confront the need to
establish the grounds on which generalizing claims are being made, and the
limits to the generalization. Equally, even when the research goal is less concerned
with policy and simply addresses enhanced understanding, if researchers
are to make extensive generalizing statements like 'A code of ethics exists
among collectors' or 'an instrumental society...tends to empty life of its
richness' on the basis of eight interviews (Kjølsrød, 2003: 469, 474), it is not
unreasonable to ask on what grounds such statements can be justified.
We can summarize the generalization issues exemplified by this article as
follows:
1 Generalization is more likely to be plausible if it is approached with caution,
moderating the range of the generalizing conclusions. Too ambitiousa set of conclusions undermines the credibility of otherwise competent
research.
2 While the distinction between evidence and conclusion needs to be clearly
demarcated, so that the reader is enabled to distinguish between the two,
generalizations are more credible if the exposition connects the generalization
to the specifics of data that provide its foundation. This can be partially
achieved by simple juxtaposition, but explicit linkages with the
relevant sub-set of data are an even better way of sustaining the generalization.
This is not simply an aspect of final reportage. Good research
design helps to identify the data necessary for the kinds of generalizing conclusions
that may be anticipated at the outset.
3 Where several elements are involved in the study, these need to be fully
reported. Thus if there are several sub-groups among the informants, if
there are a variety of activities that operationalize the focal concepts, or
if outcomes may differ for wider population groups, the reader needs to
have access to published information about similarities and difference in
those sub-groups, activities and wider populations. This enables readers to
evaluate the status of the researcher's generalizations.
4 Similarly, researchers should indicate their own assumptions about similarities
and differences between the research site and the other ('receiving')
sites, and between the research's informants and other actors. This process
would not only assist the reader, but clarify in the researchers' own minds
what it is that they are claiming.
5 Even when the quality of the data is good, and the evidence is consistent
with the conclusions drawn, research design may not have eliminated alternative
explanations or constraints on generalization. The constraints on
generalization need to become a standard part of the analytical discussion.
