OrientationObservationIn-depth interviewsDocument analysis and semiologyConversation and discourse analysisSecondary Data
SurveysExperimentsEthicsResearch outcomes
Conclusion
It is important that throughout the research process you continue to clarify and develop the main themes and issues under investigation. Nevertheless, having collected the data you may feel overwhelmed by it and unsure how to proceed with analysis.
The first task is to familiarise yourself fully with your material, reading through your expanded notes, transcribed taped-recordings or edited accounts. Listen to your tape-recordings again. Get to know your data (see, for example, CASE STUDY Urban working-class schools).
As explained in Section 3.6.1, analysing ethnographic material requires a systematic approach. After reading thorough your data vertically (that is, chronologically from start to finish) you should be able to identify major themes. If you have undertaken a series of follow-up interviews, rather than one-off interviews, you will already have started to identify these themes when constructing follow-up interview guides. Do not assume that the themes you identify early on in the research are necessarily the 'best' ones for analysing the data.
As you read your data certain themes will begin to emerge and you can copy your original chronological data and rearrange it into sections under headings and subheadings that correspond to the themes that emerge. You can then read the data 'horizontally' by theme. If, on reading, the themes prove inconsistent or illogical it will be necessary to break down your data into alternative themes until a suitable format is achieved.
Activity 4.5.1
1. Analyse the interview data collected in Activity 4.4.4.2.
2. Write a report that clearly shows how your data relates to the aims of your research. Use quotes from your respondents to illustrate your account.
3. Relate your research to appropriate sociological theory.
4.5.1 Accuracy, reliability and validity of in-depth interviews
In-depth interview data is the result of an interviewer gradually getting a respondent to provide detailed accounts of actions, attitudes, feelings, beliefs, relationships, perceptions and so on. Inevitably there are problems of accuracy, reliability and validity (see Sections 1.8 and 1.9). It is a problem establishing the accuracy of what someone tells you. In the prisoner's account of his first burglary (in CASE STUDY Distorting the truth), MichaelLittle (1990) noted the apparent inconsistency in the numbers of people involved. It would be possible, as he intended, to ask further about this to establish the accuracy of the account. However, circumstances may preclude the possibility of following up every detail in a lengthy account. You have to decide whether the accuracy of an account is important or whether it is more important to have an insight into the respondent's perception. One of Little's main theoretical propositions concerned young prisoners' ability to distort the truth, so while accuracy was important, it was not pursued to the detriment of understanding the way the respondents constructed reality.
The reliability of in-depth interviews, like participant observation, is sometimes called into question. The flexible nature of the enquiry and the difficulty of a repeat study makes it impossible to assess the reliability of the method. The issue of validity also arises, but in ways quite different from scheduled interviewing. With social surveys the major issue is whether the operationalised concept, as embodied in the schedule of questions, represents the theoretical concepts the researcher is interested in. With ethnographic interviewing, the researcher is trying to uncover the concepts that the respondents make use of in their understanding or interpretation of their world. The problem, then, is how the researcher knows that her or his understanding of key concepts is the same as the respondent's.
There is always the question, when analysing transcripts as to whether two different researchers would come up with the same analysis. A study by Gladney et al. (2003) was encouraging in suggesting that variation is analytic frames is small. The study examined the consistency of analysis undertaken by two independent multidisciplinary research groups. They were analysing semi-structured in-depth interviews with 63 middle school students about their beliefs related to violence. The study explored the extent to which the two teams derived similar themes from the interview data. The interviews, which asked three open-ended questions, were tape-recorded.
The first group analysed transcriptions of the interviews. Each member of the group independently analysed the transcripts to identify 'the salient beliefs about violence contained in the interviews, and independently determined the recurrent themes by classifying related salient beliefs into mutually exclusive categories' (Gladney et al., 2003, p. 302). The researchers then discussed individual analyses and came to consensus about the major recurrent themes.
The second group listened to the tapes simultaneously, separated by partitions to prohibit influencing others through non-verbal cues. Each analyst independently classified related beliefs into recurrent, mutually-exclusive themes. A discussion followed until a consensus was reached.
The study was based on a limited data set, which was highly structured and the analysis was limited to content analysis. The research team adopted the view, fromAjzen (1991) that 'salient themes can be captured in the five to nine most prevalent responses to an open-ended question' hence they sought a surface categorisation. However, it did show a considerable degree of consensus.
For example, the first group, looking at responses to the first open-ended question, identified the following five salient themes for adolescents' views about violence on television: (1) age dependent, (2) imitation, (3) moral, (4) negative, and (5) realism. They concluded that middle school adolescents thought that:
violence on television affected youngsters, especially those younger than themselves. This theme was followed by concerns about imitation of violence on television by people in general. There was also a belief that the violence on television is portraying behavior that is 'wrong' in a moral sense. There was a general expression of the belief that violence on television is negative, but that it is describing life realistically. (Gladney et al., 2003, p. 303)
The second group identified the following themes (1) affects young people,
(2) morals, (3) objection/quantity, (4) analogy to the real world, and (5) influence. This group also:
identified a belief in the potential negative effects of television violence on younger children...that violence on television was 'not right' and 'bad', and they identified a belief that there was too much violence on television and that it should be limited in some way. There were also predominant beliefs that the violence on television accurately depicted the 'real' world and that it had an
influence on the behavior of 'susceptible' people. (Gladney et al., 2003, p. 303)
The study suggested that there was internal reliability even with the diverse disciplinary perspectives involved in the analysis and interpretation of data.
Gladney et al. (2003) assumed that it was possible for two teams of researchers, working independently, to overlap in the coding, categories selected, and interpretation of the data.
Thus, we did not adopt a purely contructivist set of assumptions, which would argue that such an attempt to demonstrate overlap in interpretation was a futile attempt to achieve an artificial consensus (Mauthner et al., 1998; Seale, 1999: 42). On the other hand, we did not assume that researchers, including ourselves, were objective or could exempt our analysis from the influences of our values, beliefs and perspectives, both individually, and collectively. We assumed that these values, beliefs, and perspectives were rooted in disciplinary as well as personal experiences and that the comprehension, coding, and interpretation of data and the inferences drawn from them would reflect unique disciplinary and personal perspectives. (Gladney et al., 2003, pp. 298–9)
There are questions that the in-depth interviewer needs to reflect upon. How much of the information gathered derives from the informants and how much is the result of the researcher's own preconceptions and interpretation? To what extent has the information gathered been distorted by what the researcher has said or done? To what extent have informants provided the data that they think the interviewer wants? It has been shown that responses to in-depth interviews are affected by the expectations and attitudes of the interviewer (Hyman, 1954).
It is therefore important for ethnographic researchers to be reflexive (see Section 1.11). This means that they should adopt a critical attitude towards their research, reflecting on the impact they have had on the research context and thus the data gained. When understanding in-depth interviews it is, therefore, important to consider the extent to which your presence, by disrupting the natural context, influenced your informant's responses. Were you able to set respondents at their ease? Did you manage to create a relaxed atmosphere and develop a rapport with your subject? Were informants given the scope to develop their ideas, or were they constrained by badly-worded questions that 'led' them to certain responses?
Reflexivity is aided, if you keep an ongoing journal or research diary in which you record your involvement in the research, decisions you make and your reflections on the research situation and the research process (See Section 1.14).
For example, DonnaLuff (1999, p. 694), during her research into non-feminist women, made the following notes in her research diary:
She confused my stereotypes! I unconsciously found myself thinking that she appeared more like a 'feminist' than the 'moral right' (now what does that tell me, that somewhere I think feminists wear trousers and lots of earrings not printed 'frocks' – do I even fit my own stereotype?!). Her home was also very different from the others, but many of her views were actually more conservative than many of the women [who wear 'nice' dresses and have immaculate homes]. I left feeling confounded – very usefully.
As Maguire (2005, p. 30) pointed out, reflexivity involves an attempt to reveal how the researcher's
own biography, as well as their reading and reworking of the data, may detract from the authentic voice of the respondent(s) or may have merely reflected their own voice (Wolf, 1996). However, as Walkerdine et al., (2001: 84) recognize, there is no 'Holy Grail of the perfect method'. The act of producing an interview is more complex than telling life as it is. It is an act and an art of reconstructing, retelling and remembering. As in any conversation, some points get more attention than others, small asides can either get ignored or exaggerated and some aspects (which later turn out to be very important) are not always fully explored. In an extended interview between two people who know one another well, turn taking can break down; half-thoughts and conjectures are tried out and played with. What can happen is that where the researched and researchers share interests, these points of contact can produce conversations that illuminate these concerns
Another element of reflexivity, particularly for in-depth interview research is the need to be aware of being led by key informants or dominant interviewees. Barney Glaser and Anselm Strauss (1967) suggested that care needs to be taken when using key informants. Although key informants can be useful in some situations, they can distort the data without you realising it. If someone is particularly forthcoming or appears to be especially knowledgeable it is tempting to see him or her as a key informant and to make considerable use of the data provided. In certain circumstances this is perfectly acceptable. If, on the other hand, you are trying to get a broad or 'representative' view of a sample of people (asLittle 1990 was in his study of young prison inmates) then avoid key informants, or at least be reflexive about the extent to which they have influenced your understanding of the social world you are researching.
One also needs to be aware that, as Maguire (2005, p. 429) explained, 'All interviews are social occasions, performances by the interviewer as much as the interviewee, and interviews are about presentation of self and sometimes involve hyper-performativity (Atkinson and Coffey, 2002)'. How much the performativity impacts on the research is something the researcher has to assess. When an interviews is lengthy or the subject is interviewed on more than one occasion, it is hard to perform a role, especially if the interview situation is relaxed and good rapport is established between those taking part. Focus groups, potentially provide a context for performativity (see Section 4.4.3.6.4).
Analysing focus group data is much the same as analysing any other 'qualitative' data set (such as, recordings of dialogue, whether or not transcribed, plus any accompanying information on visual gestures or tone and tenor of voiced delivery). How this body of information is analysed depends on the purpose of the study. A positivistic analysis might focus on the frequency of specific types of utterance. A phenomenological analysis will try and elicit meanings from what is said or, even, what is not said. A critical dialectical analysis will explore how specifics are related to wider issues, how the current is related to the past and will identify the critique of prevailing ideology within the data.
Two examples of focus-based research looking at prisoner health will be used to explore the analytic process.
Mary Ferguson, Lauren Davies, Devon Indig and Mel Miller (2011) studied prisoner and juvenile detainee views on their health while in prison, using nine focus groups in low-to-medium security prisons, juvenile detention centres and a Community Restorative Centre (for ex-prisoners) in the greater Sydney metropolitan area in 2009. They described the analaytic process as follows:
All focus groups were audio-recorded and then fully transcribed. The data from all the groups were analysed using a 'framework analysis' approach, based on Krueger's (1994) method, which draws heavily on the framework analysis procedure first described by Ritchie and Spencer (1994). The five key stages outlined by Krueger (1994) are: familiarisation; identifying a thematic framework; indexing; charting; mapping and interpretation. The other distinctive aspect of framework analysis is that although it uses a thematic approach, it allows themes to develop both from the research questions and from the narratives of research participants.
To enhance reliability, codes were developed for identifying themes and then tested by independent coders, using the process developed by Hruschka et al., (2004) to maximise inter-coder reliability in coding open-ended data.
A total of 1,656 segments of qualitative data were analysed. A coding matrix was developed, based on themes emerging from the data (e.g., access to health care, medication, diet/exercise) and the aspects of each theme (e.g., systems/policies, physical environment, information). Each segment of data could have one or more codes, which was entered into a spreadsheet. Each segment of data was identified by a unique focus group identifier and by the gender of the speaker. Age data on the participants was not available, but the focus groups were split by adults and juveniles. A database of de-identified coded qualitative data was developed that could be sorted into various themes, and was traceable for verification and checking of the final analysis and interpretation.
Reporting the results, Ferguson et al., (2011) stated that:
The most common theme among all groups was associated with experiencing difficulty accessing health care in custody.... In particular, participants expressed frustration with the triage process...[and] were also frustrated about the delays in seeing a doctor, which led to growing concern among participants about their condition, or led to a decision not to seek treatment. In facilities where participants were involved in work (either offsite or on the prison campus), attending work was often reported as preventing access to clinics, as clinic hours were viewed as inflexible and not always accommodating those on work release.
Ferguson et al. also reported concerns about being treated as a genuine patient because of all the feigned illnesses with the result that prisoners assumed nurses made assumptions about the validity of all prisoners' health claims. There were concerns about the method and timing of dispensing of non-prescription drugs such as paracetamol (and, indeed, the overuse of paracetamol in situations where stronger pain killers were required). Diet and exercise were other health-related concerns raised in the focus groups.
Ferguson et al. also stated that privacy was a major concern:
Lack of privacy when accessing health care services in custody was a problem mentioned across all focus groups.... In most cases participants were required to discuss sensitive issues, such as why they wanted to make a clinic appointment, in front of other prisoners and prison staff. In some instances, participants chose not to seek treatment, rather than speak in front of others about their condition.... In many cases, the prison officer who escorted the participant into the clinic would be present during the consultation. On some occasions, however, the nurse or doctor would insist on closing the door so the officer was not able to hear their discussions.
Douglas et al. (2009) also explored female prisoners' perspective on their health while incarcerated. Their study, conducted in two local women's prisons in England, used six focus groups, run within prison healthcare centres, and 12 interviews conducted in prison units. Only women detained for at least one month, and therefore with sufficient experience of imprisonment, were eligible to participate. These women were approached by the researchers (prison staff were not involved) and provided with a written information sheet and verbal explanation. Most of the 37 women recruited had previously participated in a related questionnaire study and were known to the researchers (Plugge et al. 2006). Written informed consent was obtained from each woman before each group/interview. The mixed method approach benefitted from the generative processes of dialogue that a group dynamic can provide, while also enabling individual women to discuss issues too sensitive or personal to be explored in a focus group.
A prepared semi-structured topic guide was used in the focus groups and interviews, 'which aimed to explore women's perceptions of "health" and "healthiness"; health problems of women in prison; personal health status prior to imprisonment; impact of imprisonment on health; experiences of prison healthcare services; and recommendations for service development' (Douglas et al., 2009, p. 750). The authors explain the analysis as follows:
Our aim was to identify women's own assessments of the impact of imprisonment upon their health. We elected to conduct a simple thematic analysis that would allow us to provide useful information for policy-makers and practitioners in the prison setting in a responsive and timely way (Braun and Clarke, 2006). After familiarising ourselves with the transcribed interviews and focus groups, we (ND and EP) coded the recorded speech, categorising and collating major themes and subthemes within the data to form coherent patterns. We also searched for deviant cases. We then reviewed and discussed our interpretations, resolving the few minor differences in coding that emerged during this process. We were unable to verify our interpretation with the participants, as most had been released or transferred. However, we were able to review and refine our interpretations with key professional stakeholders at a feedback meeting.
The results of the study revealed, inter alia, how the immediate shock and disorientation of imprisonment, isolation from their families, impacted differently on the women's physical and mental health. A previous history of chaotic drug misuse and violent victimisation was an important distinction. The women expressed their 'stark sense of shock, disbelief and isolation on reception to prison'. They were concerned about who was looking after children and also scared of potential bullying and intimidation within prison. They reported witnessing difficult detoxifications, epileptic seizures, self-harm and other mental health problems, all of which they felt took a toll on their own mental health. Douglas et al. (2009, p. 750) included quotes from focus groups:
You've got a lot on your mind, you're a mother, you're locked away and your children have been taken away from you, you've nothing, you're nursing a wound inside you. That's a pain that no pain relief—no painkiller can kill (Focus group 5).
Coming in prison I should have had someone to counsel me about the whole prison system and whatever because I've just come in here, I would kill myself if I wasn't mentally strong. Because I've never been in this environment before and I was in a room with three heroin users plus someone who is coming off crack and I was the only person in there who has never ever taken drugs and the first time I've ever seen someone fit was in front of me (Focus group 2).
Writing up the outcomes of in-depth interview-based research follows the same process as writing up any research. The write up should refer to the purpose of the research and be focused around a central thesis, which the data confirms (or not). The account should be written in an accessible and convincing style.
The most powerful material when undertaking in-depth interviews is the words of the subjects that elaborate key points and reveal their underlying meanings or understanding of social phenomena. So, use quotes when writing up.
However, do not overdo the quotes, they are the 'pictures' that illustrate your central argument. Furthermore, avoid introducing a quote by saying what it says and then, in effect, reproducing the quote and effectively repeating yourself. This becomes tedious for the reader and they stop reading the quotes. Also avoid long lists of quotes, especially decontextualised short snippets.
Make sure all quotes are attributed to individuals, even if you keep the actual identity of the respondent concealed. For example, use codes for each respondent that indicate their age and gender or other salient fact that bears upon the central thesis. This way the reader can see the relevance of the comment without you, as author, having to explain it each time. Also, the reader can see whether or not you are using the same respondent repeatedly or whether sentiments are expressed by a range of respondents.
There is always an issue of whether you just go ahead and include quotes without first asking the respondent for permission or at least checking accuracy or whether you feel obliged to seek permission despite the quote being anonymous.Luff (1999, p. 699), for example, suggested that:
Whilst I would agree that as researcher I retained power during the writing up, negotiations with participants at this stage warrant further consideration.
In my study, I sent the participants copies of the quotes I intended to use from their interviews for their comments. Only a small minority responded to these letters, and of these only two raised substantial objections to their quotations. This process involved renegotiating rapport, after a considerable time-lapse and in different conditions of power. At this stage, for example, one woman felt very anxious about seeing her comments presented outside of the 'rapport' that had developed in the interview. If this stage of seeking interviewees' comments is to be taken seriously, then it inevitably involves some ceding of control and power over the finished write-up, albeit often small, on the part of the researcher.
