Some Problems Concerning the Use of qualitative Research Methods
J. Suoranta
Some Problems Concerning the Use of Qualitative Research Methods
Juha Suoranta
University of Lapland
Students often have difficulties in completing their master's thesis. It's rather usual that their degrees are delayed because of the problems in the use of different research methods. The purpose of this article is to describe some of the main problems that students encounter in using qualitative methods. Particularly in the context of Finnish teacher education, researchers doing qualitative research still have the burden of convincing the mainstream of educational research that their research is a meaningful undertaking.It is not only a possibility but often a sad fact that students too are told in some way or another which research is acceptable in a given institution.
In this study I asked students to think about and describe the concrete problems they have when they use different methods in their research. The subjects were students of teacher education, education and health education from three different universities in Finland. I gathered data using MEBS, that is, the method of empathy-based stories. The basic idea of the method is to vary certain elements in given descriptions such that half the students get different script than the other half. In the one they are doing their research with qualitative, in the other with quantitative methods. Students were asked to picture themselves in a situation where they had just finished their research. After that, I asked them to use their imaginations and write what would have happened. Thus, the data consist of material the subjects have a major hand in producing.
Despite its procedural emphasis, however, MEBS is not, strictly speaking, an experimental method. We can cite Spiegelberg and say that stories are only demonstrations designed as aids to the researcher's sluggish imagination (Garfinkel 1987, p. 36). It has been stressed that the methodological arsenal should not be heavier than the problem demands (Eskola, 1988, p. 251). In this particular case MEBS is a lighter alternative for studying the problems of using qualitative research than the survey or even the interview.
Students' empathy-based stories (N=55) were then analyzed using open coding (Strauss & Corbin, 1990, pp. 62-73). In coding, my basic question was: What is the main problem brought out in this sentence of the story? At first, I came up with over thirty conceptual labels, which were then reduced to the eleven categories indicated in Table 1. This categorization of students' written stories forms a typology of their problems. The order of items in the typology is not that of quantity of references to problems; rather, it tries to follow the research cycle to some extent. In what follows I will present the problems which students face when they use qualitative methods in their research.
Table 1:
Problems of using qualitative methods
1. Getting started
2. Sampling
3. Interviewing
4. Analysing data
5. Interpreting
6. Understanding and intersubjectivity
7. Theorising
8. Making truth
9. Validating
10. Writing and reporting
11. Legitimizing qualitative research
Problems of using qualitative methods
Getting Started:
As we all know from our own experience, it is almost always difficult to get things going when playing the human science game. In the data, however, students are not concerned so much about what to study as how to study: they debate whether the topic is too large to handle or, alternatively, too small to get anything out of. They all seem to have a research topic, but what they do not have is a proper research question. They are not sure what kind of data they should gather. Moreover, they feel that it is important but also difficult to ask the right kind of questions.
Another dilemma in getting started is the lack of qualitative studies and methodological literature in qualitative research. My observation is that this problem is more serious in Finland than in other contexts, although it is diminishing as the climate for exploring new forms of research becomes more encouraging. Expansion of methodological plurality can be seen in the growth of methodological literature.
Sampling:
Surprisingly and interestingly enough, students do not speak about sampling in a statistical way in the data. They are not worried about the randomness of a sample or statistical representativeness. Nor do they refer to theoretical or purposeful sampling, which are the sampling techniques usually used in qualitative research. There is, however, one implicit statement about the sampling problems in qualitative research: students reflect on whether the informants chosen are precisely the ones who could give information most relevant to the research problem.
Interviewing:
Quite another problem is interviewing. Its role as a data-gathering method is stressed in the data. That is largely because of a certain narrowness of the Finnish tradition in qualitative research. The anthropological research tradition is perhaps the most poorly developed in Finnish educational research _ assuming that there is such a tradition at all.
In any case, at least four problems can be distinguished in interviewing. First, students have problems with generating proper interview questions. In the following extract not only the design of questions but also the phenomenon of social acceptability is recognized:
Many times I recognized that interview questions, for example, were not very precise; the informant could understand them different from that intended. Many times informants gave answers which they thought to be sort of "right ones" and totally forgot their inner selves.
Second, they have difficulties in arranging meetings. In the worst case they are forced to drive around the Finnish country side to meet their respondents. Third, it is sometimes hard to handle the interview session. One extreme may be that a retired professor treats students as novices, the other that a cleaner does not know how to cope properly with academic people. Fourth, students are overloaded in transcribing their tapes or writing their field notes and feel that transcription is almost too heavy a duty.
Analyzing Data, Interpreting, Understanding and Intersubjectivity:
Analyzing, interpreting and understanding are hard to see as totally different pursuits. As a matter of fact, they are closely related, with each forming a part of a larger whole. The question of data analysis seems to be the most crucial to students. In one way or the other, it is mentioned repeatedly in the data. Students really do not know what to do after they have their data in hand. Students apparently face many uncertainties when dealing with qualitative data analysis. As one student put her problem:
I have trancribed my tapes and noticed that it is takes an enormous amount of time. I have no idea what to do next after trancribing the interviews, but maybe my advisor will tell me. Everything is so unclear. How do I start qualitative data analysis? What exactly would I like to get out of my data? Should I do some coding - but isn't that quantitative thinking already?
This raises the crucial question of whether too much attention is paid to data-gathering methods at the expense of analysis.
Another subject to notice, if we move to the problems of interpretation, is that students put great emphasis on the role of imagination in data handling. They feel that interpretation takes a great deal of creative effort. For them, to interpret is to understand and that makes interpretation such a difficult task. They do not know how to handle informants' contradictory accounts. Students were also aware of the problematic nature of understanding and intersubjectivity. They had extreme difficulties in understanding the meaning of some utterances. People use so many different words to refer to the same thing. Students seem to be captured in an assumption that the meaning of a word is somehow fixed a priori. Another implicit assumption is that meaning is a material thing.
Group discussions are one way to help students (Hutchinson & Webb 1991, p. 314). In discussions students discover that they are not the only ones experiencing anxiety. Group discussions function not only as therapy but also as forums where students and teachers can achieve a dialogue in which they raise questions of interpretations and, optimally, reach intersubjectively shared meanings about their interpretation.
Theorising:
An interesting question in the stories is students' conceptions of theory and its role in qualitative research. Nowadays it is virtually a dogma that the qualitative road to theory is through analytic induction. Interestingly, students do not deal with the idea of analytic induction. Instead, they handle the question of theory in quite a traditional way. First, they ask themselves what kind of theory they have in their research. In addition, they explicitly indicate that they are splitting their thesis into separate theory and empirical parts just like in a traditional empiricist report. Moreover, they talk about theory as an entity distinct from the empirical data and from their thesis as a whole.
Making Truth and Validating:
It goes without saying that one of the most problematic questions in human sciences is the question of truth and validity. Students' conception of truth is more or less based on naive realism. In the stories it is asked how the truth will come out. Students worry about how they will ascertain the real reasons for a given phenomenon. In the stories, the question of validity is considered as if it is something out there, something which students try to get hold of for their study. They write their stories as if they have not heard a thing about the discussion of different views of truth. However, recent developments in the sociology of science have emphasized that, as in the natural sciences, truth is also made rather than found in human and social sciences (Woolgar & Ashmore, 1988, p. 1).
Writing and Reporting:
Writing could be said to be crucially important to qualitative research because, to a great extent, the research rests on description and narration. Moreover, it could be argued that it stands or falls depending on its persuasive and rhetoric force. How then is the question of writing reflected in the stories? The role of original interviews in the report is one problem. For example, how many excerpts to include in the final report. Do the texts function as illustrations or objects in themselves. It is also difficult to build coherent descriptions which would accurately show the reader the contexts in which the research object appears.
Writing can become laborious for the student if she/he has been forced to refer many authors and to use massive amount of previous studies. Linking and comparing these studies and, at the same time, trying to produce one's own text is not easy, all the more so if a student feels that she/he lacks some essential secrets of so-called educational knowledge, whatever that might be. As one student writes:
I would have like to make my own work with my own ideas and findings, but because of my lack of knowledge it is impossible to make real science in this phase of studies.
What is encouraging to notice is that students seem to understand reporting and writing as fundamental parts of qualitative research. They write about writing and reporting together with data analysis and thus indicate, at least implicitly, the importance of language as a rhetorical device in qualitative research.
Legitimizing Qualitative Research:
Educational research methods are no longer as stable and canonised as before. They are in flux. Still, it can be seen from the data that students sometimes have trouble convincing their advisors that qualitative methods are not only possible but also effective and meaningful in educational studies. It seems to me that this phenomenon is quite universal. In the data, students are involved in situations where they are forced to think whether it is reasonable to apply qualitative methods at all. In one case, students were forced to recognize that someone in authority did not consider qualitative research worth doing at all. Furthermore, the requirements for qualitative research were higher than for traditional, that is, survey research. In the stories they still wonder if qualitative research is scientific enough.
This last fact brings us to the crucial question of how to define educational science. What do we actually mean when we use the term? More specifically, what are the methodological limits and possibilities of our science? As one student observed:
It was not easy to make up my mind and began to use qualitative methods. There were pressures all over but after all I am happy that I did not give in.
Conclusion:
As Hutchinson and Webb (1991) sum up their own teaching experience, "[L]earning how to make sense of data is perhaps the most challenging problem facing students of qualitative research methods." This observation is true and verified in many ways in the students' stories. One way or the other all the problems revolve around the process of making sense. It has many manifestations in the problems students meet.
First and foremost, the problem of sense making is seen in the analysis of textual materials. Students do not know how to start their work technically. They evidently are in serious need of proper technical advice for analysis. Second, the problem of sense making gets its manifestation in interpretation. This dimension touches not only qualitative but all researchers and all research. Students are astonished at the richness and plurality of ordinary language. They are not prepared for the constructivistic nature of peoples' speech. This is the place where they should get more advice.
A distinct problem is the legitimation struggle between qualitative and quantitative research. The original paradigm debate, as useful as it is for us to be aware of it, should not be emphasized too much in teaching qualitative research methods. Instead, when introducing and practicing qualitative methods, the emphasis should be on the multiple aspects of the social world.
The legitimation and acceptance of qualitative methodology, which supposedly have taken place in the educational sciences (Glesne & Webb, 1993, p. 254) _ at least in the United States _ go hand in hand with teaching of qualitative methods. When teaching qualitative methods, it is necessary to know that students might often have little knowledge about doing research. In the worst case, they are indoctrinated with research practices which are very limited in scope. With sufficient methodological education it is possible to teach students to understand the many details and varieties of qualitative research. After all, methodological awareness is a most important precondition for effective teaching of qualitative methods.
References:
Glesne, C. & Webb, R. (1993). Teaching Qualitative Research: Who Does What? The
International Journal of Qualitative Research in Education 3, 253-266.
Hutchinson, S. & Webb, R. (1991). Teaching Qualitative Research: Perennial Problems and Possible Solutions. In Morse, J. M. (Ed.). Qualitative Nursing Research: A Contemporary Dialogue. Newbury Park, CA: Sage, 301-321.
Strauss, A. & Gorbin, J. (1990). Basics of Qualitative Research. Newbury Park, CA: Sage.
Woolgar, S. & Ashmore, M. (1988). The Next Step: An Introduction to the Reflexive
Project. In Woolgar, S. (Ed.). Knowledge and Reflexivity: New Frontiers in the Sociology of Knowledge. London: Sage, 1-13.
Garfinkel, H. (1967). Studies in Ethnomethodology. Englewood Cliffs, N.J.: Prentice Hall.
Eskola, A. (1988). Non-Active Role-Playing: Some Experiences. In Eskola, A with A.
Kihlstr”m, D. Kivinen, K. Weckroth & O-H. Ylijoki. Blind Alleys in Social Psychology: A Search for Ways Out. Amsterdam: North-Holland.