Qualitative and Mixed Methods: Advantages and Disadvantages

Weekly Researcher Journal—Week 9-10

Guoping Feng, 11/12/2014

1. Reflective reading responses

In the past two weeks, we kept receiving input of theoretical grounding on research methods. Guba and Lincoln (2005) use paradigms to summarize the different methodological approaches: positivism, postpositivism, critical theory, constructivism, and the newly added category of participatory paradigm based on Heron and Reason’s work (p. 192). This classification of social research paradigms is basically the same as Neuman’s categorization of three major (positive social science, interpretive social science, critical social science) and two minor (feminist and postmodern) social research approaches (2006). Like Neuman’s employing ten questions to illustrate the various dimensions of difference among these approaches, Guba and Lincoln adopt a more extensive list for analysis: inquiry aim, nature of knowledge, knowledge accumulation, goodness or quality criteria, ethics, values, voice, inquirer posture, training, accommodation and commensurability, hegemony, action, control, validity, etc. The authors claim that “we stand at the threshold of history marked by multivocalities, contested meanings, paradigmatic controversies, and new text forms”, and that this has been an era of emancipation from hearing only the voices of Western Europe and emancipation from generations of science (p. 212). They even suspect we are “entering an age of greater spirituality”, with inquiries being intended to “reintegrate the sacred with the secular in ways that promote freedom and self-determination” (p. 212). Aha, the social research field may become not only a more harmonious play field with less paradigmatic contentions, struggle for legitimacy and supremacy, but probably or ideally also a paradise in which all deities and demons dance together.

Focusing on qualitative research, Cresswell (2013) analyzes, within a framework of ontology, epistemology, axiology and methodology, five social science research approaches: Postpositivism, Social Constructivism, Transformative/Postmodern, Pragmatism, Critical Theory. Cresswell’s inclusion of pragmaticism in the qualitative research seems to be unique, for pragmaticism serves as a strong foundation for mixed-method approach (Johnson & Onwuegbuzie, 2004).

Johnson and Onwuegbuzie enthusiastically advocate mixed methods, announcing that the time has time for this paradigm (2004, p. 14). They elicit pragmatism as the theoretical foundation for this advocation, and compare the strengths and weakness of quantitative, qualitative, and mixed-method researches. They differentiate between mixed-model designs and mixed-method designs, with the former mixing qualitative and quantitative approaches within or across the stages of the research process while the latter including a quantitative or a qualitative

phase in an overall research study. From previous responses in class, I feel most people are open-minded to the use of methods and ready to adopt the mixed-method approach.

But Johnson and Onwuegbuzie’s promotion of the mix-method approach doesn’t lack doubters. Yanchar and Williams (2006) argue that this methodological eclecticism often ignores the incompatibility of theoretical assumptions, leading to a failure to critically examine the theoretical background of methods and conceptual frameworks of research. They don’t just make criticisms, but also offer a set of 5C’s in evaluating the choice of research methods: contextual sensitivity, creativity, conceptual awareness, coherence, and critical reflection in research and evaluation. Aren’t they creative in proposing these 5C’s? I admit many researchers don’t engage in careful thinking and planning in terms of research method design, but at the same time I believe various explorations should be allowed or even encouraged. Many times, we take certain research methods as natural choices for certain research questions and give up in trying new methods and breaking new ways.

One article endorses evaluation as advocacy (Greene,1997). While I agree that sometimes it’s beneficial or even necessary to advocate for the research findings, evaluation in general should not take sides before it finishes. In fact, Greene’s article reminds me of a recent Pew poll: Americans now choose news channels in line with their worldviews, with the liberals tending to choose CNN, NPR and  MSNBC, and conservatives choosing Fox News. It seems that more news agencies are giving up the stance on objective reporting and opting for ideological stances (with huge commercial success). If evaluation can be advocacy, surely journalism can be advocacy, and research can be advocacy, and everything can be advocacy. And then we will never get rid of our own assumptions, presumptions, biases, and prejudices. In my view, research is different and nobler than any other enterprise just because it may bring about results we don’t want to face.

2.     Reflective class responses

I am frightened I couldn’t find any notes for last week’s class, except the work-sheet Jayne gave us for group discussion on our own research proposal (to be discussed in the next part). This is a good lesson for me to remember the importance of taking and keeping detailed notes in class. What I can remember is Kevin’s suffering from an injury on his eyelid and coming late to class from the University Health Service with one eye wearing a cover.

In this week’s class, Jayne first arranged us into different discussion groups according to our shared jigsaw readings, enabling me to engage in a first-ever conversation with Kristana (I now know she is from Des Moines, Iowa) who and I had both read the case study tracing an underachieving urban school’s decade-long reform (Mac Iver, 2007). Urban schools seem to have deep-rooted problems which evade quick solutions. Even though the school had seen significant improvements in attendance and graduation rates with two successive big reforms (Talent Development High School, and High Schools That Work; the author obviously regrets the discontinuation of the first reform because of the change of the president), some key achievement measurements are still horribly low: The number of ninth-grade students passing the sixth grade–level mathematics test rose from 35.1% in 1993 to 73.8% in 2003, but only 2.2% passed the ninth-grade algebra test in 2002 and just 14.3% passed it in 2004, and the number passing ninth-grade biology declined from 7.6% in 2002 to 3.7% in 2004 (p. 23). Obviously, the poor performance can be traced back to the middle school or even the elementary school period (in many countries, grade 6 is the last year of elementary school). I asked Kristana whether she had classmates meeting learning difficulties in the high school or even middle school period, and she said that she had attended best schools all along from elementary school to high school and known no such things, and the majority of her high school classmates (over 90%) went on to college. This may serve as another example of the polarities of American education.

At the end of our talk, we used the work-sheet provided by Jayne to evaluate this study using Yanchar and Williams’ 5C’s framework. Even though we agree this is a good study, it doesn’t fully meet quite a few standards, such as creativity of methodology, conceptual awareness of theoretical foundations, and critical reflection on assumptions. This in turn leads me to wonder whether this 5C evaluation framework is a valid one.

Our class discussion centered on the practicality and validity of mixed methods. Justin expressed his concern that mixed-method users may know neither the quantitative nor the qualitative methods fully well and therefore affect the quality of the research. I think this is a valid doubt. As the saying goes, we may become Jack of all trades but master of none.

3.     Documentation of research process

In week 9, Daniela and I worked through the many dimensions (ontology, epistemology, axiology, action, control, validity, voice) of Lincoln and Guba’s 4 paradigm framework (postpositivism, critical theory, constructivism, participatory) included in the work-sheet offered by Jayne, in thinking about our own research proposal. This week we kept thinking about the outline of our proposal to be presented next week. We seem to be stuck in our choice of research methodology. Kevin wants to employ a simple and straightforward quantitative study based on a single questionnaire with some open-ended qualitative questions included, while I have been thinking of getting data for a control group and the group to be studied. Both Kevin and Daniela believe this quasi-experimental design will evoke a whole series of questions. We are going to meet this Friday to talk further about our proposal, and I believe we will settle down on the methodology then and use the weekend to prepare for the forthcoming presentation.

References

Cresswell (2013). Qualitative inquiry and research design: Choosing Among Five Approaches, 3rd ed. Thousand Oaks, CA: SAGE Publications, Inc.

Greene, J. (1997). Evaluation as advocacy. Evaluation Practice, 18(1), 25-35.

Guba, E., & Lincoln, Y., (2005). Paradigmatic controversies, contradictions, and emerging confluences. In N. Denzin, & Y. Lincoln (Eds.). Handbook of qualitative research, 3rd edition. Thousand Oaks, CA: Sage.

Johnson, R, B. & Onwuegbuzie, A. J. (2004). Mixed methods research: A research paradigm whose time has come. Educational Researcher, 33(7), 14-26.

Mac Iver, M.A. (2007). What reform left behind: A decade of change at one urban high school. Education and Urban Society, 40(1), 3-35.

Neuman, W.L. (2006). Social research methods: Qualitative and quantitative approaches (6th Ed.). Boston: Pearson.

Yanchar, S.C., & Williams, D.D. (2007). Reconsidering the compatibility thesis and eclecticism: Five proposed guidelines for method use. Educational Researcher, 25(9), 3-12.