BonusAppendix: The Qualitative-Quantitative Debate
©2000-2021 Mark L. Mitchell & Janina M. Jolley
Allrights reserved.
Overview
Inthis appendix, we bravely and foolishly go where few textbook authors have gonebefore— into the jaws of one of the most controversial and incendiarytopics in all of psychology: the qualitative-quantitative debate. We would liketo rise above the war of words between the quantitative and qualitative camps.However, to do this, we must first address some of the rhetoric used by bothsides.
War of Words
Tobegin to understand the controversy, you must realize that some scientificresearchers find even the terms “quantitative research” and“qualitative research” offensive. For most scientificpsychologists, for most of the past 50 years, the only legitimate researchmethods have been the traditional, objective, scientific methods discussed in Researchdesign explained. As advocates ofthe qualitative approach sadly concede, it is virtually impossible to do adissertation in psychology that does not use these traditional methods (Tolman& Brydon-Miller, 1997). Thepeople pushing for different methods, however, did not want to call theirapproaches "different methods," “unscientific methods,”“non-traditional methods,” or “alternative methods”because such terms would suggest that their methods were inferior. Therefore,they called traditional scientific methods “quantitative,” andtheir methods “qualitative.” Using these terms was a clever way ofimplying that (1) the two approaches were of equal value, and that (2)exclusion of qualitative methods would be narrow and unfair.
As qualitative advocates admit, there were twodrawbacks to this rhetorical strategy. First, the term “qualitativeresearch methods” is misleading because (a) qualitative methods have verylittle in common, except that they are not traditional methods and (b) qualitative research is sometimesanalyzed quantitatively. Second, because of the fundamental differences between“qualitative” and “quantitative” approaches, mostadvocates of qualitative research view the approaches as incompatible (Burman,1997; Rabinowitz & Weseen, 1997). Rather than having qualitative researchincorporated into the traditional research methods, qualitative advocates wantto fundamentally transform the way psychologists think about psychology andresearch (Brydon-Miller & Tolman, 1997; Burman, 1997).
To begin to understand what qualitative research isand why qualitative research is so incompatible with traditional scientificresearch, let us look at one strategy that qualitative advocates have suggestedfor promoting qualitative research. This strategy is to reclaim the word"empirical" from the quantitative researchers (Brydon-Miller &Tolman, 1997). That is, psychologists often say that psychology is an empiricalfield of study. However, by "empirical" most psychologists meanobjective experience. That is, quantitative researchers have defined"empirical" as the kind of data that their research produces.However, as qualitative researchers point out, the literal definition of theterm "empirical" means "from experience." Thus, yourfeelings toward a participant, as well as your own reflections on yourexperiences during the research, although not objective, are “empirical.”In other words, under the literal definition of "empirical," personalexperiences and subjective impressions would be "data" —ifpsychology were defined as "the empirical study of the mind and behaviorof individuals."
Now that you have a hint as to what qualitative researchis, let us define it. We will tentatively define qualitative researchmethods as a wide range ofnon-objective tools (including interviews and participant observation) designedto provide rich, in-depth, meaningful descriptions.
The key is that these methods, unlike all the othermethods we have discussed in this text, avoid objectivity. Their focus is moreon interpreting events and finding themes in experiences, rather than infocusing on the objective facts (Mason, 1996). Qualitative researchers usuallyavoid objectivity for at least one of the following four reasons:
1. They think that thereare certain phenomena (such asguilt about lying or the experience of being powerless) that objective methodscannot address.
2. They think thatobjective methods, by ignoring the researcher’s and participant’sinsights, lose too much information.
3. They do not believethat “objective” methods are truly objective.
4. They think that the objective approach does notallow them to tell their story. They want to allow their participants' voicesand, in some cases, their own voice to be heard. Objectivity, by trying tominimize the role of the observer and by treating the participants as"objects" of study, muffles both the voices of the participant andthe observer.
Aswe have seen, qualitative advocates do not pride themselves on being objectivescientists. Some openly reject the scientific approach. Others believe whatthey are doing is scientific, but argue that objectivity is not a definingcharacteristic of science. In contrast, the field of psychology prides itselfon being an objective science (Porter, 1996; Tolman & Brydon-Miller, 1997).Thus, qualitative methods are infrequently used in psychology. However, insocial sciences that are less wed to objectivity, qualitative methods are morefrequently used. For example, anthropologists use participant observation;sociologists conduct in-depth interviews; teachers and journalists intensivelyobserve and interview a single person for days; marketing consultants use focusgroups; and philosophers may write papers using their own reactions—oreven how they think someone else would think—as “data.”
To get a better idea of (a) what qualitative methodsare and (b) how qualitative methods differ from traditional quantitativemethods, we will look at two examples of qualitative research from the point ofview of a traditional, quantitative researcher. Note that we are looking atthese examples from the viewpoint of a quantitative researcher merely tohighlight the differences between the two perspectives. We are quite aware thatsome qualitative researchers would say that looking at qualitative researchfrom the perspective of a quantitative research is unfair (just as manyquantitative researchers would object to looking at quantitative research froma qualitative perspective). The first tool we will look at is a tool thatbusinesses often use—the focus group. Then, we will look at a qualitativemethod that some people use for getting insights into human nature—thein-depth interview.
The Focus Group
Inthe focus group, six to twelve paid volunteers are brought into a room todiscuss an issue. Typically, the moderator has about five set questions he orshe asks during the 90-minute group interview. Beyond asking these questions,the moderator will ask questions to (a) follow-up on what volunteers say and to(b) involve all members of the group. Often, the client and another researcherwill be in an adjoining room watching the session through a one-way mirror.
Soon after the session is over, the researcher willgive the client a tape of the session and a short summary of the session. Thesummary will focus on the themes that the researcher “saw” in thesession.
Although many businesses and most qualitativeresearchers would accept the focus group summary as valid, the quantitativeresearcher would not. As you will see, the psychological scientist would haveat least three questions about the conclusions contained in themoderator’s summary.
First, did these themes come from the participantsor from the moderator? Rather than representing the reality of the situation,the moderator’s “findings” may only represent themoderator’s biases. Unlike the traditional scientist, the moderator doesnot have to meet some objective standard, like showing that the results arestatistically significant, before declaring to have discovered a pattern in thedata. Instead, the focus group moderator may have to do little more than claimthat she sensed a certain feeling in the group. Admittedly, she may back upthis feeling with a few selected quotes from the focus group members. However,she might have backed up the opposite conclusion by selecting some differentquotes.
In rebuttal, the focus group moderator could makesix points:
1. The client has accessto a videotape of the entire session, thus the client can decide whether theinterpretation is biased. At the very least, unlike in structured surveys, theclient is hearing the participants’ own freely chosen words.
2. The moderator is awareof the possibility of bias and has been trained to deal with it.
3. The moderator’s“lack of objectivity” could also be called “insight.”
4. There is probably morethan one subjective reality. Some participants may hate the product; others maylove it. One moderator may pick up on one feeling that group members arehaving; another moderator might pick up on a different, possibly conflictingfeeling.
5. The moderator may ask participantswhether the moderator’s interpretation “rings true.”
6. With such rich,complex data and with such few participants, statistical analysis is notfeasible.
The second question a quantitative researcher mighthave is, "Even if the moderator correctly captured what participants said,can we believe what the participants said?" There are numerous problemswith self-report data. Participants may lie, try to please the moderator,exaggerate, forget, and very often, just not really know. As we mentionthroughout Research design explained (e.g., pages 212-215), self-reports can be inaccurate,even for such simple things as reporting how one uses a new product. (Indeed, some attribute much of Japan's dominance of the U.S. personal electronics market to Japanese companies using observation to study how consumers interact with their products whereas American companies relied on surveys.)
The qualitative researcher might reply that onlyself-report data can do justice to understanding what people are thinking. Anyother method would be too incomplete and too indirect.
The third question a quantitative researcher mighthave is, "Even if the moderator is (a) correct about the participants and(b) the participant’s verbal statements accurately reflect their thoughtprocesses, how can we make any generalizations from the focus group?" The12 participants are a small, non-random sample of the population of interest.Furthermore, the participants are not responding as independent individuals. Instead, the participants areresponding as a group. Consequently, the more dominant participants’voices will be heard more, and there will be a tendency for members to conformto those leaders.
Because of concerns about the external validity ofsmall, non-random, non-independent samples, the popularity of focus groups hasdeclined. For example, some ad agencies have stopped using focus groups becauseof cases of television ads that were approved by focus groups, but were laterlabeled by a large percentage of television viewers as offensive. In some cases,companies had to remove those ads after only one airing and apologize.
In another notable case, two-term President BillClinton may have won re-election because he put less faith in focus groups thanBob Dole, his republican challenger. Bob Dole asked voters to vote for thepresidential candidate they trusted more. Then, Dole, in an attempt to convincevoters that they trusted him more than Clinton, asked voters to think aboutwhom they would rather have baby-sit their kids. He was sure people would choosehim because his focus groups had told him that was the case. However, in atelephone survey of a random sample of voters, over 75% of the respondents saidthey would prefer Clinton to baby sit their kids.
Qualitative researchers acknowledge that errors canbe made when generalizing from samples, but point out that such errors occur inboth qualitative and quantitative research. Quantitative pollsters, forexample, have made large mistakes in predicting election results. Furthermore,the problem with the focus groups in the cases mentioned above could have beendue to the investigator improperly selecting the sample of participants—aproblem that can also ruin external validity in quantitative research.
In-DepthInterviews
Notall qualitative research is designed to address an applied problem. Often, asin quantitative research, the goal is to either describe or understand someaspect of human nature or experience, such as love. However, rather than usethe objective methods we described in this text, qualitative investigatorsadopt more subjective, personal approach. For example, consider a study inwhich an investigator follows two classmates over the period of a year. Sheextensively interviews her classmates about their relationships and intensivelyanalyzes the transcripts of those interviews. She concludes that people oftendo not want to commit to a relationship because they have a “fear oflosing themselves, of having themselves devoured.”
As with focus groups, some people consider this typeof investigation quite acceptable. Many philosophers believe in this approach.Many qualitative researchers believe in this approach. The traditionalpsychological scientist, however, would have at least three questions aboutthis approach.
First, how does the investigator know that theclassmates have this extreme fear of losing themselves? A psychologist would beskeptical about whether the construct “fear of losing oneself”exists. If it does exist, the psychologist would still need objective evidencethat the measure of this construct was reliable and valid.
The qualitative investigator would argue that shehas done a thorough job of establishing that the construct exists. Thedifference is that whereas the quantitative researcher would attack the problemfrom the outside in (from observable behavior to internal thoughts), thequalitative researcher attacks the problem from the inside—by empathizingwith the clients. To rule out bias, she came into the project with anopen-mind. She was not, like some “objective” scientists, trying toconfirm a theory or hypothesis.To determine what participants thought, she has extensively analyzedtranscripts of what they said, empathized with them, and asked them if herinterpretation was accurate.
The second question a quantitative researcher mighthave is, "How does the investigator know what caused the break up ofrelationships?" Traditional psychologists are very skeptical aboutcause-effect conclusions. They realize how difficult it is to isolate the causeof an effect. The qualitative investigator would say that (a) as a result ofthe in-depth interviews, she was aware of any other relevant, potential causalvariables and was able to rule them out, and (b) the participants agreed withher analysis, and they ought to know.
The third question a quantitative researcher mighthave is, "How can anyone make generalizations about most people based onstudying only two participants?" This is a question that many qualitativeresearchers would also ask. The qualitative investigator would probably concedethat the study has only documented the inner lives of the two participants andthat replication would be a good idea. However, the qualitative researchermight argue that there are universal truths that apply to everyone.
Conflicts betweenQuantitative and Qualitative Approaches
Asour discussion of the focus group and the in-depth interview suggest,quantitative researchers do not readily accept qualitative research. Indeed,some traditional psychologists see these alternative methods as beingfrighteningly similar to those methods used by infomercials, journalists,novelists, and some talk show hosts. Some traditionalpsychologists—although accepting that these methods are serious,systematic, legitimate methods of inquiry—still believe that thesemethods have no place in psychology.
The conflict between quantitative and qualitative researchers is notone-sided. Some qualitative researchers, for example, find scientific researchappalling.
In short, qualitative and quantitative researchers have quite differentperspectives. In the next sections, we will highlight seven of the maindifferences:
1. Disagreements aboutthe need for an alternative to traditional scientific methods,
2. The value ofself-report,
3. The value of objectivity,
4. The value of usingnumbers to describe data,
5. The value of trainingand standardization,
6. The best way to dealwith the fact that human thought and behavior is complex, and
7. The way to evaluatethe internal, construct, and external validity of research.
The Value of theScientific Approach
Psychology is usually defined as the science of human and animal behavior. As we discussed inChapter 1, the scientific approach is what separates psychology from many otherfields, including philosophy. As you also saw in Chapter 1, the scientificmethod has been incredibly productive for psychology. Indeed, psychologicalscience is progressing so fast that it is difficult for even professors to keepup with the developments in their own specialty areas, much less with all thedevelopments in the entire field.
The qualitative researcher, however, claims that,despite the apparent progress of scientific psychology, the human world is toodifferent from the natural world to use the objective scientific method. Humantraits such as goals, plans, traditions, and the ability to reflect requirethat humans be studied through non-scientific means. Thus, qualitativeresearchers think the objective scientific approach is oversimplified andleaves too much out. According to some qualitative researchers, if we want arich picture of how people think and feel, objective scientific psychology haslittle to offer.
The Value ofSelf-Report
Qualitativeresearchers believe that one reason quantitative research is limited is thatquantitative researchers do not take full advantage of what participants haveto say. Unlike quantitative researchers, qualitative researchers tend to placea great deal of faith in self-report. This belief in self-report allowsqualitative researchers to use participants as collaborators.
According to quantitative researchers, this faith isunjustified. If people were aware of their own minds, human behavior would notbe so mysterious. Indeed, there would not have been a need for psychologists.Long before the beginning of scientific psychology, people would have knowneverything there was to know about the human mind. However, as scientificresearch has clearly established, self-report data is highly questionable(Sechrest & Sidani, 1995). Research has clearly established that peopleoften
1. Do not remember whatthey did;
2. Do not know why theydo what they do; and
3. Cannot accuratelypredict what they would do in a certain situation.
Furthermore, if one really believedself-reports, one would have to believe in all sorts of things, such as mengiving birth, Elvis sightings, and UFO abductions (thousands of peoplehave reported them).
Thehistory of psychology testifies to two facts: (a) introspective reports cannotbe trusted and (b) people would like to believe in introspective, self-reports.By 1867, 12 years before Wilhelm Wundt established the first laboratory inpsychology, the problems with self-report had been clearly articulated.Nevertheless, in this lab, Wundt and his followers relied heavily onintrospective self-reports. Admittedly, they went to great lengths to try to make such self-reportvalid. His students went through a year of training and more than 10,000practice self-reports before they were considered qualified to make self-reports.Despite these efforts, there was little evidence that these self-reports wereaccurate. Furthermore, Wundt was never able to successfully address theproblems with self-report that had been clearly articulated back in 1867. Alittle later, other psychologists tried a more natural, common sense approachto introspection. Once again, there was little evidence that these self-reportswere valid. Once again, the problems with self-report identified in 1867 werestill not successfully addressed. Today, after more than 100 years of trying torescue introspective self-reports, the main outcome is that we have discoveredthat there are even more problems with self-report than those identified in1867.
The Value ofObjective Evidence
Tomost quantitative researchers, a careful analysis of the history of bothscience and scientific psychology shows not only that we should questionself-report data, but also that we should try to be as objective as possible.Objectivity, as we pointed out in Chapter 1, is central to the scientificapproach. Scientists try to eliminate bias and subjectivity to find knowabletruths. Without objectivity, we may end up just seeing what we expect to seerather than testing reality. Without objectivity, you selectively noticeinformation that fits your biases and you selectively interpret ambiguousinformation to fit your biases. According to quantitative researchers,objective methods help you find the truth, whereas as non-objective methodsallow you to find what you want to find. Thus, quantitative researchers wouldargue that people using non-objective methods are being seduced. In otherwords, the person who has strong beliefs may not be pleased when the cold,objective truth, unearthed by quantitative methods, does not support all thesepassionately held beliefs. That same person may be pleased when, by usingqualitative methods, the person discovers strong support for those samebeliefs. Thus, the person may come to believe in qualitative research.
In short, quantitative researchers view objectivity as a defining characteristic of science. In that view, to claim to be doing science when not being objective is to be engaged in pseudoscience (Scott, 2007). Thus, some psychologists view their problems with the qualitative researchers as similar to the problems that astronomers have with those who dispute the idea that the earth revolves around the sun. Astronomers would prefer that the scientific position that the earth revolves around the sun be called the scientific view and that the view that the sun revolves around the earth be referred to as the pseudoscientific (or at least nonscientific) view. Their opponents, however, refer to the astronomer's position as the "heliocentric view" and call their earth-centered view the "geocentric view." These labels distract some people from noticing a central difference between the positions: One is supported by the objective evidence, the other is not. Likewise, some psychologists believe that the labels "qualitative" and "quantitative" have distracted people from seeing that the "quantitative" approach is superior because it is objective.
Asyou will see, qualitative researchers do not accept the quantitativeresearchers’ arguments about the value of objectivity. Some argue thatthe price of objectivity is too high. Others claim that the objectivity ofquantitative methods is just an illusion. Still others argue that objectivityis not a valuable goal.
The Costs of Objectivity
Somequalitative researchers claim that, even if objectivity removes the researcherfrom bias, the cost of objectivity is too high. To be more specific,qualitative researchers see two main drawbacks to objectivity. First,objectivity costs the researcher access to rich sources of information. Second,by treating the participant as objects, participants are not treated in a moraland ethical way.
Loss of Information
Theobjective approach means that you have to exclude many sources of information.The researcher cannot use any personal insights he or she may have about theparticipant. The researcher is similarly cut off from valuable information thatthe participant might provide. Consequently, many questions about people cannotbe answered because there is no objective way to study those questions. Forthose questions that researchers can study objectively, the cost of finding outis that they must restrict how the participant responds. Thus, the participantmay be forced to respond in some artificially simple way (pressing a key on acomputer or circling a number on a rating scale) rather than in a complex andnatural way. As a result of objective researchers isolating themselves fromparticipants, objective researchers lose access to the in-depth knowledge ofthe human condition that art, literature, philosophy, and other less objectivefields can provide.
Ethical Costs
Aswe have suggested, being objective may involve treating the participant as the object of study. As we mentioned earlier, some qualitativeresearchers believe that the cost of this impersonal, “objective” treatment is that the researcherloses out on the valuable information the participant has to share with theresearcher.
Part of this loss of information is due to theone-sided relationship between the researcher and the “object” ofstudy. If the researcher would treat the participant as another thinking,intelligent life form—rather than as an object—then the researchercould benefit from the participant’s insights. The participant may havemuch to teach the researcher, but the researcher does not allow the participantto collaborate with the researcher. Indeed, from the objectiveresearcher’s point of view, collaboration between the researcher and theparticipant would bias the results.
Toprevent collaboration—and to keep the participant in the role of anobject of study—the participant in a quantitative study is often lessthan completely informed about the study—and may even be deceived aboutthe study. Furthermore, the participant expected to do what the researcherinstructs the participant to do. The conventional researcher views thismomentary failure to equally share power and knowledge as the cost of gettingobjectively valid information.
Somequalitative researchers view this impersonal, “object-ive”treatment as highly unethical. Treating participants like objects is demeaning,and it gives the researcher the excuse to exert power over the“objects” of the research. Rather than “oppressing” the participant, the participantshould be treated as an equal. In short, another cost of objectivity is thatthe participant is treated as an object, instead of in a moral and ethical way.
The Road to Objectivity
Somequalitative researchers charge that "objective methods" are notreally objective because objectivity is impossible. They may attack thescientists’ claim of objectivity by making such mocking comments as,“scientists believe in ‘the immaculate perception’ ofbehavior” and that “scientists believe they have a‘God’s eye view’ of reality.” Qualitative advocatespoint out that objective researchers ignore research showing that humanobservers are not completely objective (Rabinowitz & Weseen, 1997).Furthermore, qualitative proponents argue that no procedure can guaranteeobjectivity. Ultimately, the objectivity of the research lies in the honesty,integrity, and objectivity of the individual researcher rather than in theparticular method used.
Even when the observation itself is objective,scientists still make several subjective decisions and interpretations. Theresearcher decides what to look for, which, in turn, affects what theresearcher will find. In questionnaire research, for example, the researcherchooses to ask certain questions and not others. Furthermore, the researcherinterprets participants’ responses. In questionnaire research, theresearcher assumes the participant understands the questions and interprets thenumber the participant circles on the rating scale as reflecting how theparticipant really feels. That is, there is a leap from the participant’sactual behavior (e.g., circling a few circles on a multiple-choice form) andthe researcher’s description of that behavior (e.g., “the participantis prejudiced”).
To the qualitative researcher, the fact thatquantitative researchers make these subjective decisions is especiallyunsettling because quantitative researchers do not come into the researchprocess with what qualitative researchers consider an open-minded, objectiveattitude. Instead, the quantitative researcher is viewed as trying to findevidence to support a favored hypothesis or theory.
A few qualitative researchers would go further,saying that scientists are trying to support traditional political views.Indeed, it is not uncommon to hear qualitative researchers consider traditionalpsychological research sexist, racist, and “heterosexualist”(Rabinowitz & Weseen, 1997). Some qualitative researchers argue that"objective" science, while outwardly appearing to be objective andfair, is really a tool for oppressing and marginalizing less powerful groups.They do not think that qualitative research oppresses or marginalizes othergroups. Indeed, it is not unusual to hear a Marxist or feminist qualitativeresearcher say that they were drawn to qualitative research because it allowedtheir voice to be heard.
In short, qualitative researchers believe thatquantitative researchers do not go into a research project with an open-minded,objective attitude. Instead, quantitative researchers have some hypothesis theywant to confirm or some political position they want to justify. Thispreconceived idea drives what the how quantitative researchers conduct theirresearch. Giventhat—according to qualitative researchers—quantitative researcherstend to find what they look for, the quantitative researchers’“objective” system fails. The key to objectivity, according toqualitative investigators, is to adopt an objective attitude by going into theresearch without any preconceptions.
Most quantitative researchers strongly disagree withthe qualitative position on how to achieve objectivity. Indeed, they think thequalitative researchers have it backwards.As Whyte (2004, p. 31) points out, it is easy to "dress up prejudice as intuition, faith, mystery, or the limits of science."
According to quantitative researchers, the solutionof “trying to be open-minded” is really the problem. People thinkthey can be open-minded when they cannot. As science has shown, people processinformation automatically and unconsciously according to their preconceptions.Because this processing does not occur at the conscious level, people foolthemselves into thinking that they are being fair and open-minded.
The way to avoid fooling oneself is to use thescientific approach. Over the centuries, scientists have developed a system forovercoming bias. As evidence that the system works, scientists do not findexactly what they expect to find. Indeed, when investigating a new area,scientists are almost as likely to find just the opposite of what they expectto find than to find exactly what they expect to find.
In summary, qualitative researchers believe that we can trust theindividual researcher to be fair, objective, and trustworthy; whereasquantitative researchers believe that the only way we can get trustworthy datais to trust the safeguards that scientists have devised to prevent people fromfooling themselves. Thus, qualitative researchers may trust theindividual’s conscious intentions, whereas quantitative researchers trustthe scientific method. Put another way, quantitative researchers are concernedabout the hidden, unspoken biases of the individual researcher; whereasqualitative researchers are concerned about the hidden, unspoken biases of thescientific community.
The Value of Objectivity
Ratherthan arguing about whether quantitative methods provide objective information,a few qualitative researchers argue that objectivity is not a worthwhile goal.These qualitative researchers do not believe that there is one objectivereality to be found. Instead, the social constructionists believe that realityis an opinion, or as they call it, “a social construction.” Realityis different for everyone. A heterosexual’s reality is different from ahomosexual’s; a chauvinist’s reality is different from afeminist’s, and so on. Given that there is no one reality,everyone’s reality is equally valid. (As Whyte [2004] points out, this means that the sun revolves around the earth for some people but that the earth revolves around the sun for others. Whyte also argues that this view, by putting yourself above reality, is extremely egocentric.) Therefore, the job of psychologistsshould be to make sure that everyone’s voice is heard. We should not letthe voice of the dominant majority drown out other equally valid views ofreality.
The Value ofQuantitative Analysis
Becauseof this view that there is no objective reality, social constructionists do notaccept the idea of objectively analyzing data. Some other qualitativeresearchers also reject quantitative analysis.
Most traditional scientific researchers, on theother hand, want to see what the numbers say. Indeed, to many quantitativeresearchers, the name “qualitative research” suggests that itsadvocates are people who know very little about statistics. Quantitative researchers argue that (a) if you are observing something, you should--at the very least--be able to count how many times that thing occurred, and (b) these counts (indeed, almost any data) can analyzed quantitatively. If you want to find out whether men or women usedifferent words or whether men are more likely to use a certain phrase, you canuse statistics. In fact, you should use statistics. Without statistics, you mayeither (a) see relationships that do not exist or (b) fail to findrelationships that do exist.
To reiterate, if there really is a relationshipbetween variables, we have the mathematical tools to uncover that relationship.If, on the other hand, numerical analysis cannot find the relationship, therelationship is not some “glimmer” in the data— it is in theeye of the beholder.
Although the quantitative researchers’arguments may initially seem compelling, qualitative researchers have strongrebuttals to these arguments. We will now look at the qualitativeresearchers’ rebuttal for each of four main points that quantitativeresearchers make.
1. “Numbers speakfor themselves.” Numbers do not speak for themselves. Numbers alone canbe quite meaningless. Prematurely reducing experience to numbers may prevent,rather than facilitate, understanding. For example, the rush to measureintelligence in terms of numbers occurred before intelligence was properlydefined and understood (Sternberg, 1996). Similarly, numbers may not adequatelyshed light on how it feels to be a person who is discriminated against.Finally, we all know of meaningless and irrelevant quantitative studies thatproduced little more than numbers. Because of the limitations of numbers,non-numerical analyses may be necessary to find out meaningful information andto powerfully communicate findings to others.
2. “Qualitativeresearchers just do not understand statistics.” Rather than beingstatistically illiterate, some of the qualitative researchers, in theirarguments for not using statistics, show a deep understanding of theassumptions underlying statistical tests. Indeed, experts in quantitativeresearch share some of these concerns about the way statistical tests are used.For example, some quantitative researchers often act, when doing statisticalanalyses, like their survey was a random sample of participants—even whenmore than half the people in the original sample refused to participate in thesurvey. Similarly, when conducting hypothesis tests, some quantitativeresearchers act as though they are testing the null hypothesis that there isabsolutely no relationship between their variables. In some cases, theassumption that a treatment, for example, would have absolutely no effect onany participant (or that the positive effects and negative effects wouldbalance out perfectly) is hard to defend (Dillon, 1999). If the hypothesis thatquantitative researchers are trying to disprove is almost certainly false, onecould argue that the quantitative researcher is just using statistics to find whatthey want to find. Finally, in many qualitative studies, samples are small andobservations that are not independent. In such cases, it may not be reasonableto do statistical tests.
3. “Qualitativeresearchers simply ‘eye ball’ the data.” Qualitative researchersuse a variety of techniques to analyze their data. They may make tables,charts, matrixes, concept maps, graphs, and diagrams to summarize the data(Cresswell, 1998; Miles & Huberman, 1994). They may use computer programsto help sort, index, and organize their data. They may even use statisticaltechniques.
4. "Qualitativeresearchers ask the reader to trust that their impression of what the data‘feel like’ is correct.” This is usually not true. Thequalitative researcher knows that the article will be reviewed and read byskeptical peers. Qualitative researchers have to support their claims. Onecommon way that they support their claims is by making all or most of their rawdata available to the reader.
The Importance ofTraining and Standardization.
Aswe suggested in the previous section, some quantitative researchers think thatqualitative researchers are too soft, casual, and informal in analyzing data.Quantitative researchers also think that qualitative researchers are too lax interms of training researchers and in terms of following standardizedprocedures.
Part of this perceived lack of rigor in trainingcomes from the fact that training in qualitative research does not requiretraining in statistics. Part of this perceived lack of rigor comes from theview that the training of qualitative techniques is often very brief.Allegedly, one result of this inadequate training is that qualitativeresearchers do not know how to judge whether a qualitative study is poorly orwell executed, much less to know how to do “good” qualitativeresearch.
Some qualitative researchers share the concern thatit is hard to know whether qualitative research is good or bad (Rabinowitz& Weseen, 1997). They also express some concern that it is unclear aboutwhen the analysis of a qualitative study, such as the intense study of anindividual or a participant observation study, is complete. Some point out thatthe standards of training and quality are better for some qualitative methodsthan for others (Cresswell, 1998). Many qualitative researchers hope that asqualitative methods become more accepted, more formal rules— and moretraining in those rules— will follow (Cresswell, 1998; Tolman &Brydon-Miller, 1997).
Other qualitative researchers respond that trainingin qualitative training is different from, but not less rigorous than, training inquantitative research methods. Still others claim that the “rigor” in quantitative trainingsimply reflects the desire of a power elite to impose initiation rites onwould-be scientists. This rigorous initiation rite serves to (a) restrictaccess to acceptable ways of generating knowledge to members acceptable to theelite, thereby providing a way to exclude people who are oppressed or who havedifferent viewpoints; and to (b) make scientists more committed to both thescientific fraternity and to these traditional methods.
Although some quantitative researchers stronglybelieve that training in qualitative research lacks rigor, most are moreconcerned about the failure of qualitative researchers to follow rigorouslystandardized procedures. The quantitative researcher sees two problems with thelack of standardization. First, without standardizing your procedures, otherscannot replicate your study. Second, because the researcher influences theparticipant, we can only make sense of the participant’s behavior if weknow what the researcher did.
Qualitative researchers respond that it isimpossible to have such hard and fast rules in the complex situations thatqualitative researchers study. Even if you could start out with some hard andfast rules, since your relationship with the participants is evolving, yourresearch question is changing, and you are often interacting with participantsfor a long period of time, you could not follow those rules for the entirestudy. The best that one could do is to (a) come into the situation with nopreconceived ideas and (b) to see if others, who also adopt an open-mindedattitude, obtain similar results.
Not surprisingly, quantitative researchers do notcompletely agree with either of these points. Quantitative researchersvigorously object to the idea that investigators can go into a situationwithout any preconceptions. They point to research showing that peopleautomatically and unconsciously process information according to theirpreconceived notions. If you say you are unbiased, you are either lying orfooling yourself. If you admit that you have biases and claim that you will tryto counter them, you will still fail because much of the bias happens at anonconscious level.
Quantitative researchers are in partial agreementwith qualitative researchers about the value and meaning of replication. Bothqualitative and quantitative advocates agree that if a person with a differentset of biases finds the same results, confidence in the original study'sresults are increased. However, quantitative and qualitative people disagreeabout the meaning of just about any other outcome of a replication. Forexample, suppose that a researcher with a similar set of biases obtains thesame results. In that case, the quantitative researcher would argue that littlehas been proved. Bias, perhaps dueto lack of standardized procedures, could account for the similarity offindings.
If the replication obtains a different set offindings, the quantitative researcher would believe that the credibility of oneor both studies was seriously damaged. The qualitative researcher, on the other hand, might not be soconcerned. The qualitative researcher might explain the failure to replicate bysaying either that (a) there are multiple realities or that (b) each situationis unique and the two investigators are not studying the same exact samesituation.
Dealing with the Complexity of Reality
Notethat some of the conflict between quantitative and qualitative researchersabout standardization might be avoided if qualitative researchers would studyphenomena in simpler, more controlled situations. If qualitative researcherswere to study simpler processes and situations, standardization would beeasier. However, the qualitative researcher thinks that the way to studycomplex phenomena (such as aggression, forgetting, etc.) is to study thesephenomena in natural settings using qualitative means.
The quantitative researcher’s response to the complexity ofpsychological phenomena, on the other hand, is that “the more complex aphenomenon, the greater the need to study it under controlled conditions andthe less it ought to be studied in its natural complexity” (Banaji &Crowder, 1989, p. 1192). As quantitative researchers point out (Banaji &Crowder, 1989), it has been the ability to study complex phenomena undercontrolled conditions that has led to our understanding of phenomena such asobedience, eye-witness memory, helping behavior, and so forth.
Skepticism aboutInternal, External, and Construct Validity
Theargument about simplifying the situation highlights a difference betweenquantitative and qualitative researchers on the three types of validity. Ingeneral, the quantitative researcher thinks the qualitative researcher is notskeptical enough when evaluating the validity of research. The qualitativeresearcher, on the other hand, does not point to these three validities as thestandard for which validity should be judged (Tolman & Brydon-Miller,1997). Indeed, some qualitative researchers argue that thinking about proposedresearch in these scientific terms should be avoided because it will steer theresearch toward being scientific and away from being humane (Denzin, 1989).
Internal Validity
Thequantitative researcher sees the task of identifying the cause of some event asvery difficult, requiring the isolation of potential causes. The qualitativeresearcher thinks the cause can be determined, even in a deeply complex,naturalistic setting. A few qualitative researchers think that the cause can bedetermined simply by asking participants why they are doing the behavior.
ConstructValidity
Similarly,some qualitative researchers are willing to accept the construct validity ofintrospective self-reports. The quantitative researcher, on the other hand,would be skeptical about the validity of any measure, especially self-reportmeasures. For example, the quantitative researcher would like objectiveevidence of the reliability and validity of the self-report measure.
External Validity
Althoughthe qualitative researcher likes to study participants in complex, naturalisticsettings, quantitative researchers still see qualitative researchers asnaïve when it comes to external validity (Banaji & Crowder, 1989). Forexample, they question how a qualitative researcher can generalize from onecomplex naturalistic, real-world context to another, especially since, asqualitative researchers acknowledge, each naturalistic, real world situation isunique?
Quantitative researchers particularly object tothose few qualitative researchers who do try to make generalizations based onstudying only one individual or after extensively interviewing only a fewparticipants. They find such qualitative researchers incredibly naïve.
What Qualitativeand Quantitative Researchers Have in Common
Wehave highlighted the significant differences between people on either side ofthe quantitative-qualitative debate. The differences are so great that manypeople on either side of the debate would not accept the other’sresearch. Many quantitative researchers sneer at focus groups, unstructuredsurveys, and in-depth interviews of a single person. Some quantitativeresearchers view these techniques like inkblot tests: They tell you more aboutthe investigator making the interpretation than they tell you about whatparticipants are really thinking or doing. Others view these techniques astelling cute stories, but not providing real information. Conversely, somequalitative researchers sneer at surveys and experiments as being artificialand shallow. However, as you will see in the next sections, (a) people on bothsides of the debate share some common concerns; and (b) not everyone iscompletely one side or the other of the quantitative-qualitative debate.
Common Concerns
Despitetheir differences, even the most adamant quantitative researcher and the mostadamant qualitative researcher would probably agree on three problems withquantitative research. First, some quantitative research is unimaginative orpoorly done. Second, some quantitative studies would benefit from obtainingricher data. Third, some quantitative analyses are so sophisticated that theyare hard to communicate.
Bothquantitative and qualitative researchers would agree that not all quantitativeresearch is well planned and well executed. For example, as we discussed inChapter 8,
1. Many questionnairesare hastily and poorly written;
2. Some surveys have a 2%response rate; and
3. Some researchers useconvenience rather than random samples.
One approach to the problem of poor quantitative research is to usequalitative research. For example, consider the following scenario proposed bya qualitative researcher:
Quantitative researchersdiscover a correlation between children with behavior problems and those withparents with marital problems. However, those with behavior problems may nothave actually been exposed to their parent’s conflict, but instead werehostile before the conflict arose.
A qualitative researcher would propose a qualitative solution. Forexample, a qualitative researcher might propose intensively interviewing someof the children to find out what was happening.
Solving the problem, however, does not require a qualitative approach.Indeed, intensive interviews may be misleading because children may notaccurately remember whether they themselves were hostile before they learned oftheir parent’s conflict.
The quantitative approach to problem would be to administerquestionnaires several times over a period of years. In this way, we coulddetermine whether parental conflict came before or after the children’shostility.
Bothquantitative and qualitative researchers would agree that some quantitativeresearch should obtain richer, more detailed descriptions of behavior andexperience. The qualitative researcher’s solution is to use qualitativemethods, such as in-depth interviews of individual’s or even reportingthe researcher’s own conscious experience.
The quantitative researcher, on the other hand,would solve the problem by using quantitative methods more imaginatively. Forexample, Baumeister (1995)collected stories from college students about their experiences. In one study,he asked them to write about an event in which they had made someone angry andanother event in which someone had made them angry. Although many of thesestories were fascinating, complex, and rich in detail, the stories could becoded objectively. That is, coders could agree on whether each story had certainfeatures. For example, coders could agree on whether the story contained anapology from the perpetrator. The findings of this study yielded interesting,quantitative data showing, among other things, that “victims describeperpetrators as inconsistent and immoral and rarely mention an apology, whereasperpetrators often say they apologize and made amends and often state theycould not help what happened.”
Finally,both quantitative and qualitative researchers admit that communicating theresults of some quantitative studies is quite difficult. Some qualitativeresearchers see the solution as avoiding statistics and using qualitativestudies in which the raw data are the participants’ own words. Thesequalitative analyses can be quite powerful and moving. Quantitative researchers think that theproblem can be solved without abandoning quantitative methods. Specifically,they envision a two-part solution. First, researchers should use more graphsand more descriptive statistics to help others understand the pattern in theirdata. Second, schools need to do a better job of teaching quantitativeliteracy so that fewer people are alienated from science and more people are empowered to consume and produce research.
Common Methods
Thusfar, we have talked about cases in which quantitative and qualitativeresearchers saw the same problem, but found different solutions. Qualitativeresearchers found qualitative solutions, quantitative researchers foundquantitative solutions. However, as you will see in the next sections,researchers are not always such purists: Some qualitative researchers usequantitative methods, some quantitative researchers use qualitative methods.
QuantitativeAspects of Qualitative Research
Somequalitative researchers, for example, have decided to use quantitative methodsto analyze their data. They see that there are problems with relying on memoryand intuitive statistics to see relationships among variables. Therefore, theyuse computer programs such as NVivo® to link data from such sources asin-depth interviews or participant observation with statistical software. Someof these qualitative researchers see the main difference between their researchand that of quantitative researchers is that their data are richer, moredetailed, more naturalistic, and more unstructured (“messier”) thanthat of quantitative researchers.
Qualitativeaspects of Quantitative Research
Justas some qualitative researchers use quantitative methods, some quantitativeresearchers use qualitative methods. Most quantitative researchers usequalitative methods as exploratory tools. For example, when trying to get a hypothesisfor a study, the quantitative researcher probably will search the literature.In addition, he or she may do some qualitative research to get some startingpoint. She may reflect on her own experiences in similar situations, interviewsome friends, or informally observe some people who are similar to the groupshe wants to study. In short, the quantitative researcher may use qualitativeresearch to get research ideas.
Beforeconducting a full-blown study, many researchers conduct a pilot study. Thispilot study may involve having one’s friends participate in the study andasking them about their impressions of the study. The pilot study may eveninvolve having the researcher take the role of a participant.
Whendevising a questionnaire, one of the first steps that many researchers take isto do some qualitative research. They may ask people who are like those theywant to survey questions, such as
1. What does _____ (theconstruct the researcher plans to measure, e.g., stress) mean to you?
2. How does someone whois _____ (high in the construct) behave differently than someone who is _____(low in the construct)?
Inaddition, the researcher would test out the questions on a sample of people tosee if they understood those questions. Without such qualitative research, thequestionnaire is likely to be meaningless to participants and, consequently,invalid.
Whengiving an oral report of their research, psychologists are likely to reportsome qualitative findings that support the quantitative results. For example,they may make their findings more accessible to the audience by describing howa participant acted, repeating an interesting quote from a participant, or evenshowing video clips of participants' reactions. Researchers may even talk abouttheir own experiences about being in similar situations. In so doing, they areapplying a well-known finding from quantitative research— thatqualitative evidence is more convincing to most people than statisticalevidence.
Finally, we should mention that some well-knownresearch has a qualitative component. From your introductory psychology course,you may recall an interesting study in which the researchers checked themselvesinto mental hospitals to see what the experience was like (Rosenhan, 1973). Onereason that study is so influential is that the participants’ reports offeeling powerless are so moving. Similarly, some of the remarkable findings ofZimbardo’s prison study (Zimbardo, Haney, Banks, & Jaffe, 1975) werebased on the qualitative data about guards abusing prisoners and prisonerssiding with unfairly abusive guards against a prisoner who wanted to be treatedfairly. In short, many landmark studies, including those by Darwin, Freud, andPiaget, have included a qualitative component.
Conclusions
Inconclusion, both qualitative and quantitative research involve serious,systematic scholarship and both can yield insights. The fact that there arepsychologists who advocate qualitative methods highlights some problems withhow traditional research is conducted and communicated. It is quite possiblethat qualitative methods will be more frequently used in psychology. In fact,some methods that were once considered to be “qualitative,” such assome forms of content analysis, are now considered traditional researchmethods.
However, as we have discussed and as you can see from Table 1 (below), there are substantial differences between thetwo methods. The differences stem primarily from the fact thatqualitative methods are not objective. Consequently, the non-objective evidencethat qualitative researchers consider "data" (themes that theinvestigator senses, the investigator's subjective impressions, etc.) is notwhat quantitative researchers consider "data." Given the strongtradition of objective methods in psychology (Burman, 1997; Porter, 1998) and the success of quantitativemethods in psychology, it is unlikely that qualitative methods will replacequantitative methods. The methods you have learned in this course should serveyou and psychology well into the future.
Table1
QualitativeMethods in the Context of Principles Taught in Selected Chapters of ResearchDesign Explained
| Chapter | Key Principle Challenged | Key Principle Agreed With | Comments |
| 1 | Objective methods are productive. | Publicly sharing findings is important. Replication is very important. Ethical considerations are all important. | Qualitative researchers tend to view themselves as more moral and ethical because they tend not to manipulate variables, they never deceive participants, and they treat participants as equals rather than as objects. |
| 2 | Research should be designed to test a specific hypothesis | Literature review and relating one’s investigation to theory are vital elements in many types of qualitative research. | To the qualitative researcher, being open-minded, rather than wed to a hypothesis, is very important. The hypothesis may actually change in the middle of the study. |
| 4 | Try to eliminate the role of the human observer and beware of participant biases, especially in self-report measures. | Trying to reduce bias and get converging evidence that impressions are correct. | The human observer is the most valuable measuring instrument the qualitative researcher has. |
| 8 | Establishing cause and effect is difficult. However, control of variables may make it easier. | | In qualitative research, establishing cause-effect is often not a goal. |
| 9 | Statistical considerations should affect almost every aspect of the design process. | | In qualitative research, (a) small samples are legitimate, and (b) statistics may or many not be used. |
References
Brydon-Miller, M. & Tolman,D. L. (1997). Engaging the process of transformation. Journal of Social Issues, 53, 803-810.
Burman, E. Minding the gap: Positivism, psychology,and the politics of qualitative methods. Journal of Social Issues, 53, 785-802.
Creswell, J. W. (1998). Qualitative inquiry andresearch design: Choosing among five traditions. Thousand Oaks, CA: Sage.
Denzin, N. K. (1989). Interpretive interactionism. Newbury Park, CA: Sage.
Dillon, K. M. (1999). I am 95% confident that theearth is round: An interview about statistics with Chris Spatz. Teaching ofPsychology, 26, 232-234.
Mason, J. (1996). Qualitative researching. Thousand Oaks, CA: Sage.
Miles, M. B. & Huberman, A. M. (1994). Qualitativedata analysis: An expanded sourcebook. Thousand Oaks, CA: Sage.
Rabinowitz, V. C. & Weseen, S. (1997).Elucidating epistemological impasses: Re-Viewing the qualitative-quantitativedebates in psychology. Journal of Social Issues, 53, 605-630.
Seechrest, L. & Sidani, S. (1995). Quantitativeand qualitative methods: Is there an alternative? Evaluation and ProgramPlanning, 18, 77-87.
Tolman, D. L. & Brydon-Miller, M. (1997).Transforming psychology: Interpretative and participatory researchmethods. Journal of SocialIssues, 53, 597-603.
Whyte, J. (2004). Crimes against logic. New York: MicGraw-Hill.