6. Bridging the Quantitative-Qualitative Divide

Sidney Tarrow

In Designing Social Inquiry (hereafter KKV), Gary King, Robert O. Keohane, and Sidney Verba have performed a real service to qualitative researchers. I, for one, will not complain if I never again have to look into the uncomprehending eyes of first-year graduate students when I enjoin them—in deference to Przeworski and Teune—to ‘‘turn proper names into variables.’’ The book is brief and lucidly argued and avoids the weighty, muscle-bound pronouncements that are often studded onto the pages of methodological manuals.

 

But following KKV’s injunction that ‘‘a slightly more complicated theory will explain vastly more of the world’’ (105), I will praise the book no more, but focus on an important weakness in the book: KKV’s central argument is that the same logic that is ‘‘explicated and formalized clearly in discussions of quantitative research methods’’ underlies—or should—the best qualitative research (3). If this is so, then the authors really ought to have paid more attention to the relations between quantitative and qualitative approaches and what a rigorous use of the latter can offer quantifiers. While they offer a good deal of generous (if at times patronizing) advice to qualitatively oriented scholars, they say very little about how qualitative approaches can be combined with quantitative research. Especially with the growth of choice-theoretic approaches, whose practitioners often illustrate their theories with narrative, there is a need for a set of ground rules on how to make intelligent use of qualitative data.

 

KKV does not address this issue. Rather, it uses the model of quantitative research to advise qualitative researchers on how best to approximate good models of descriptive and causal inference. (Increasing the number of observations is its cardinal operational rule.) But in today’s social science world, how many social scientists can simply be labeled ‘‘qualitative’’ or ‘‘quantitative’’? How often, for example, do we find support for sophisticated game-theoretic models resting on the use of anecdotal reports or on secondary evidence lifted from one or two qualitative sources? More and more frequently in today’s social science practice, quantitative and qualitative data are interlarded within the same study. In what follows, I will discuss some of the problems of combining qualitative and quantitative data, as well as some solutions to these problems.

 

CHALLENGES OF COMBINING QUALITATIVE AND QUANTITATIVE DATA

 

A recent work that KKV warmly praises illustrates both that its distinction between quantitative and qualitative researchers is too schematic and that we need to think more seriously about the interaction of the two kinds of data. In Robert Putnam’s (1993) analysis of Italy’s creation of a regional layer of government, Making Democracy Work, countless elite and mass surveys and ingenious quantitative measures of regional performance are arrayed for a twenty-year period of regional development. On top of this, he conducted detailed case studies of the politics of six Italian regions, gaining, in the process, what KKV (quoting Putnam) recommends as ‘‘an intimate knowledge of the internal political maneuvering and personalities that have animated regional politics over the last two decades’’ (5) and what Putnam calls ‘‘marinating yourself in the data’’ (KKV: 5; Putnam 1993: 190). KKV (38) uses Making Democracy Work to praise the virtues of ‘‘soaking and poking,’’ in the best Fenno (1977: 884) tradition.

 

But Putnam’s debt to qualitative approaches is much deeper and more problematic than this; after spending two decades administering surveys to elites and citizens in the best Michigan mode, he was left with the task of explaining the sources of the vast differences he had found between Italy’s northcentral and southern regions. In his effort to find them, his quantitative evidence offered only indirect help, and he turned to history, repairing to the halls of Oxford, where he delved deep into the Italian past to fashion a provocative interpretation of the superior performance of northern Italian regional governments vis-à-vis southern ones. This he based on the civic traditions of the (northern) Renaissance city-states, which, according to him, provided ‘‘social capital’’ that is lacking in the traditions of the South (chap. 5). A turn to qualitative history—probably not even in Putnam’s mind when he designed the project—was used to interpret cross-sectional, contemporary quantitative findings.

 

Putnam’s procedure in Making Democracy Work pinpoints a question in melding quantitative and qualitative approaches that KKV’s canons of good scientific practice do not help to resolve. In delving into the qualitative data of history to explain our quantitative findings, by what rules can we choose the period of history that is most relevant to our problem? What kind of history are we to use; the traditional history of kings and communes or the history of the everyday culture of the little people? And how can the effect of a particular historical period be separated from that of the periods that precede or follow it? In the case of Making Democracy Work, for example, it would have been interesting to know by what rules of inference Putnam chose the Renaissance as determining the Italian North’s late twentiethcentury civic superiority. Why not look to its sixteenth-century collapse faced by more robust monarchies, its nineteenth-century military conquest of the South, or its 1919–21 generation of Fascism (not to mention its 1980s corruption-fed pattern of economic growth)? None of these are exactly ‘‘civic’’ phenomena; by what rules of evidence are they less relevant in ‘‘explaining’’ the northern regions’ civic superiority over the South than the period of the Renaissance city-states? Putnam doesn’t tell us; nor does KKV.

 

To generalize from the problem of Putnam’s book, qualitative researchers have much to learn from the model of quantitative research. But quantitative cousins who wish to profit from conjoining their findings with qualitative sources need, for the selection of qualitative data and the intersection of the two types, rules just as demanding as the rules put forward by KKV for qualitative research on its own. I shall sketch some useful tools for bridging the quantitative-qualitative divide from recent examples of comparative and international research (see table 6.1).

 

Table 6.1. Tools for Bridging the Qualitative-Quantitative Divide

Tool	Contribution to Bridging the Divide
Process Tracing	Qualitative analysis focused on processes of change within cases may uncover the causal mechanisms that underlie quantitative findings.
Focus on Tipping Points	Qualitative analysis can explain turning points in quantitative time series and changes over time in causal patterns established with quantitative data.
Typicality of Qualitative Inferences Established by Quantitative Comparison	Close qualitative analysis of a given set of cases provides  leverage for causal inference, and quantitative analysis then serves to establish the representativeness of these cases.
Quantitative Data as Point of Departure for Qualitative Research	A quantitative data set serves as the starting point for framing a study that is primarily qualitative.
Sequencing of Qualitative and Quantitative Studies	Across multiple research projects in a given literature, researchers move between qualitative and quantitative analysis, retesting and expanding on previous findings.
Triangulation	Within a single research project, the combination of qualitative and quantitative data increases inferential leverage.

 

TOOLS FOR BRIDGING THE DIVIDE

 

Tracing Processes to Interpret Decisions

 

One such tool that KKV cites favorably is the practice of process tracing in which ‘‘the researcher looks closely at ‘the decision process by which various initial conditions are translated into outcomes’ ’’ (226; quoting George and McKeown 1985: 35). KKV interprets the advantages of process tracing narrowly, assimilating it to their favorite goal of increasing the number of theoretically relevant observations (227). As George and McKeown actually conceived it, the goal of process tracing was not to increase the number of discrete decision stages and aggregate them into a larger number of data points but to connect the phases of the policy process and enable the investigator to identify the reasons for the emergence of a particular decision through the dynamic of events (George and McKeown 1985: 34–41).

 

Process tracing is different in kind from observation accumulation and is best employed in conjunction with it—as was the case, for example, in the study of cooperation on economic sanctions by Lisa Martin (1992) that KKV cites so favorably.

 

Systematic and Nonsystematic Variable Discrimination

 

KKV gives us a second example of the uses of qualitative data but, once again, underestimates its particularity. The authors argue that the variance between different phenomena ‘‘can be conceptualized as arising from two separate elements: systematic and nonsystematic differences,’’ the former more relevant to fashioning generalizations than the latter (56). For example, in the case of Conservative voting in Britain, systematic differences include such factors as the properties of the district, while unsystematic differences could include the weather or a flu epidemic at the time of the election. ‘‘Had the 1979 British elections occurred during a flu epidemic that swept through working-class houses but tended to spare the rich,’’ the authors conclude, ‘‘our observations might be rather poor measures of underlying Conservative strength’’ (56–57).

 

Right they are, but this piece of folk wisdom hardly exhausts the importance of nonsystematic variables in the interpretation of quantitative data. A good example comes from how the meaning and extension of the strike changed as systems of institutionalized industrial relations developed in the nineteenth century. At its origins, the strike was spontaneous, uninstitutionalized and often accompanied by whole-community ‘‘turnouts.’’ As unions developed and governments recognized workers’ rights, the strike broadened to whole sectors of industry, became an institutional accompaniment to industrial relations, and lost its link to community collective action. The systematic result of this change was permanently to affect the patterns of strike activity. Quantitative researchers like Michelle Perrot (1986) documented this change. But had she regarded it only as a case of ‘‘nonsystematic variance’’ and discarded it from her model, as KKV proposes, Perrot might well have misinterpreted the changes in the form and incidence of the strike rate. Because she was as good a historian as she was a social scientist, she retained it as a crucial change that transformed the relations between strike incidence and industrial relations.

 

To put this point more abstractly, distinct historical events often serve as the tipping points that explain the shifts in an interrupted time-series, permanently affecting the relations between the variables (Griffin 1992). Qualitative research that turns up ‘‘nonsystematic variables’’ is often the best way to uncover such tipping points. Quantitative research can then be reorganized around the shifts in variable interaction that such tipping points signal. In other words, the function of qualitative research is not only, as KKV seems to argue, to peel away layers of unsystematic fluff from the hard core of systematic variables; but also to assist researchers in understanding shifts in the values of the systematic variables.

 

Framing Qualitative Research within Quantitative Profiles

 

The uses of qualitative data described in the two previous sections pertain largely to aiding quantitative research. But this is not the only way in which social scientists can combine quantitative and qualitative approaches. Another is to focus on the qualitative data, using a systematic quantitative database as a frame within which the qualitative analysis is carried out. Case studies have been validly criticized as often being based on dramatic but frequently unrepresentative cases. Studies of successful social revolutions often focus on characteristics that may also be present in unsuccessful revolutions, rebellions, riots, and ordinary cycles of protest (Tilly 1993: 12– 14). In the absence of an adequate sample of revolutionary episodes, no one can ascribe particular characteristics to a particular class of collective action.

 

The representativeness of qualitative research can never be wholly assured until the cases become so numerous that the analysis comes to resemble quantitative research (at which point the qualitative research risks losing its particular properties of depth, richness, and process tracing). But framing it within a quantitative database makes it possible to avoid generalizing on the occasional ‘‘great event’’ and points to less dramatic—but cumulative—historical trends.

 

Scholars working in the ‘‘collective action event history’’ tradition have used this double strategy with success. For example, in his 1993 study of over 700 revolutionary events in over 500 years of European history, Charles Tilly assembled data that could have allowed him to engage in a large-N study of the correlates and causes of revolution. Tilly knows how to handle large time-series data sets as well as anybody. However, he did not believe the concept of revolution had the monolithic quality that other social scientists had assigned to it (1993: chap. 1). Therefore, he resisted the temptation for quantification, using his database, instead, to frame a series of regional time-series narratives that depended as much on his knowledge of European history as on the data themselves. When a problem cried out for systematic quantitative analysis (e.g., when it came to periodizing nationalism), Tilly (1994) was happy to exploit the quantitative potential of the data. But the quantitative data served mainly as a frame for qualitative analysis of representative regional and temporal revolutionary episodes and series of episodes.

 

Putting Qualitative Flesh on Quantitative Bones

 

An American sociologist, Doug McAdam, has shown how social science can be enriched by carrying out a sustained qualitative analysis of what is initially a quantitative database. McAdam’s 1988 study of Mississippi Freedom Summer participants was based on a treasure-trove of quantifiable data—the original questionnaires of the prospective Freedom Summer volunteers. While some of these young people eventually stayed home, others went south to register voters, teach in ‘‘freedom schools,’’ and risk the dangers of Ku Klux Klan violence. Two decades later, both the volunteers and the no-shows could be interviewed by a researcher with the energy and the imagination to go beyond the use of canned data banks.

 

McAdam’s main analytic strategy was to carry out a paired comparison between the questionnaires of the participants and the stay-at-homes and to interview a sample of the former in their current lives. This systematic comparison formed the analytical spine of the study and of a series of technical papers. Except for a table or two in each chapter, the texture of Freedom Summer is overwhelmingly qualitative. McAdam draws on his interviews with former participants, as well as on secondary analysis of other people’s work, to get inside the Freedom Summer experience and to highlight the effects that participation had on their careers and ideologies and their lives since 1964. With this combination of quantitative and qualitative approaches, he was able to tease a convincing picture of the effects of Freedom Summer activism from his data.

 

As I write this, I imagine KKV exclaiming, ‘‘But this is precisely the direction we would like to see qualitative research moving—toward expanding the number of observations and re-specifying hypotheses to allow them to be tested on different units!’’ (see chap. 7). But would they argue, as I do, that it is the combination of quantitative and qualitative methods trained on the same problem (not a move toward the logic of quantitative analysis alone) that is desirable? Two more ways of combining these two logics illustrate my intent.

 

Sequencing Quantitative and Qualitative Research

 

The growth industry of qualitative case studies that followed the 1980–81 Solidarity movement in Poland largely took as given the idea that Polish intellectuals had the most important responsibility for the birth and ideology of this popular movement. There was scattered evidence for this propulsive role of the intellectuals; but since most of the books that appeared after the events were written by them or by their foreign friends, an observer bias might have been operating to inflate their importance in the movement vis-à-vis the workers who were at the heart of collective action in 1980–81 and whose voice was less articulate.

 

Solid quantitative evidence came to the rescue. In a sharp attack on the ‘‘intellectualist’’ interpretation and backed by quantitative evidence from the strike demands of the workers themselves, Roman Laba demonstrated that their demands were overwhelmingly oriented toward trade union issues, and showed little or no effect of the proselytizing that Polish intellectuals had supposedly been doing among the workers of the Baltic coast since 1970 (1991: chap. 8). This finding dovetailed with Laba’s own qualitative analysis of the development of the workers’ movement in the 1970s and downplayed the role of the Warsaw intellectuals, which had been emphasized in a series of books by their foreign friends.

 

The response of those who had formulated the intellectualist interpretation of Solidarity was predictably indignant. But there were also more measured responses that shed new light on the issue. For example, prodded by Laba’s empirical evidence of worker self-socialization, Jan Kubik returned to the issue with both a sharper analytical focus and better qualitative evidence than the earlier intellectualist theorists had employed, criticizing Laba’s conceptualization of class and reinterpreting the creation of Solidarity as ‘‘a multistranded and complicated social entity . . . created by the contributions of various people’’ whose role and importance he proceeded to demonstrate (1994: 230–38). Moral: a sequence of contributions using different kinds of evidence led to a clearer and more nuanced understanding of the role of different social formations in the world’s first successful confrontation with state socialism.

 

Triangulation

 

I have left for last the research strategy that I think best embodies the strategy of combining quantitative and qualitative methods—the triangulation of different methods on the same problem. Triangulation is particularly appropriate in cases in which quantitative data are partial and qualitative investigation is obstructed by political conditions. For example, Valerie Bunce used both case methodology and quantitative analysis to examine the policy effects of leadership rotation in western and socialist systems. In her Do New Leaders Make a Difference? she wrote: ‘‘I decided against selecting one of these approaches to the neglect of the other [the better] to test the impact of succession on public policy by employing both methodologies’’ (1981: 39).

 

Triangulation is also appropriate in specifying hypotheses in different ways. Consider the classical Tocquevillian insight that regimes are most susceptible to a political opportunity structure that is partially open. The hypothesis takes shape in two complementary ways: (1) that liberalizing regimes are more susceptible to opposition than either illiberal or liberal ones; and (2) that within the same constellation of political units, opposition is greatest at intermediate levels of political opportunity. Since there is no particular advantage in testing one version of the hypothesis over the other, testing both is optimal (as can be seen in the recent social movement study, Kriesi et al. 1995).

 

My final example of triangulation comes, with apologies, from my own research on collective action and social movements in Italy. In the course of a qualitative reconstruction of a left-wing Catholic ‘‘base community’’ that was active in a popular district of Florence in 1968, I found evidence that linked this movement discursively to the larger cycle of student and worker protest going on in Italy at the same time (Tarrow 1988). Between 1965 and 1968, its members had been politically passive, focusing mainly on neighborhood and educational issues. However, as the worker and student mobilization exploded around it in 1968, their actions became more confrontational, organized around the themes of autonomy and internal democracy that were animating the larger worker and student movements around them.

 

Researchers convinced of their ability to understand political behavior by interpreting ‘‘discourse’’ might have been satisfied with these observations; but I was not. If nothing else, Florence was only one case among potential thousands. And in today’s global society, finding thematic similarity among different movements is no proof of direct diffusion, since many movements around the world select from the same stock of images and frames without the least connection among them (Tarrow 1994: chap. 11).

 

As it happened, quantitative analysis came to the rescue by triangulating on the same problem. For a larger study, I had gathered a large sample of national collective action events for a period that bridged the 1968 Florentine episode. And as it also happened, two Italian researchers had collected reliable data on the total number of religious ‘‘base communities’’ like that in Florence throughout the country (Sciubba and Pace 1976). By reoperationalizing the hypothesis cross-sectionally, I was able to show a reasonably high positive correlation (.426) between the presence of Catholic base communities in various cities and the magnitude of general collective action in each city (Tarrow 1989: 200). Triangulation demonstrated that the findings of my longitudinal, local, and qualitative case study coincided with the results of cross-sectional, national, and quantitative correlations. My inductive hunch that Italy in the 1960s underwent an integrated cycle of protest became a more strongly supported hypothesis.

 

KKV does not take the position that quantification is the answer to all the problems of social science research. But the book’s single-minded focus on the logic of quantitative research (and of a certain kind of quantitative research) leaves underspecified the particular contributions that qualitative approaches make to scientific research, especially when combined with quantitative research. As quantitatively trained researchers shift to choicetheoretic models backed up by illustrative examples (often containing variables with different implicit metrics) the role of qualitative research grows more important. We are no longer at the stage when public choice theorists can get away with demonstrating a theorem with an imaginary aphorism. We need to develop rules for a more systematic use of qualitative evidence in scientific research. Merely wishing that it would behave as a slightly less crisp version of quantitative research will not solve the problem.

 

This is no plea for the veneration of historical uniqueness and no argument for the precedence of ‘‘interpretation’’ over inference. (For an excellent analysis of the first problem, see KKV 42–43; and of the second, see KKV 36–41.) My argument, rather, is that a single-minded adherence to either quantitative or qualitative approaches straightjackets scientific progress. Whenever possible, we should use qualitative data to interpret quantitative findings, to get inside the processes underlying decision outcomes, and to investigate the reasons for the tipping points in historical timeseries. We should also try to use different kinds of evidence together and in sequence and look for ways of triangulating different measures on the same research problem.

 

CONCLUSION

 

KKV gives us a spirited, lucid, and well-balanced primer for training our students in the essential unity of social science work. Faced by the clouds of philosophical relativism and empirical nominalism that have recently blown onto the field of social science, we should be grateful to its authors. But the book’s theoretical effort is marred by the narrowness of its empirical specification of qualitative research and by its lack of attention to the qualitative needs of quantitative social scientists. I am convinced that had a final chapter on combining quantitative and qualitative approaches been written by these authors, its spirit would not have been wildly at variance with what I argue here.