The EITM Approach: Origins and Interpretations

John Aldrich,(Duke University),  James Alt(Harvard University),  Arthur Lupia(University of Michigan)

  This version: May 10, 2007

 

Many scholars seek a coherent approach to evaluating information and converting it into useful and effective knowledge. The term “EITM” (“Empirical Implications of Theoretical Models”) refers to such an approach. EITM’s initial public use was as the summarizing title of a workshop convened by the National Science Foundation’s Political Science program in July 2001. The workshop’s goal was to improve graduate and postgraduate training in the discipline in light of the continued growth of an uncomfortable schism between talented theoreticians and innovative empiricists. Since then, the acronym has been applied to a growing range of activities such as summer institutes and scholarship programs. At the time of this writing, the acronym’s most common use is as an adjective that describes a distinct scholarly approach in political science. This essay explains the approach’s origins and various ways in which NSF’s call to EITM action has been interpreted.

Since the late 1960’s, a growing number of political scientists turned to increasingly rigorous methods of inference to improve the precision and credibility of their work. Theorists, through the development of a new and intricate class of formal mathematical models, and empiricists, through the development of a new and dynamic class of empirical estimators and experimental designs, made many important contributions to Political Science over the decades that followed. The precision with which more rigorous theorists could characterize conditional logical relationships amongst a range of factors and the consequent rigor with which empiricists could characterize relations between variables, generated exciting new research agendas and led to the toppling of many previously focal conjectures about cause and effect in politics.

The evolution of game theoretic, statistical, computational, and experimental forms of inference increased the transparency and replicability of the processes by which political scientists generated causal explanations. In so doing, these efforts revolutionized the discipline and increased its credibility in many important domains. One such domain was the hallways of the National Science Foundation.

At the same time, a problem was recognized.

The problem was that the growing sophistication in theory and method were proceeding all too often independently of one another. Many graduate students who obtained advanced training in formal modeling had little or no understanding of the mechanics of empirical inference. Similarly, graduate programs that offered advanced training in statistical methods or experiments often offered little or no training in formal modeling. Few formal theorists and rigorous empiricists shared much in the way of common knowledge about research methods or substantive matters. As a result, they tended not to communicate at all.

There were exceptions, of course. And over the years, intermittent arguments were made about the unrealized potential inferential value that might come from a closer integration of rigorous theorizing and empiricism. But few scholars had the training to capitalize on such opportunities.

The challenge that NSF put to participants in the EITM workshop was to find ways to close the growing chasm between leading edge formal theory and innovative empirical methodology. Participants were asked to find ways to improve the value and relevance of theoretical work by crafting it in ways that made key aspects of new models more amenable to empirical evaluation. They were also asked to find ways to use methods of empirical inference to generate more effective and informative characterizations of focal model attributes.

The ideas generated at and after this workshop sparked an aggressive response by the National Science Foundation. Since then, NSF Political Science has spent – and continues to spend -- a significant fraction of its research budget on EITM activities. The most notable of these activities are its month-long summer institutes which have been held at places like Berkeley, Duke, Harvard, Michigan, UCLA, and Washington University and which have served hundreds of young scholars.

As a result of these and related activities, EITM training now occurs at a growing number of universities around the world. This training not only educates students about highly-technical theoretical and empirical work but encourages them to develop research designs that provide more precise and dynamic answers to critical scientific questions by integrating the two approaches from the very beginning of the work. Most importantly, perhaps, there is a critical mass of scholars (particularly in the discipline’s junior ranks), who recognize how this approach can improve the reliability and credibility of their work. New and vibrant scholarly networks are forming around the premise that the EITM approach can be a valuable attribute of research.

EITM, however, has not been without controversy. Some saw it as unnecessary contending that the theory-empirical gap was natural in origin and impossible to overcome. Others feared that EITM would lead students to have false ideas and expectations about the extent to which formal theories could be evaluated empirically. And, in a particularly infamous example, the American Journal of Political Science briefly instituted a policy saying that it would not consider for review articles containing a formal model that did not also include empirical work – with part of the journal’s explanation being that they believed such a practice was consistent with the goals of EITM. The AJPS policy was widely attacked, opposed by leaders of the EITM movement, and quickly reversed by the journal. However, the episode demonstrated that several years into the endeavor there was still considerable uncertainty about what EITM was and its implications for the future of graduate training and scholarly work in Political Science. No one disputes that a logically coherent, rigorously tested explanation is desirable, but “what logic” and “which tests” is less obvious.

In what follows, we make a brief attempt to explain why the EITM approach emerged, why it is valuable, and how it is currently understood. With respect to the final point, we contend that EITM has been interpreted in multiple ways. We highlight a subset of extant interpretations and, in the process, offer our own views about the most constructive way forward.

Developments in Political Science: 1960s to 1990s

Political Science is a discipline that shares important attributes with the other sciences. For example, many political scientists are driven by the motive not only to understand relevant phenomena, but also to provide insights that can help humanity more effectively adapt to important challenges. Consequently, many political scientists desire to have their conclusions and arguments attended to and acted upon by others. As a result, many political scientists believe that arguments backed by strong evidence and transparent logic tend to be more credible and persuasive than arguments that lack such attributes, all else constant.

What distinguishes political science from other disciplines is that it focuses on politics. In other words, political scientists are unified by a context, and not by a method. Political science departments typically house faculty with training from a range of traditions. It is not uncommon to see faculty with backgrounds in fields such as economics, journalism, philosophy, psychology or sociology. As a result, political science is not defined by agreement on which method of inference is best.

For this reason, political scientists tend to be self-conscious about the methods they use (see Thomas 2005 for a review of five recent texts). They must be, because they cannot expect to walk into a broadly attended gathering of colleague-scientists and expect an a priori consensus on whether a particular method is optimal – or even suitable – for the effective examination of a particular problem. Indeed, compared to microeconomics or social psychology, where dominant inferential paradigms exist, consensus on the best way to approach a problem in any comparable subset of political science, including the study of American Politics, is rare. Methodological diversity runs deep. So scholars must, as part of their presentational strategies, be able to present a cogent argument as to why the approach they are using is necessary or sufficient to provide needed insights.

This attribute of political science has many important implications. One is that scholars who are trained in a particular research methodology tend to have limited knowledge of other methods. In the case of scholars who use formal models, scholars who use advanced statistical methods, scholars trained in experimental design and inference, and scholars trained in computational models (each of which became far more numerous in political science over the final three decades of the twentieth century), the knowledge gaps with respect to methods other than their own followed the prevailing pattern. However, given the training in logic and inference that scholars must acquire to be competent in formal modeling, experimentation, or the brand of statistics that became known as political methodology, an inconvenient truth existed alongside of the groups’ intellectual isolation. The truth was that insufficient interaction between theory and empirics yields irrelevant deductions and false empirical inferences:

Empirical observation, in the absence of a theoretical base, is at best descriptive. It tells one what happened, but not why it has the pattern one perceives. Theoretical analysis, in the absence of empirical testing, has a framework more noteworthy for its logical or mathematical elegance than for its utility in generating insights into the real world. The first exercise has been described as "data dredging," the second as building "elegant models of irrelevant universes." My purpose is to try to understand what I believe to be a problem of major importance. This understanding cannot be achieved merely by observation, nor can it be attained by the manipulation of abstract symbols. Real insight can be gained only by their combination.” (Aldrich 1980)

While many scholars would likely agree with such statements, acting on them was another matter. In the early days of formal modeling in political science, the barrier to closer interactions between formal modeling and political methodology was a lack of “know how.”

Many classic papers of the early formal modeling period recognized the importance of empirical-theoretical integration, but the tools did not exist to allow these scholars to engage in such activities with any rigor. Consider two examples from the first generation of what were widely known as “rational choice” theories.

The decision to turnout to the vote and the decision to run for (higher) office were initially formulated in decision theoretic terms (e.g., Riker and Ordeshook, 1968, McKelvey and Ordeshook, 1972; and Black, 1972, Rohde, 1979, respectively). Using decision theory meant that they were formulated as decisions made without consideration of strategic interaction. Only later were they able to modeled in true game theoretic settings (e.g., by Palfrey and Rosenthal, 1985 and Banks and Kiewiet, 1989, respectively). The early modelers did not make that choice because they believed there was no significant strategic interaction (which they may or may not have believed in the first case but certainly did not in the second). Rather, they assumed a decision theoretic context for at least two reasons. First, that was the technology available for use in addressing those problems. Second, even this technology was sufficient to allow the authors to the discipline’s stock of theoretically-generated knowledge in important ways.

Even when game theory replaced decision theory in these early applications, the models were “primitive” in the sense that they often assumed complete and perfect information. An often unrecognized consequence of these assumptions was that individual behavior was assumed to be deterministic. In retrospect, we can ask why “rational choice” theorists modeled people in these simplistic and evidently empirically false ways. But if we look at the resources that such scholars had available – logical constructs from fields of logic, mathematics, philosophy, and microeconomic theory -- there were no better alternatives available for the early formal modeler. For all of the 1960’s and most of the 1970’s, there was no decision-theoretic or game theoretic technology that permitted more realistic representations of human perception and cognition. Not until the work of Nobel laureates such as John Harsanyi and Reinhardt Selten was there a widely-accepted logical architecture for dealing with incomplete information, asymmetric information, and related problems of perception and cognition.

Even if theory had been able to advance to the Harsanyi/Selten level years or decades earlier, it is not clear how one would have integrated their mechanics or insights into the statistical estimators of the day. Indeed, these early theorists, including William Riker, Richard McKelvey, and Peter Ordeshook ran experiments as a means of evaluating certain aspects of their models empirically (e.g., Riker, 1967; McKelvey and Ordeshook, 1980, 1982). But such examples were rare and not statistically complex. Rarer still were attempts to evaluate such model attributes using statistical models with an underlying parallel logic sufficient for rigorous evaluation. Intricate estimation of choice models was only beginning to be developed.

McFadden (1977) developed the first model for estimation of discrete choice problems in the early to mid-1970s (something for which he also won a Nobel Prize). His and subsequent work provided rigorous ways to integrate statistical inferences for the testing of hypotheses derived from decision theory. These early statistical models, though, did not extend to testing hypotheses about strategic behavior as derived from game theory. Subsequent advances of this kind in statistical choice models took longer to develop (Aldrich and Alt 2003). Thus, through at least the 1970s, it is arguable that neither game theory nor statistical methodology had the technology to close the gap between the two in any rigorous manner.

In the 1980s and 1990s, theory and method had advanced. Not only were political scientists importing inferential advances from other fields, but they were also creating their own advances – tools and instruments that were of particular value to, and specifically attuned to the special problems associated with the context of politics. To contribute to such enterprises, an increasing number of faculty and graduate students in political science sought and obtained more intense technical training. While these advances and graduate training trends led to many positive changes in political science, they also fueled the gap between formal theorists and rigorous empiricists. Certainly, widespread perception of such cause-and-effect was the impetus behind NSF’s decision to begin investing in an EITM approach in the 1990s (see, e.g., Brady 2004; Smith 2004; Granato and Scioli 2004).

By the mid 1990s it was possible to point to research agendas that powerfully leveraged the advantages of leading edge formal modeling and empirical work. There were formal-model-oriented experiments of various kinds (see, e.g., Druckman, et. al. (2006) for a review) though the number of such efforts remained miniscule in comparison to the number of models produced. There were also notable successes in attempts to leverage the inferential power of high end game theory and statistical methods such as McKelvey and Palfrey’s (1992, 1995) Quantal Response Equilibrium concept and Signorino’s (1999) empirical estimation strategy that embedded that equilibrium concept directly into the derivation of therefore-appropriate estimators. However, sparse examples does not a full approach make. As Aldrich and Alt (2003) emphasized, political science articles were less likely than their economics counterparts to connect theory and empirics. But the underlying problem was deeper: theorists and empirical researchers were not reading, citing, or building upon each others’ work.

For the most part, the intellectual streams of formal theorists and rigorous empiricists in the discipline were not converging. Theorists tended to come from and gather at a few focal universities, beginning with the graduate program William Riker founded at the University of Rochester in the 1960s, then extending to departments that housed political scientists and highly-technical economic theorists under one roof (such as the interdisciplinary programs at Carnegie Mellon, Cal Tech and Stanford’s Graduate School of Business) and then to departments that employed graduates of such programs. These scholars formed vibrant intellectual communities and could publish in outlets such as Public Choice, the Journal of Political Theory and the Journal of Law, Economics and Organization (as well as parallel journals in economics) without paying much attention to empirical evaluations of the new models’ key attributes.

At roughly the same time that theorists from Rochester and universities began to make waves, the 1960s saw an explosion of interest in quantitative data by Political Scientists while the 1970s saw the spread of training in econometrics. The subsequent founding of the Society for Political Methodology, the development of its methods summer conference, and the emergence of the journal Political Analysis (from its origin as Political Methodology), decisively shifted the standard graduate curriculum in political science toward rigorous inference and the development and use of more appropriate, powerful, and flexible tools, along with more powerful, computer-intensive estimation methods. Later, political scientists with training in experimental design followed suit, designing and conducting a growing range of experiments that were particularly pertinent to key questions in the discipline. Critical masses in departments such as Cal Tech, Minnesota, Yale, and Stony Brook have transformed graduate training in those places through their emphasis on experimental design.

But theorists and empiricists tended to lead parallel, rather than integrated existences.

By the 1990s, a political science formal theory with its own identity and distinct accomplishments had clearly developed as had a parallel political science statistics. The success of formal modeling and rigorous empirical work in challenging long-held maxims about causality led to increasing demand for training in these areas. But many departments chose to specialize in one method of inference (with training in statistics being more common than training in formal modeling and both being more common than experimental design). While these departments became hotbeds for many valuable scholarly advances, they also fueled increasing specialization. These dynamics also produced increased isolation. Scholars with advanced inferential skills tended to congregate with people like them to the exclusion of others.

Here is one example of this excessive specialization, though we believe there are many. Austen-Smith modeled the income distribution and political choice of taxes under proportional representation in comparison with a first-past-the-post system. Among the implications of his model is that in comparison with a two-party system, legislative bargaining in a three-party PR system results in equilibrium tax rates that are higher, with more equal post-tax income distributions, lower aggregate income, and higher unemployment (Austen-Smith 2000, 1236).

These results are consistent (and inconsistent) with a range of non-formal political science conjectures about the size and consequences of the welfare state, as well as insider-outsider politics (e.g. Rueda 2005). Model features like parties representing specific groups in the labor market or labor market choices responding to tax rates are not unrelated to measures present in the empirical analysis of redistribution. One would think that Austen-Smith’s model could inform and advance empirical work. The narrative is clearly argued and the logic expertly presented. At a minimum, the conditional relationships he identifies suggest a variety of modeling changes and evaluating the robustness of causal claims made in the empirical literature. In our view, nothing about the model itself explains why a scholar.google count showed that, from his working paper until the appearance of Iversen and Soskice (2006) last year, a total of five political scientists had cited this paper, none doing empirical evaluation of its conjectures. A generation-long norm of isolation amongst rigorous theorists and empiricists is a more plausible explanation.

The growth of logical rigor – in formal theoretic and statistical contexts -- since the 1960s has served political science well. The discipline has seen much progress in theory development and testing. But questions also arose such as, “How much more can be learned if bridges are built, so that young scholars can be systematically trained to use theoretical and empirical methods as complements rather than substitutes?” and “How can we integrate rigorous and coherent hypothesis generation with equally rigorous inference that reflects both the integrity and coherence of the theory and the information within and problems associated with the political process generating the data?” The answers to these questions are the challenge of EITM.

Interpretations of the EITM Approach

Of what value does an EITM approach add, beyond an exhortation to integrate credible theory and rigorous empirical evaluation? A narrow construction emphasizes the major task of science as testing the causal implications of theories against reality. But many suggest views of the research program that are much broader. In this section we present five views of EITM, some that have been presented as self-conscious attempts to formalize boundaries for the approach and two (by Christopher Achen and Richard McKelvey, respectively) that offer important insights despite predating the NSF workshop by years or decades. In the process, we offer some commentary on these various presentations as a means of advancing our own views on the matter.

We begin by illustrating three views drawn from a discussion at the EITM Summer Institute held at Berkeley in 2005. This discussion was led by James Fearon, Henry Brady, and Gary Cox and yielded three views of EITM that are commonly encountered and go beyond the narrow construction.

View 1. Observing reality and developing theoretical explanations for its causal regularities is just as important as testing those causal implications.

View 2. Science requires the development and testing of concepts and measurement methods as much as the development of causal theories.

View 3. A division of labor between deriving and testing of implications must exist, but that division risks over-specialization in which the opportunities of fruitful interactions between theory and empirical work are missed.

As to view 1, rather than a narrow perspective that emphasizes going only from theory to new implications and testing them, going in the opposite direction at the same time (observing reality, developing intuitions and ideas, and positing explanations and theories) is equally important. It is difficult to object to this idea when stated in such generality. But treating the two approaches (theory first, observation first) symmetrically is also problematic. The reason is that scientifically legitimate interpretive claims (i.e., claims whose logic are, at least in principle, transparent and replicable) about the meaning (and any attempt at the labeling) of an observation are impossible in the absence of a categorization scheme that is itself housed in a theory of conceptual relations. In other words, it is hard to credibly claim to observe a causal regularity without an underlying theory.

View 2 is premised on the belief that developing concepts and measurements for theoretical ideas is just as important as developing explanations. Research design develops a test of a theoretical explanation. To do so, the researcher must focus on the empirical relationships between multiple operationalizations of these concepts that grapple with various complexities of the real world. In this account, teaching the conduct of research should be further broadened to consist of both testing the validity of causal relationships and the validity of concepts and measurements.

Finally, in View 3, the relative emphasis upon empirical and theoretical work in any one project depends upon both the “production function” for a valuable research paper and the skills of the researcher. Consider three research production functions. “Either one [derivation or testing] alone is enough.” That is pure maximization where the accumulation of knowledge is assumed to be simple and additive. Second, “Only as good as the weakest link.” This case is rather like a bivariate interaction or, perhaps, like a Leontief production function. But how are we to know whether the problem, when something goes wrong, is due to the elegant model being of an irrelevant universe or due to the data having been dredged too indiscriminately? Thirdly, knowledge is generated by a “Linear production with interactive complementarities.” In this view, combining empirical and theoretical work is the best way to advance science because it leads to more than the sum of its parts, though empirical or theoretical work alone also produces great value. Because science is a social process and because people have differing skills, gains from specialization are possible. The danger of specialization, of course, is that nobody will combine theory and method because insufficient numbers of scholars will know how to do so, or how to evaluate others attempts to do so. The belief of many observers in the late 1990’s was that political science suffered from an excess of specialization. It was for this reason that the original EITM conference was called.

Achen’s Approach

To the extent that such views reflect the EITM approach, they are not new or unique in political science. In particular, they echo arguments offered amongst rigorous empiricists decades earlier. For example, Bartels and Brady (1993: 148) argued that in the field of political methodology “there is still far too much data analysis without formal theory – and far too much formal theory without data analysis.”

Achen (2002b) set up the problem in the following way, and suggested a way to bring the two together. Starting from a critique of contemporary statistical work

Too many of the new estimators in political methodology are justified solely because they are one conceivable way to take account of some special feature of the data…. [M]any techniques are available, and … choosing the right one requires consulting the quantitatively sophisticated researchers in a given field…. (436-7)

 

He reached the conclusion that

For all our hard work, we have yet to give most of our new statistical procedures legitimate theoretical microfoundations, and we have had difficulty with the real task of quantitative work – the discovery of reliable empirical generalizations. (424)

There were important differences between older and newer styles of quantitative analysis:

The old style … is content with purely statistical derivations from substantively unjustified assumptions. The modern style insists on formal theory. The old style dumps its specification problems into a strangely distributed disturbance term and tries to model the [result]; the new style insists on starting from a formal model plus white noise errors. The old style thinks that if we try two or three familiar estimators… each with some arbitrary list of linear explanatory variables …and one …fits better, then it is … right …. The modern style insists that, just because one … fit is better, that does not make them … coherent …. Instead a new estimator should be adopted only when formal theory supports it … (440-1)

His conclusion was (p. 439) “That is what microfoundations are for.”

But in Achen’s view there was still a problem of appropriate domain.

…often the causal patterns are dramatically different across the cases. In those instances, subsetting the sample and doing the statistical analysis separately for each distinct causal pattern is critical. Happily, these causally homogeneous samples will need far fewer control variables…. (447)

So not only does quantitative analysis need analogues (measurable devices) that represent formal and empirical concepts, but one might also recommend separating the observations into two or more sorts of theoretically-generated sub-samples with modest controls in each: the extra controls are what separate the sub-samples. One can take the model from the formal theory side and add detail on the conditionalities (or on Achen’s interactions) or take the statistical model and add some detail on what you don’t know. Either way you get enough structure to the model to simplify the theory but still generate expectations and you get enough specification to look for instruments to address identification and endogeneity. Or at least that should be so on enough important cases to make this approach worthwhile.

A related approach to an EITM-based empirical methodology is exemplified by Snyder and Ting (2003), who start from a situation where there are well-established empirical regularities observed over large numbers of previous studies. They develop an account of parties that allows scientific knowledge to cumulate as they find a “minimal” theoretical model that for the first time allows the analyst to deduce all the stylized facts, then additionally derive (at least) one new empirical implication and examine the evidence. Similarly, in a comparative theory building exercise, Achen (2002a) specified a Bayesian version of the retrospective account of partisanship and showed that one could derive from it many of the same empirical implications that were hallmarks of the social psychological theory of party identification.

How big is the gain in understanding what is theoretically at stake when an empirical regularity is accompanied by a carefully worked-out micro-model? Consider how Carroll and Cox (2007) outline the evolution of recent work on Gamson’s Law, the proposition that parties forming a coalition government each get a share of portfolios proportional to the seats that each contributed to the coalition (Gamson 1961). Empirically corroborated by a long list of scholars, Gamson’s Law conflicts with standard bargaining theories (Baron and Ferejohn 1989; Morelli 1999 and others), in which a party’s ability to pivot between alternative minimal winning coalitions and its ability to propose governments determine its portfolio payoff. Laboratory experiments that confront Gamson’s Law these bargaining models (see Diermeier and Morton 2004 or Fréchette, Kagel, and Morelli 2005) broadly support the bargaining models. But subsequently Warwick and Druckman again find empirical support for Gamson but believe it “in acute need of a firm theoretical foundation” (2006, 660). Therefore Carroll and Cox specify a theoretical reason why portfolios should be handed out in proportion to seat contributions (but only by assuming that some parties can make binding commitments to one another) and thus provide an empirical model that not only encompasses Gamson as well as bargaining models, but also yields a refinement on earlier results as “the first to find [that] the constant term is nil and the slope term on seats is unity” and even produces an entirely new result “that governments based on publicly announced pre-election pacts will allocate portfolios more in proportion to seat contributions than will other governments (2007, 310). This represents an extraordinary increase in being able to examine “what obtains if …” in empirical work, compared to the earlier recording of underspecified correlations.

McKelvey’s Approach

As we have argued elsewhere (Aldrich, Alt, and Lupia 2007a), no one personified the EITM approach in political science earlier and more effectively than Richard McKelvey. McKelvey is the rare figure who is known to formal modelers as an innovative and seminal figure in the development of their approach in political science and is known to many empirical scholars – principally through the advances presented in McKelvey and Zavoina (1975) and Aldrich and McKelvey (1976) – as an important figure in the advance of rigorous statistical techniques. More importantly, however, he regularly developed theoretical models alongside of means for evaluating key aspects of them simultaneously. Since he never published a book length manuscript, this tendency in his work was difficult to derive from his articles, which did not advertise this point, but a new book length treatment of his research (Aldrich, Alt, and Lupia 2007b) verifies the pattern.

McKelvey’s approach to EITM was different than those described above. A good way to motivate his approach is to begin with a critique of formal theory by Sanchez-Cuenca. His critique highlights attributes of the method that make many non-formal theorists uneasy:

The introduction of rational choice theory in political science [produces] … greater rigor, greater clarity, and the promise of a unified language for the discipline. Yet, rational choice has produced theories that are either far removed from reality or simply implausible, if not false. The problem has to do with the explanatory role of preferences. In economic analysis, the market induces preferences: the entrepreneur maximizes profits and the consumer, given her preferences, chooses according to the price system. But out of the market it is more difficult to define the preferences of the actors.

This critique echoes McKelvey’s advice to the EITM Workshop (EITM Report, 2002). He argued that, because there is no equivalent in political science to general equilibrium theory in economics, there is no central core from which to launch political analysis and perform comparative statics or other stock-in-trade economic methods of analysis. In the ordinary absence of general political equilibrium, he said, theory must incorporate specific details of the situation.

Sanchez-Cuenca also points out that while the consumer is decisive (that is the consumer obtains what she chooses) the voter is non-decisive, since her vote is one among many and her vote rarely if ever is pivotal in determining the outcome. The result is that many non-instrumental considerations may enter into her decision, and no single easy numeraire like money serves to induce voter preferences. On the politicians’ side, preferences may be induced, rather like producers’ preferences are. However, there are different bases of induction that may apply, resulting in different types of politicians’ preferences (are they seeking to capture political rents, win votes, or create the politician’s sense of good public policies?) and, perhaps produce different equilibria. This argument is reminiscent of McKelvey’s view of institution-based modeling: standard non-cooperative game theory models typically found in the study of institutional politics will have equilibria of the Nash or Nash-with-refinements sort, generally very many of them, each applicable to specific circumstances with limited general applicability elsewhere.

McKelvey thought that this state-of-affairs would create a role for “applied theory.” Empirical modelers might not be able to find a model that they could just plug in and use on any given set of data. He opined that they might have to develop some of the theory themselves, opening up opportunities for those researchers who can “speak both languages” at the empirical and theoretical ends of the spectrum to find common ground in the development of specific models that successfully integrate theoretical and data generating processes for superior testing.

Conclusion

The observable implications of formal models, once derived, invite empirical evaluation. The current NSF-sponsored EITM initiative involves a strong commitment to causal inference in service to causal reasoning. At its broadest, it emphasizes addressing well-defined problems with some combination of clearly-stated premises, logically coherent theory explaining relations among the premises, and logically coherent empirical work developed to evaluate the premises and/or the relations. Theory is broader than game theory and methods are broader than inference from statistical models and controlled experiments. In addition to game theory, formal models include differential equation dynamic models, simple decision theory, more complicated behavioral decision-making models, and computational models. The empirical toolkit should include not only statistical inference and experiments but also focused analytically-based case studies and computational models.

In the above, we may all be in general agreement. However, we further believe that theory is not independent of empirical investigation. There are so many choices that must be made – questions to ask, parameters to consider, and so on – that a theorist operating in some way independent of understanding and investigating the empirics of the problem at hand will, as a general rule, make incorrect and sterile choices of how to model. But, at the same time, estimation requires micro-foundations and thus a strong sense of the theory of the problem – the richer that sense the better. In this way, the separation of derivation and testing is false and there is an iterative and integrative process. As a fringe benefit to this view, the prospect that the theorist or the methodologist is somehow favored as the “more important” contributor is reduced. Indeed, it may well be the empirical scholar who bridges the work of theorist and methodologist who is favored.

The idea of EITM is to bring deduction and induction, hypothesis generation and hypothesis testing close together. That is not quite the same thing as assuming that an individual should be good in all aspects. The challenge is to be able to structure training programs so that theory, methods, and substantive people know enough about each part to be able to work together well. Each has to be able to develop serious competency in (at least) one thing. The required combination of all of these parts is similar to the challenges of creating an effective interdisciplinary research team – each has their own specialized language and technologies, and they must collectively negotiate a common language. Fortunately, it is easier because we share a common context of politics, but only by doing the comparable negotiations can we take that knowledge out to the research frontier.

As a starting point for such negotiations, we close by offering a variant of a list of questions recently posed my de Marchi (2005). He suggests that an appropriate way to begin an effective strategy for evaluating its empirical implications is to ask a series of questions. Questions such as:

What are the assumptions / parameters of the model? Do the assumptions spring from a consideration of the problem itself, or are they unrelated to the main logic of the model, chosen arbitrarily, perhaps solely to make derivations possible? Are values chosen for parameters derived from qualitative or quantitative empirical research, or are they chosen arbitrarily, maybe for convenience? How many of the parameters are open, to be “filled in” by data analysis?

How sure can we be that the main results of the model are immune to small perturbations of the parameters? That is, is there an equivalence class where the model yields the same results for a neighborhood around the chosen parameters? Alternatively, is the model “brittle,” and if so, in what dimensions (that is, on the basis of perturbations of which parameters? How central are these particularly brittle parameters to the theoretical questions at hand?

Do the results of the model map directly to a dependent variable, or is the author of the model making analogies from the model to an empirical referent? While “toy” models have their place in developing intuition, they are difficult to falsify, and even more difficult to build upon in a cumulative fashion. de Marchi defines “toy” models as a class of simple models without any unique empirical referent. For example, the iterated prisoner’s dilemma (IPD) is a simple game that investigates cooperation. He argues that it seems unlikely the all or most of human cooperation is a two player contest with the exact strategy set of the IPD, and there is enormous difficulty in analogizing from the IPD to actual human behavior with enough precision to do any sort of predictive work.

Are the results of the model verified by out-of-sample tests? Are there alternatives to a large-N statistical approach that tests the model directly? Or is the parameter space of the model too large to span with the available data – this is the “curse of dimensionality?” Can one alleviate the impact of this curse by deriving a domain-specific encoding? Or is the specification of limited spheres of relevance just another arbitrary way to make our models appear more complete than they are?

For many people trained in formal modeling or statistics, such questions can be uncomfortable. But our discipline has already embarked on dual processes of inquiry into the development of rigorous theory and empiricism. So the questions, while seemingly inconvenient, will not go away. Nor should they, because underneath the sometimes arcane content of such methodological questions are issues that pertain directly to the substantive relevance of a particular theory or to the substantive generality of an empirical claim. As a discipline, we benefit from being able to offer a growing range of logically-coherent and empirically robust explanations of focal political phenomena. Dealing with such questions in an aggressive and straightforward manner is a means to new levels of credibility and influence for political science. Developing a generation of scholars who are up to such challenges is the main goal of the EITM approach and is the reason that the approach merits broad and continuing support.