Conventionally, counterfactual analysis has not been endorsed by historians and social scientists. This post addresses some typical objections.

The first objection is that, given the multiplicity of components present in any web—a nearly infinite number of alternatives exist—how does one choose the relevant subset? For example, in Taylor (2005, Chapter 4, section A), why did I stop at multi-objective techniques? Why not analyze farmers’ decision making in terms of, say, neural nets or genetic algorithms? My answer is that, when analysts use counterfactuals to expose resources and to support the corresponding causal claims, they must make the counterfactuals practically plausible. The set of counterfactuals should not include just any conceivable idea. To persuade readers that a counterfactual was practically plausible to the agents, the counterfactual should not be too dissimilar from what actually occurred. Or, more precisely, the counterfactual should, given the cross-linkage of resources, build on most of the same resources as the actual situation.

Now, evaluations of whether resources linked into the altered (counterfactual) context are still the same cannot be neutral, but must be made by, or with reference to, two groups: some agents to whom the alternatives are also practically relevant, and some audience that has to imagine how the agents were acting in the given situation. Using such an evaluation a finite subset of alternatives can, in principle, be delimited in a non-arbitrary way. Hawthorn (1991) provides three detailed cases to illustrate his valuable discussion of the use of counterfactuals in historical explanation.
The need to show practical plausibility of alternatives can lead to a second objection: To do this well one should have a strong picture of what the relevant causes are, yet the very reason for counterfactual analysis is to assess the causal significance of resources. This apparent circularity dissolves, however, if we think of explanation as an iterative process, beginning with causal ideas (see post on Causes) borrowed from situations deemed similar. These ideas are then successively refined and reformulated to address the situation at hand. It is true that iterative methods cannot in general guarantee that one successively approaches a correct account, but such certainty need not be a decisive criterion for good interpretation.

Lynch (1989), following Elster (1978), advocates a different way of resolving the circularity problem, namely, building a more explicit theoretical basis for counterfactuals from other sources, such as the sociology of scientific knowledge. Besides Hawthorn (1991), who eschews generalizing theory, the most developed use of counterfactual scenarios has been in economic history, in which explicit econometric models have been used to examine issues such as the impact of railroads on the U.S. economy in the nineteenth century (Fogel and Engerman 1969). This work takes seriously the idea of going back in time until there is a point at which counterfactuals to railways, e.g., greater development of canals, can be smoothly inserted into the model of the developing U.S. economy (see below). Notwithstanding this virtue, econometric counterfactual analysis does not provide satisfactory exemplars of theory-based counterfactuals. Because econometric scenarios are formulated as sets of regression equations, their “causality” is based on statistical association and is difficult to express in terms of actions of agents. On neither count does this match the view of causality here (see post on Causes).

The specific example of the Kerang Farm Model in Taylor (2005, Chapter 4, section A) presents an easy case for convincing readers that the counterfactuals were practically plausible and relevant. That is because the modeler actually attempted to pursue or envisage the alternatives that I have mentioned earlier. Neural nets and genetic algorithms, on the other hand, were in their very early days of development, in disciplines and places far removed from the economics of the Institute. However, we should not make too much of the special insight the modeler provided. Counterfactual analysis must be possible regradless of whether the agents have attempted to implement alternatives. Because of this, the Kerang study and the modeler’s engagement should not be taken as the exemplar of how to expose and identify alternatives. Observations that were more sociologically distanced or systematic than the modeler provided would help make the analysis something others could borrow from. To make systematic choices among the multiplicity of counterfactuals, it helps to be prepared to expose and develop one’s own agenda for engagement. In formulating the set of eight contrasts in Taylor (2005, Chapter 4, section B), my interest in participatory rather than technocratic approaches to socio-environmental studies came into play. This interest made it clear why I discounted Cockrum’s alternative to Picardi’s system dynamics modeling, and instead examined alternatives in which the pastoralists’ situation became less system-like.

Finally, some observations can be made on why interpreters of science might want to depart from the naturalistic approach of staying close to the scientist’s vantage point. As mentioned earlier, counterfactual analyses are strongly shaped by the positioning of the sociological explainers and their audience. A counterfactual that the audience at hand does not consider practically plausible for the relevant agents in its time can sometimes be suitably modified. By moving back in time, to another place, or among different people, we can render it into something practically plausible for the new agents. (But we may need to move our audience also, or move in front of a different audience, in order to do so.) To help explain the Kerang study, we could ask whether a different modeler, say, one more senior and more experienced in multi-objective techniques, could have made a different model “right for the job.” This would mean going back to the time when the project was formulated, or even further, to the time when other scientists were devising the courses on modeling that went into the training of this different modeler. A shift in the counterfactuals and their point of insertion—the time, place, focal agents, and audience—would also be required if we want to test a hypothesis, say, that policies formed through studies conducted in-house by a government agency are more likely to be taken up than policies formed through research contracted out to university researchers. Clearly, it is a challenging task to define and delimit counterfactuals when agents are building on webs of heterogeneous resources. This task is a facet of the broader challenge of analyzing the composition, shape, and structure of those webs.

Excerpted from the Notes section of Taylor, Peter J. 2005. Unruly Complexity: Ecology, Interpretation, Engagement. U. Chicago Press.

References
Elster, J. (1978). Logic and Society: Contradiction and Possible Worlds. New York: Wiley.
Fogel, R. W. and S. L. Engerman (1969). “A model for the explanation of industrial expansion during the nineteenth century: With an application to the American iron industry.” Journal of Political Economy 77: 306-328.
Hawthorn, G. (1991). Plausible Worlds. Cambridge: Cambridge University Press.
Lynch, W. T. (1989). “Arguments for a non-Whiggish hindsight: Counterfactuals and the sociology of knowledge.” Social Epistemology 3(4): 361-365.