By Omer Ali

‘Natural Experiments of History’ is a scholarly book with an ambitious aim: to coax historians into using the comparative method, and to encourage an explicitly quantitative approach to history – explicit because the editors argue that historians already make quantitative comparisons when they write statements such as ‘a kingdom grew more prosperous’ or ‘trade between two societies decreased over time’ but rarely present the numerical data on which such comparisons are based.

Furthermore, the authors argue that the comparative method is the only way to draw causal links. Studying one society, event or time period provides no reference point for the variation in the outcomes observed and hence the researcher is limited to speculative arguments about the causal origin of these phenomena. When comparing two or more events however, a researcher is better placed to identify cause and effect

Co-editor Jared Diamond (author of ‘Guns, Germs and Steel’ among other titles) is renowned for his interdisciplinary research. Despite occupying an office in UCLA’s geography department, his academic research has spanned evolutionary biology, the humanities and the social sciences. He is certainly not an academic who shies away from crossing boundaries in search of an answer.

Despite the valiant effort however, both students and academics alike are unlikely to jump at the offer of abandoning familiar methodologies and terminology and to wander into new territory. He is accompanied in this endeavour by another celebrated social scientist.

His co-editor, James Robinson, based at Harvard’s Department of Government is equally well placed to call for an approach to history that disregards territorial lines between academic disciplines. He co-authored ‘Economic Origins of Dictatorship and Democracy’, which won the Woodrow Wilson foundation award in 2007. That book summarizes a decade of joint research with Daron Acemoglu of MIT on the behaviour of dictatorships and the mechanisms of democratization. It challenged the methodological approach to questions formerly confined to political science departments and practically unveiled a new field of research in economics, hence redefining the boundary between the two disciplines.

Scientists frequently use the laboratory to identify causal relationships. The experimenter can control the environment and the variables under consideration, hence tailoring the experiment to directly address questions of interest. However, when the experiment cannot be confined to a laboratory, field experiments are used instead.

The John Bates Clark medal, awarded each year to the most promising economist in the US under the age of forty, was recently awarded to MIT economist Esther Duflo whose field experiments have put many questions to rest. One of those experiments concluded that promising to give a bag of rice to mothers who brought their children to immunization centers in rural communities was the most effective way of increasing the immunization rate. Another found that the intervention that improved the performance of Kenyan school children most was deworming, which dwarfed the effect of distributing textbooks to schools.

The morality of some field experiments however is questionable. In one experiment attempting to find the effectiveness of malarial bed nets, some villages were selected to receive them while others were used as the ‘control’ group. Does the randomness (and hence somewhat arbitrariness) of the selection justify giving these bed nets to only some? If not, does the high quality data generated by the field experiment and the causal link between the policy and the outcome that these data are hoped to elicit justify temporary preferential treatment?

Furthermore, when it comes to collecting data on historical events, and especially the kind that we don’t want to see repeated, the conduct of a field experiment is out of the question: consider a historian attempting to understand the phenomenon of genocide by conducting an experimental one! The approach is also appalling in medical sciences. In order to determine whether a particular gene is disease-resistant, a researcher cannot simply collect a sample of humans, some with the gene, others without it, inject them with the disease to then check which ones survived.

Through a careful selection of historical cases however, the coveted ideal experimental conditions can be approximated. The ’natural experiment’ is used to find causal relationships when the researcher simply cannot conduct their own experiment. It involves comparing the outcome of different systems whose environments are similar in most respects, but differ in the factor of interest.

Under such stringent conditions, through observing the outcomes of the various systems, a researcher is sure that the differences in the outcome are a result of differences in the factor of interest as opposed to the environment. Essentially, these are experiments ‘conducted by nature’ and the challenge for the researcher is to find them and then study them in detail. The book contains a number of interesting examples of natural experiments.

There is a selection of 8 such studies in 7 chapters by 11 authors. The studies range from the comparison of two nations, Haiti and the Dominican Republic occupying the Caribbean island of Hispaniola, to comparing the public good provision in 233 districts in India. What the studies have in common is that they tease out conclusions about causal links through observing unique events in the past that make these links easier to identify.

The first chapter, by Patrick Kirch, an anthropologist based at the University of California in Berkeley, presents a comparison of three Polynesian island societies at different stages of development. The author devotes a large portion of the chapter to a description of the methodology which can be tedious for those uninterested in the specifics, rigorous comparison of the three societies reveals remarkable parallels as well as sharp contrasts.

During a period of expansion, when the population increased to unsustainable levels and resources became scarce, tension and competition between traditional rulers, priests and warriors intensified. It is argued that human sacrifice rituals became more important due to their suitability as a policy (to justify aggression towards other groups) rather than due to any spiritual significance. This lends weight to the position that cultural and religious practices have their foundations in economic realities.

This position is sometimes overlooked in contemporary analysis of events. Often, religion and culture are presented as the reasons why nations, communities or individuals behave the way they do, whereas underlying economic considerations may have featured more prominently in their reasoning than did cultural or religious ones.

James Belich of Victoria University in New Zealand explores the phenomena of booms and busts in a chapter that demonstrates the striking frequency and similarity of economic cycles in modern societies across the globe. At least five consecutive booms and busts occurred in the American West by the First World War. The author compares the American West with the British West (Canada, Australia and New Zealand) and the Russian ‘Wild East’ (Siberia) finding commonalities.

However, the analysis in this chapter is less clinical than the rest of the book. Sweeping statements manage to make it into some of the arguments. The economic analysis is also superficial: when describing post bust export-led recoveries, the author frames an increase in export volumes as “a way to compensate for the lower price” disregarding the endogenous nature of prices; low prices stimulate demand which, in times of excess capacity (as a result of the bust), is matched by an increase in supply instead of an increase in prices.

In the third chapter, Stanford University’s Stephen Haber conducts a politico-economic analysis of the development of the banking system in the United States, Mexico and Brazil. Through a careful historical study, the author traces the evolution of banking institutions to political dynamics, thereby arguing that reforms are neither deliberate nor easy to predict and that economic and political institutions are deeply intertwined.

The author argues that the unique nature of the US’s nineteenth century banking system was a consequence (perhaps unintended) of the political institutions set up by the US constitution. The relative autonomy of individual states spawned regulatory competition, and hence banking institutions improved much faster than even in Great Britain, the most advanced economy at the time.

The absence of such competition resulted in stunted reforms in both Mexico and Brazil where business interests and powerful politicians were able to influence regulation to their advantage. By observing the different outcomes in the three cases, the author is able to ascribe changes in the banking system to specific political developments conclusively.

In the fourth chapter, Diamond presents two case studies. The first examines the Caribbean island of Hispaniola, shared by Haiti to the west and the Dominican Republic to the east. The second examines what caused the extinction of the Polynesian Easter Islanders.

The first study presents a case that is remarkably close to the ideal natural experiment. Two countries occupy the same island and yet experience divergent outcomes: today, Haiti is one of the poorest nations on earth, while the Dominican Republic enjoys modest economic success with an average per capita income six times that of its neighbour.

The eastern part of Hispaniola (modern day Dominican Republic) was first claimed by Spain in the late fifteenth century. The western part of the island first served as a pirate dock for British, French and Dutch pirates seeking to intercept Spanish trading ships, and was then occupied by France in the late eighteenth century. Slaves were brought to the island, and within less than ten years, French occupied Hispaniola’s population was 85% slaves dwarfing the eastern part’s 10-15%.

Once slaves were transported to the western part of the island, French slave ships carried timber as they returned to Europe. This had a devastating effect on the island’s ecology and was a precipitating factor of the deforestation that followed. The slave population in Haiti developed Haitian Creole, a language not spoken anywhere else, while Dominicans spoke Spanish, a language that facilitated trade with other Spanish speaking countries later on.

Eventually Haiti’s slaves rebelled, fought the French armies stationed there, and killed the remaining whites. They ransacked farms and developed a deep suspicion of Europeans. These growing differences were further heightened by the policies of two dictators, Trujillo in the Dominican Republic and ‘Papa Doc’ Duvalier in Haiti. Trujillo, who sought to grow rich by extracting rents from businesses, encouraged economic activity such as exports of raw materials and sustainable forestry.

Duvalier on the other hand did not have such an interest. Instead, the majority of his attention was occupied with political survival – purging potential rivals and tightening his control of society at the expense of institutions that encouraged economic activity. This difference is attributed to a ‘long standing interest in exports and foreign trade’ in the Dominican Republic and Haiti’s ‘long standing hostility to foreign trade’ – these are insufficient explanations and the differences in policies more likely stem from deeper political-economic differences that made one dictator favour one set of policies while the other favoured another (Robinson’s aforementioned book deals beautifully with these questions).

The second study presented in this chapter investigates the mysterious collapse of Polynesian society in Easter Island. The issue is commonly discussed yet the explanations that are advanced seem to lay the cause of the Easter Islanders’ extinction on the islanders themselves.

It is commonly argued that short sightedness, or a maniacal preoccupation with the erection of statues led to the islanders unsustainably felling trees, resulting in deforestation, war and eventual extinction. A statistical analysis including data on Easter, as well as 68 other Polynesian islands, of environmental factors that influence rates of deforestation finds that Easter was one of the most environmentally fragile islands in the Pacific.

Indeed Easter Islanders were simply unlucky to find themselves on such a fragile ecosystem. Nine variables were identified by the analysis as important. Some, like temperature and rainfall, were expected to be important in advance, but others, like the proximity of islands to volcanoes that released wind-borne ash as opposed to lava, were not. Such a conclusion would have been impossible to arrive at had the authors instead limited the analysis to a comparison of Easter Island with another island where the population flourished.

Harvard economic historian Nathan Nunn explains how Africa’s slave trades precipitated its modern day impoverishment in the book’s fifth chapter. The argument splits into three steps: establishing the statistical relationship between slave trading and lower incomes today; determining the causal link between the two; identifying the mechanisms through which this causality works.

The second step ascertains that it is, in fact, the slave trade that causes lower incomes today, rather than areas that were historically underdeveloped trading more slaves and remaining till the present relatively poorer than other areas. Through meticulous and determined data work covering the Atlantic, trans-Saharan and Indian Ocean slave trades, the author concludes that parts of Africa that supplied more slaves do, indeed, have lower incomes today.

A statistical relationship alone can be interpreted in a number of ways: one could argue that parts of Africa that were already relatively underdeveloped supplied slaves, and hence these areas are prone to economic failure; therefore, the slave trade didn’t influence their fate. The author rules this out, arguing that, initially, Europeans were looking for trading partners in Africa and invariably chose developed African societies where social sophistication enabled the production of trade-worthy goods.

The example of Portuguese traders is given; they sailed down the west coast of Africa looking for such societies until they found the Kongo Kingdom. It was long after a relationship between these African societies and European traders was established that they were asked to supply slaves.

The author goes on to claim that more developed areas were more populated since primitive societies were in a ‘Malthusian trap’: potential increases in the standard of living were readily translated into a swelling of the population before the improvement could even materialise. This hypothesis is corroborated by the author’s use of population estimates of areas that traded slaves to find that areas that supplied a higher number of slaves were indeed more populous.

The key function of controlled experiments (whether they be laboratory or field experiments) is to determine causal relationships. Thus far, two reasonable arguments for the same statistical relationship can be put forth: one in which the economic backwardness of a region made its inhabitants easy prey for slave traders, hence contributing more slaves than economically advanced regions, while their historic economic disadvantage persisted till the present day with no causal role played by the slave trade; another (which the author advances) where the slave trade resulted in the impoverishment of initially prosperous regions. In other words, does relatively low income at the time of the slave trade explain the volume of slaves traded? Or does the volume of slaves traded explain low incomes today?

The volume of slaves traded and contemporary income are correlated across regions; to determine the direction of causality, a common statistical tool called an ‘instrumental variable’ is used. To show that a variation in the volume of slaves traded causes a change in income across regions (and not the opposite), a third variable, namely the distance of the regions of slave extraction from slave ports is used as an instrument.

A crucial characteristic of this third variable is that, while explaining the volume of slaves traded (the closer a region was to a slave port, the more slaves were extracted from it), it should not explain contemporary income. Since the distance from port is not related to income today in any way, variation in the distance will only influence the volume of slaves traded, not incomes. If we find a relationship between the distance from port and contemporary incomes then it must be the case that this relationship comes about through the volume of slaves traded. We rule out any reverse causality between incomes and the volume of slaves traded by using distance from port as an instrument in this way.

Although the accuracy of the data is questionable, there is no reason to believe that this inaccuracy is biased towards the author’s conclusion. Exemplary dataset construction is demonstrated when the author proceeds to trace slaves to their areas of origin. Ethnographic data is used from slave records, and this is superimposed with data on the geographical concentrations of ethnic groups at the time the slaves were captured.

The study effortlessly cuts across disciplines fluently handling statistical subtleties as well as anthropological nuances. This makes it a prime example of the work advocated by the editors – namely scholarly work that pursues questions of importance by any means or methodology necessary, regardless of the discipline that houses them.

Why was it the case that slave trading precipitated lower incomes today? The author proposes that the slave trade fostered a general environment of insecurity; in turn, this did not support the development of large stable states that would provide the institutional foundations for modern growth – namely ethnic and linguistic homogeneity, specialization and the development of commerce.

Indeed, slaves were captured not only in wars but also in cross village raids, sometimes between villages with extant trading partnerships. Not surprisingly, these relationships disintegrated; a period during which local bandits wielded power over a frightened population ensued. This fractured governance structure hampered the development of large, stable states – during the period in question, only one such state is identified in the historical record.

While it advances a convincing argument for the impact of slavery on today’s economic prosperity, the study also reverses the common intuition that the slave trade relied on the abundance of small, weak communities for a supply of slaves – instead, it was the more developed, organized and populous societies that provided the most slaves. A comparison with another similar historical case is missing however. Although the study is careful, all its data are from the same continent, which leaves the door open to continent-specific characteristics being influential in the impact of slavery on modern economic outcomes.

In chapter six, Abhijit Banerjee and Lakeshmi Iyer, of MIT and Harvard Business School respectively, study the effect of colonial land tenure systems on contemporary public good provision in India. Three types of land tenure systems, landlord-based, individual cultivator-based, and village-based, were imposed by the British administration on different districts in India.

The choice of tenure system was based on ideological preferences in England rather than on characteristics of the districts in question; it thus could be considered independent of local conditions. They conclude that there are large differences in the development trajectories of areas with different land tenure systems, particularly between areas that experienced landlord-based tenure systems and those that didn’t. The provision of public goods such as education, health and infrastructure is, today, much lower in the former. The study provides yet another example of how historic institutions can influence contemporary realities.

In the 7th chapter, a panel of four authors that includes co-editor James Robinson isolates the causal relationship between legal institutions and economic development. They argue that feudal institutions retarded economic development in Europe – a claim that was advanced by many, including Engels, who is quoted at the beginning of the chapter. However, the question is not straightforward to tackle since the interaction between legal institutions and economic outcomes is rather complex.

It is reasonable to consider that causality could run both ways, and so the authors’ claims cannot be verified through a statistical relationship that, in this case, does not give the direction of causation. With the use of a similar approach to the one in the previous chapter, they find an element that is external to local institutions and examine its effect on economic outcomes. In this case, the element is the spread of the Napoleonic armies across Europe. Once captured, Napoleonic law was to have jurisdiction on the territories: feudal institutions were abolished and legal reforms, including equality before the law, were introduced.

By looking at those parts of Europe that were invaded by French armies and those that weren’t, the causality of this institutional reform can be evaluated. The authors find that economic prosperity in the period that followed was persistently higher in those territories that were invaded than those that weren’t. Hence, they conclude that legal reforms unambiguously influence economic outcomes.

The final chapter, an afterword by the editors, takes a look at the methodological challenges of the natural experiment. It ends with an invitation for historians to use this method more widely. Indeed, it is difficult to imagine causal links being drawn conclusively without recourse to the comparative method.

The cases presented in this book are ample evidence that the quantitative analysis of history is not only useful, but also necessary if academics are to make conclusions that are causal, and hence go beyond well-reasoned speculation. The book issues a warning that seeking conclusions about causal relationships through the study of isolated cases is unlikely to yield results, independently of how detailed these studies may be.

It is only when the academic is able to draw comparisons between societies, and across time periods, that they are able to extract deeper insights about causality. The natural experiment has been in use for at least a decade, yet this is the first attempt to formalize and popularize it as a method of historical analysis. The book is likely to have a major impact on the way historical analysis is conducted.

Natural Experiments of History
Edited by Jared Diamond and James A. Robinson
Belknap Press of Harvard University Press

288 pages

Omer Ali is an economist based at the University of Warwick and writes on economics, politics and world affairs. He is a former editor of the Voice Magazine. His “Devil’s Advocate” column appears every other Thursday.

(This review benefitted extensively from comments and suggestions by Guillaume Sublet and George Stefanidis.)