> Catherine de Fontenay CASE #59 - Multiculturalism economics

Research on trial

December 03, 2020

Research on trial

The 2019 Nobel Prize in Economics was awarded to Abhijit Banerjee, Esther Duflo, and Michael Kremer for their ground-breaking work in introducing randomised-controlled trials (RCTs) into the social sciences. RCTs were well-known in medicine: for a new medication to be approved, several clinical trials must take place. Participants are randomly assigned to receive either the new medication, or a placebo. The medication is proven effective if those receiving the medication improve more on average than those with the placebo, and the participant group is large enough.

Their insight was that a number of important economic and social policies can be tested in a similar way. If we think it’s valuable to make visual aids such as flip-charts available to schools in Africa, then we can randomly distribute flip-charts to some schools, and the remaining schools serve as a control. Using this technique, flip-charts were shown to have no impact on student learning.[1] Prior to using RCTs, the school performance of schools with flip-charts would be compared to the school performance of schools without flip charts. But even when comparing like with like (e.g. low-income schools with other low income schools), there is a selection effect: if I see a school that has chosen to try out flip charts, it has probably done so because someone in the school is motivated to try new solutions. A motivated school will be more successful than an un-motivated one, whether or not flip-charts have any impact. Thus we would falsely conclude that flip-charts were effective. There are statistical techniques to help deal with selection problems, but they are not nearly as reliable as RCTs.

Banerjee, Duflo and Kremer have led a revolution in Development Economics, in which RCT techniques are widely used. Using volunteers to assist students in learning basic mathematics and reading has hugely improved literacy and numeracy in India, and in partnership with a charity called Pratham, has already assisted 5 million Indian students.[2] Using regular teachers (whose performance is dismal) to undertake similar teaching in summer schools, with similar results, showed that the problem was teachers’ perceptions and incentives. Most Indian teachers believe that they are meant to be teaching to a select few at the top of the class.[3] Providing a payment to low-income families for sending their children to school has been rolled out in tens of countries.[4] There are strong positive effects on school attendance and reductions in cortisol (stress chemicals) in the whole family. Variations on the payment program have also led to improvements in performance at school,[5] and a reduced risk that teenage girls would drop out to depend on ‘sugar daddies’.[6] Each of these RCTs required the mobilisation of huge resources— an army of surveyors, partnership with the government of a low-income country and usually a large charity. It is truly a war against poverty. Poor Economics by Banerjee and Duflo (2012) provides an excellent insight into the work undertaken and the lessons learned.

As with all revolutions, there are some good outcomes and some collateral losses. Aside from the efforts of Banerjee himself, there has been little attempt to draw on economic theory and learn deeper lessons from an RCT. Problems like corruption depend a great deal on past history, and local expectations; these have proved less amenable to RCTs and we have learned less. And RCTs can raise serious moral questions. Researchers have recently sought to establish ethical protocols for RCTs: if a beneficial intervention such as de-worming will be delivered to the children in some randomly-chosen schools, other schools should receive the intervention with a lag.

But the passion that these three have shown for making a real difference in developing countries has changed economics. They have inspired a generation of economics students to mobilise resources, engage with governments, and solve real problems. It has been a true blessing.

Dr Catherine de Fontenay is a Commissioner at the Productivity Commission and holds an honorary position in the Department of Economics, University of Melbourne.

ENDNOTES

[1] P. Glewwe, M. Kremer, S. Moulin, and E. Zitzewitz, ‘Retrospective vs. Prospective analyses of school inputs: The case of flip charts in Kenya.’ New Research on Education in Developing Economies, Journal of Development Economics Vol.74(1), 2004, pp251-268.

[2] thehindubusinessline.com/blink/meet/ esther-duflo-it-pays-to-care-about-the-poor/article29843571.ece (Accessed November 2020).

[3] A. Banerjee and E. Duflo, Poor Economics (Penguin, 2012), p90.

[4] Ibid., p78.

[5] P. A. Dubois, A. de Janvry and E. Sadoulet, ‘Effects on School Enrolment and Performance of a Conditional Cash Transfer Program in Mexico’. Journal of Labor Economics Vol.30(33), 2012, pp555-589.

[6] S. Baird, C. Macintosh, and B. Özler, ‘Cash or Condition? Evidence from a Cash Transfer Project’ The Quarterly Journal of Economics, vol.126(4), 2011, pp1709-1753.