One of J-PAL's studies tested the effectiveness of computer-assisted learning in certain Indian schools. Photo: niyam bhushan (flickr)

A research group thinks the best way to determine whether aid programs work is to evaluate them using the scientific method.

J-PAL, short for for the Abdul Latif Jameel Poverty Action Lab, is a group of researchers - loosely affiliated with MIT - who help design and publish academic-quality studies of existing poverty alleviation programs in an effort to find out exactly what works and what doesn't. These researchers partner with non-governmental organizations (NGOs) to evaluate their programs. They also offer courses for other researchers to share their methods. (Class materials are available here for free.)

As J-PAL's website explains, the key to their approach lies in randomization, which is an important part of a well-designed study:

Suppose we would like to see whether one thing (e.g. "schooling") really improves a life outcome (e.g. "health"). The natural instinct is to compare the health of those who have schooling to those who don't. But this would be like comparing apples and oranges: People who have been to school are different in so many ways from those who haven't. Perhaps they have more advantaged social backgrounds, greater access to government services and so better access to schools. And only a few of these factors can be measured and accounted for in a standard statistical analysis. So this simple comparison — those with schooling to those without — may tell us little about the effect of schooling: It may instead be the effect of any of these numerous other differences like social background. If policy are set on the basis of such apples and oranges comparisons, quite a bit of disappointment may result.

J-PAL uses this approach to evaluate existing programs. For example, J-PAL researchers cooperated with an NGO called Pratham to study how much weekly computer use could boost Indian students' academic achievement. They introduced computer-based learning only into randomly selected schools that Pratham was already serving. Students at these schools received basic computer skills training and two hours per week of independent computer time with educational software. After a year, J-PAL found that these students' math test scores had risen, but that their other skills hadn't changed significantly — and all for slightly more money than another effective Pratham program J-PAL had also evaluated.

What makes J-PAL's work innovative is that such randomized studies haven't typically been used in evaluating poverty-alleviation programs, or even in the wider field of economics.

Such data is designed to help aid organizations and governmental bodies decide the most effective way to allocate their funds. For the extra cost of designing an extra study now, J-PAL believes, more money can be directed toward the most effective programs for better poverty-alleviation strategies in the future.