World Bank research impact questioned

6 April 2011

World Bank evaluations of the influence of its research and reports paint a mixed picture, while the institution continues to stoke controversy over its approach to measuring poverty. A July 2010 Bank paper on The World Bank’s publication record, authored by Martin Ravallion and Adam Wagstaff of the Bank’s development research group, found “evidence that many [Bank] publications have influenced development thinking … [but] a non-negligible share of the Bank’s publications have received no citations, suggesting that they have had little scholarly influence”. Unsurprisingly, they found that Bank-published books and articles were less influential than the minority – 45 per cent – that were published by others including in scholarly journals.

A January blog by Bank staffer Adam Wagstaff examined whether the Bank’s flagship World Development Reports (WDRs) are as influential as they claim to be. He found that while internet searches on flagship reports of the Bank, IMF and UN had declined significantly in number in the past decade, “on average the search frequency for the WDR was 60 [per cent] that of the HDR [Human Development Report, the UN’s flagship report].”  Similarly, the number of scholarly references to all flagship reports “continued to grow … through to around 2003 but has since fallen off except in the case of the IMF’s World Economic Outlook, WEO.” After 2000 the WDR was “left behind” by the HDR and the [IMF’s World Economic Outlook] in news item coverage. This confirms previous criticism that evidence of the WDR’s influence was “noticeably thin” (see Update 66).

Meanwhile the debate between the Bank and others on how to measure poverty (see Update 62, 61) bubbled over again with the launch of the HDR last November. In a guest blog for Oxfam, the Bank’s director of research, Martin Ravallion criticised the UN for including the Oxford poverty and human development initiative’s Multidimensional Poverty Index (MPI) in the HDR.  The MPI has ten indicators grouped into three clusters – health, education and living standards – each of which has equal weighting. Ravallion accuses the index of “essentially adding up ‘apples and oranges’ without knowing their relative price” and argues that though health, education and other factors are important in measuring poverty, they cannot be directly compared to measures of consumption and so should be examined separately.

In a response blog, the co-creator of the MPI, Sabina Alkire argues that Ravallion has missed the point. The MPI allows drilling down into the different complexities of each cluster, so you can “zoom in and see more”, but examining the different dimensions of poverty together is a strength because it means you can see “the [different] deprivations that batter poor people’s lives at the same time”.  She also argues that Ravallion’s focus on price is “not unproblematic in practice”. By contrast, “instead of using prices, the MPI sets weights as value judgements.  Amartya Sen among others sees this feature as a strength not an embarrassment” because it makes assumptions explicit and encourages debate about what weightings should be, and therefore about how important different dimensions of poverty are.