In June the World Bank’s Independent Evaluation Group (IEG, see Observer Winter 2017) published its annual evaluation of the “development effectiveness” of four of the five World Bank Group (WBG) institutions: the International Bank for Reconstruction and Development (IBRD), the World Bank’s middle income lending arm; the International Development Association (IDA) – its low income lending arm; the International Finance Corporation (IFC), the Bank’s private sector lending arm; and the Multilateral Investment Guarantee Agency (MIGA).
This year’s report included a thematic focus on environmental sustainability. The report, Results and Performance of the World Bank Group 2017, outlined concerns about the specific development effectiveness of the Bank’s institutions and raised serious questions about whether the World Bank’s organisational environment encourages it to use “findings from evaluation[s] – whether from IEG or others – [to] inform the World Bank’s operations as a matter of course.” This finding raises important issues ahead of the formalisation of the World Bank’s general capital increase at this year’s Annual Meetings in Indonesia. The report provides additional evidence in support of civil society’s analysis that the key challenges faced by the Bank in meeting its development objectives are structural rather than financial (see Observer Summer 2018).
The IEG used two parameters to evaluate the Group’s development effectiveness: development outcome and World Bank performance. The IEG’s analysis of the former was based on the extent to which project results at completion met the development outcome objectives predicted at the design phase. According to the IEG, “the World Bank performance rating relies on factors within the control of the Bank – for example, whether the project design is suited to the needs of its beneficiaries.”
The report ... raised serious questions about whether the World Bank’s organisational environment encourages it to use “findings from evaluation[s] – whether from IEG or others – [to] inform the World Bank’s operations as a matter of course.
The environment: Despite progress, significant shortcomings remain
On a positive note, the report found that, compared with the last report, the Bank’s “share of project components with potential environmental benefits increased from 33 percent to 37 percent” with the IFC witnessing an increase from 31 per cent to 35 per cent in “potential environmental benefits from financed activities.” For MIGA, “the share of political risk guarantee projects with potential environmental benefits increased from a low base of 8 percent to 36 percent.” In noting the positive trend, however, the IEG, echoing long-standing CSO concerns (see Observer Spring 2018), asked whether “these increases are sufficient, given the scale of pollution, environmental degradation, and climate change.”
The report also echoed persistent civil society concerns that the WBG lacks adequate information and data on the environmental impacts of its entire portfolio, such as its aggregate greenhouse gas emissions, impact of development policy lending and supervision of financial intermediary clients (see Observer Winter 2017). It stressed that “existing systems are limited by what can be observed at the point of evaluation and are often not able to capture longer-term results.” Further aligning itself with persistent critiques of the IFC in particular (see CSPF meeting notes Spring 2018), the IEG stressed that “for IFC, disclosure of project-level environmental and social information from monitoring and supervision reports during implementation is still inadequate, and third-party monitoring could be used more widely.”
The report, in considering management’s implementation of its past environment-related recommendations, found that, “Environmental recommendations to generate knowledge, enhance use of metrics, or measure the impact of Bank Group interventions… were the least likely to be fully implemented.” The IEG noted the need for better evidence on the impact of WBG interventions and improved reporting of environmental impacts, especially outside of core environmental interventions. It also identified several weaknesses in the Bank’s monitoring and evaluation capacity, including “secondary prioritization of knowledge and learning behind operational delivery.”
Further doubts raised about Bank’s MFD strategy
The report’s findings provide further evidence in support of civil society’s critique of the Bank’s “Maximising Finance for Development” (MFD) strategy – which some within civil society have taken to calling “Minimising Finance for Development” (see Observer Summer 2017). It raises important questions about the developmental logic and consequences of the Bank’s increasing reliance on the IFC to effectively ‘leverage private sector’ investment to support its development mandate.
The IEG noted that, “Development outcome ratings for IFC investment projects continued a downward trend” started in 2008, stressing that “only half the projects evaluated” for the current report “were rated mostly successful or better.” The report was particularly damning for the MFD agenda, which relies on IFC work ‘upstream’ through its Advisory Services, where “the decline in development effectiveness rating was even sharper than for IFC Investment Services projects,” with projects rated mostly successful or better for development effectiveness falling “to 49 percent in FY14–16.” The IEG stressed that “this is the lowest positive development effectiveness rating registered since IEG first started reviewing IFC Advisory Services projects in 2008.” Equally concerning, the report noted that, the “IFC typically engages in projects at the stage leading up to the transaction rather than the early design stage; this means that IFC is less able than the World Bank to influence the design of projects it finances.” This contradicts IFC’s often-repeated justification that, whatever the challenges faced in terms of project implementation, it continues to play an important role in changing private sector behaviour so that it is more closely in line with development objectives.
Inside the ‘Solutions’ Bank’s echo chamber
Perhaps the most troublesome of the IEG’s findings relates to the unwillingness of the ‘Solutions Bank’, as the WBG markets itself, to engage with, learn from and adapt the implementation of its activities to robust critique. The report noted that while management is only required to report on the IEG recommendations to which it has agreed, after four years “Only 52 per cent of the recommendations have been implemented to a high or complete degree.” Considering management action plans that respond to IEG recommendations, the report also noted that, “action plans are not discussed by CODE [the Bank’s Development Committee] or Board members, and IEG acceptance of action plans is not required”, continuing that “the risk exists that IEG recommendations remain underaddressed even when action plans are fully implemented.”
It seems from the IEG report that trends highlighted in the 2006 Evaluation of World Bank Research, 1998 – 2005 led by Angus Deaton about the nature of the Bank’s self-referential approach to research and learning remain sadly apt. The IEG’s conclusion that the Bank’s organisational environment may not make the institution well-suited to engaging with outside critique echoes that of the Deaton report, which noted that “a high proportion” of the citations in a significant number of Bank papers “are to other Bank papers, many of them unpublished. In some cases … the degree of self-reference rises almost to the level of parody.”
While the IEG report did not assess the degree to which Bank policy recommendations are based on robust analysis and data, it raised questions about evidence, data and monitoring and evaluation that are sadly consistent with the thrust of the Deaton report findings, which underscored that in many cases policy conclusions in Bank papers “are rarely well based on the preceding analysis.” The unwillingness of Bank management to change its position when faced with the lack of robust evidence in support of its policy recommendations can also be seen in its continued use of the country ratings in its Doing Business Report, despite the recommendations of a 2008 internal evaluation and extensive civil society and academic criticism (see Observer Winter 2017-2018). The substance of the Bank’s upcoming World Development Report on the Changing Nature of Work and reluctance to consult with critics further highlight the relevance of the IEG’s findings (see Observer Summer 2018).
The findings should raise alarms among the Bank’s shareholders, who have just recently agreed a significant capital increase to support a “bigger and better” Bank.
