The 2005 Annual Report on Operations Evaluation (AROE) by the Independent Evaluation Group (IEG) of the World Bank, has taken a step back from previous strong criticism, electing to give the Bank more time to see the impact of new measures taken in response to previous editions.
The focus of this year’s report is on measures taken by the Bank to improve results at the country level:
- First, the IEG says that strategic planning is needed to align Bank activities with desired country outcomes. The report finds that significant improvement is needed in developing baselines and targeted performance measures.
- Second, plans need to be monitored and evaluated. A lack of capacity for data collection and limited country interest is constraining the development of measurement systems. IEG calls for a diagnosis of a country’s performance measurement capacity and recommendations for Bank action to strengthen that capacity.
- Finally, the information from monitoring and evaluation needs to be useful for operational decisions. IEG has suggested that the Country Portfolio Performance Review – where borrowers get the opportunity to carry out their own assessment of the portfolio and to raise any problems with the Bank, importantly this document is not disclosed – be made into a tool for the government and the Bank to jointly manage performance, and to “address the disconnect between overall country and individual instrument results”. There needs to be greater emphasis on the outcomes of projects and advisory activities, and their linkages to country development goals.
The report also takes stock of measures taken since the 2003 and 2004 AROEs to strengthen monitoring and evaluation (M&E), concluding that it is “not being approached in a systematic or strategic manner”.
- At the project level, the IEG said “efforts are under way” to strengthen evaluation, pointing to the Development Impact Evaluation Initiative, a pilot initiative using ex-post poverty and social impact assessment, falling short of calls which civil society groups have been making for ex-ante impact assessment for years.
- In response to previous charges that M&E in analytical work was “rare”, beginning 1 July 2004, Bank task team leaders were required to define at the beginning of economic and sector work and technical assistance activities the desired outcomes and to identify monitorable indicators of success. IEG calls this a “positive step”.
- The IEG’s most severe criticism in 2004 was reserved for the Bank’s sector strategies which it said lacked both selectivity and guidance for staff. Since that time, the IEG says that “progress is being made”, pointing to the creation of the Sector Strategy Implementation Update of February 2005 – a new report that documents progress in all the sectoral strategies.
- An internal review of the trust fund portfolio in 2004 concluded that “little progress has been made in developing standard performance indicators to measure the results and development impact associated with a trust fund program”. The IEG is in discussions with the Trust Funds Operations unit and the Global Programs and Partnerships group on ways to advance M&E for trust funds and global programmes. In its response to the report, management said it would be introducing independent evaluations of global programmes every three to five years.
- Finally, the 2003 AROE recommended that management issue an operational policy that sets out the “mandate, framework and roles and responsibilities for self and independent evaluations”. Such a policy remains to be issued.
With so many new initiatives to respond to gaps identified in monitoring and evaluation pending or just initiated, the IEG has taken a ‘wait and see’ approach. Bankwatchers will be expecting a much more exacting analysis of these initiatives in next year’s report.
