Letter on EVO procedures and topics for evaluation

Montek Ahluwalia
Director EVO
IMF
700 19th Street NW
Washington DC 20431

21st August, 2001

RE: EVO procedures and topics for evaluation

Dear Mr Ahluwalia,

We have very much appreciated the opportunity to discuss with you proposals for operationalising the Independent Evaluation Office (EVO) and evaluation topics. The creation of the EVO is an important step forward for IMF transparency, good governance and learning and we welcome your appointment as Director.

We would like to take this opportunity as you begin drafting your work plan for EVO to clarify what we consider should be key features of the EVO‘s operational procedures and to suggest priority evaluation topics.

1. Mechanisms/Procedures

Civil society involvement

An essential feature of the new unit is that it is independent and that it provides a formal mechanism for outside stakeholders to raise concerns with IMF programmes and procedures. To make this operational the EVO‘s procedures should include the following features:

• a right for outsiders to submit suggestions for evaluation topics to the office and to receive a written response. These should be posted on the EVO‘s website;
• an obligation for the EVO to discuss the suggestions it receives and its responses thereto in its annual report;
• the EVO‘s draft work programme should be made available within sufficient time for outside stakeholders to comment on it before it is finalised
• draft terms of reference for evaluations should be published in order for outside stakeholders to comment on them before they are finalised.

It is equally important that civil society’s views are considered, where appropriate, during evaluation. It will be necessary for the EVO to draw up clear guidelines as to when and how outside stakeholders will participate in the evaluations. For example, how much notice stakeholders (including parliamentarians, government departments and the private sector) should be given in advance of an evaluation team arriving in the country, which IMF documents will be made available; what feedback participants can expect etc. This is particularly necessary given that the IMF does not have guidelines on participation.

From discussions with IMF staff we understand that the EVO is not required to follow IMF procedures as stipulated in its Articles of Agreement. The EVO‘s legal accountability is to the Board. This implies that the EVO is not required to liaise with member states through the Ministry of Finance and is not constrained as to whom it consults with. The terms of reference for your post indicate that you should broadly consult, this applies beyond the EVO start-up period.

Reporting back views

EVO recommendations and conclusions will not be able to satisfy all the concerns of all stakeholders involved. It will therefore be important for the evaluation reports to summarise the variety of concerns raised and to clarify how these views have been taken into consideration and how recommendations and conclusions have been arrived at.

Release of evaluation documents

You have suggested that recommendations contained in EVO reports could be changed in the light of further discussions with IMF staff and management before the final document is sent to the Board. Whilst from a “learning” perspective such a process could be helpful the danger is that it will diminish the credibility of the EVO from an “accountability” perspective. Since IMF staff will be consulted during the course of an evaluation we do not agree that EVO recommendations should be open to change following further conversations with staff. A more preferable route is to publish staff comments alongside the evaluation report, in addition to the Board’s. It should be the Board’s responsibility to judge between the differing points of view. Bringing these into light would be a very useful learning exercise in its own right.

All participants in a review should receive notice that the evaluation report is publicly available and from where they can obtain copies. Where evaluations have focussed on a specific country reports should be made available in the national language.

Comments on evaluation reports

In the interests of equity and independence we believe that it is essential that civil society and other external stakeholders as well as IMF staff and the Board should have the opportunity to formally and publicly submit their comments on EVO reports. Thus we suggest that once evaluation reports are made public there should be a clearly stated period during which any member of the Board, staff and outside stakeholders can submit comments. These should all be made available on the EVO‘s website.

Monitoring implementation

The Board’s comments should include details of which EVO recommendations it expects the IMF staff to implement. This is essential for effective monitoring. The EVO‘s annual report should detail what action has been taken to implement the agreed recommendations.

Staffing

It is essential that the EVO staff include experts in social and environmental issues as well as macro-economic issues. We are concerned that to date we have only seen notice of job vacancies for economists. The added value of the EVO is that it should be able to analyse IMF programmes and operations from perspectives other than those commonly held by economists. Whilst non-economic expertise can be brought in an a case by case basis, we believe that it is essential that there is an appropriate skills mix, including political economy and social policy analysts and evaluation specialists, amongst the permanent staff to ensure that full consideration is given to these perspectives when work programmes and terms of reference are being drawn up.

In addition, it is extremely desirable that there be a lawyer on the EVO staff. This would ensure that:

• the EVO is not dependent on the opinions of the IMF legal office when legal issues arise relating to such issues as the powers of the EVO and the interpretation of its procedures arise. The role that the general counsel plays in the Inspection Panel process in the World Bank has been a serious problem for the Inspection Panel;
• the EVO is able to deal with legal issues that could arise in the evaluations – for example, in evaluation of the governance aspects of IMF programmes and in relation to the social and environmental aspects of programmes.

2. Topics for evaluation

The IMF‘s work overlaps with that of several different institutions, most notably the World Bank. We are encouraged that you are keen to collaborate with the Bank’s Operations Evaluation Department. In addition we urge you to work with other agencies, such as the UNDP, ILO and UNCTAD, to take into consideration in all relevant evaluations the linkages between the Fund’s work and other agencies’ work, and to examine the broader impacts of programmes beyond the IMF‘s core objectives.

A Review of the Fund’s Experience With and Record on Privatisation

We urge that the EVO undertake a review of the Fund’s experience with and record on privatisation. Privatisation has been a significant component of Fund conditionality, particularly in its programmes in the poorest countries and the former communist countries. The 1998 external review of the Enhanced Structural Adjustment Facility highlighted privatisation as an area in which World Bank/IMF collaboration had been poor, resulting in poor sequencing, negative social impacts, and a failure to generate sufficient rents for the government. The IMF staff are currently re-considering the use of structural conditionality, when and how it should be applied and coordination with the Bank. Privatisation, has not been discussed by the staff during their review. An evaluation of this topic would be a helpful contribution to the current debate.

We urge you to look at the following aspects of privatisation:

• Returns earned from privatisation. Have governments sold state-owned enterprises for a reasonable price? How has the sale price compared with the book value and earnings streams? Has the pressure to privatise quickly affected the sale price? How were the proceeds used? Did privatisation free up government resources for social and environmental expenditures? What is the evidence from IMF programme countries that state owned enterprises are more viable after privatisation and what changes are required in privatisation design and implementation to ensure such an outcome?
• Regulatory framework and privatisation process. Have appropriate regulatory frameworks been put in place prior to privatisation? What efforts has the IMF made to work with the World Bank to ensure regulations are in place prior to privatisation? Are these efforts sufficient? What is the ownership structure post-privatisation of the enterprise and in the industry more generally? In particular, in which cases has IMF-led privatistation led to a transfer of ownership to companies with headquarters in developed countries and what are the consequences for the development of domestic industry? In which sectors has privatisation led to regional (and wider) monopolies for service provision, and what are the consequences of this? What have been the impacts of privatisation on distribution of property rights and management of public goods? Has the privatisation process been properly regulated and prospective buyers vetted so as to remove incentives for corruption? Have measures been pursued to give opportunities to workers or community members to acquire shares of the privatised enterprise? Were alternatives to full privatisation considered?
• Sector breakdown. In which sectors has privatisation been advocated? Has the IMF treated privatisation of natural monopolies or of basic services, such as the provision of water and sanitation, energy, or infrastructure, differently from other sectors? What has happened to the prices and quality of service of privatised enterprises, particularly in the area of public services? In which sectors has privatisation catalysed new investment in increased financial flows as well as in expanded geographic coverage?
• Impacts on poverty. How has privatisation of basic services impacted on poor peoples’ access to them, eg. water and electricity, in terms of physical access, pricing, user fees and quality? Have measures been included to ensure that the poorest sectors of society are guaranteed some basic service provision post-privatisation? How effective are these measures? Have workers laid off from privatisation received adequate transitional assistance?

Financial liberalisation and its contribution to sustainable livelihoods and poverty reduction

There has been considerable rethinking about the steps towards full capital account liberalisation since the crisis in Asia in 1997. However, whilst the IMF insists that it supports appropriate sequencing it is suspected that some countries, particularly poor ones, are pursuing liberalisation (or previously liberalised) without appropriate regulatory mechanisms in place. The pressure to liberalise stems from the perception that it will encourage capital inflows and therefore growth. However, a review of the literature suggests that capital account liberalisation guarantees neither. In particular, it is unclear how private flows can substitute for aid, particularly in the poorest countries. Moreover, there is evidence that aid resources are also pro-cyclical and therefore do not help stabilise countries during capital outflow periods.

• Appropriate liberalisation. To what extent have countries which have taken steps to liberalise elements of their financial markets since 1997 done so with or without appropriate safeguards and regulatory systems in place? What role has the IMF played in advising countries on the liberalisation path, and in particular, on appropriate safeguards? To what extent has the IMF advised the poorest countries to maintain controls where regulatory regimes are not sufficiently developed?
• Development impact. Given the apparent lack of a clear positive relationship between new investment, growth and capital account liberalisation the biggest impact capital account liberalisation is likely to have on poorest groups is through its impacts on government spending and on their access, and the access of small and medium enterprises, to credit at affordable prices and to a broad range of financial services. The EVO should examine to what extent both domestic liberalisation and capital account liberalisation has increased access to credit amongst the poor and what impact it has had on the size of government budgets and their allocation. The EVO should examine the impact of financial sector and budgetary reforms on actual poverty and environment outcomes in client countries.
• Role of IMF assistance for stability. Evidence suggests that multilateral and bilateral resources are pro-cyclical which reinforces the destabilising effects of private capital outflows. The EVO should examine to what extent the IMF‘s assistance, particularly in the poorest countries, helps to provide stability during periods of capital outflow or whether it is actually cut back during these periods thereby contributing to further instability.

Staff expertise and allocation between departments

There have been considerable changes in IMF priorities in the last couple of years. An increased focus on financial market issues and surveillance is one such priority. Another is the IMF‘s commitment to the goal of poverty reduction in the poorest countries. Moreover, the IMF is moving towards an “ownership” approach to programme design, particularly in the poorest countries with the introduction of the PRSP and as evidence mounts that conditionality to induce policy change is likely to be less effective in the future.

• Appropriateness of staff expertise. Currently staff resources are being reorganised to give greater capacity to address financial market issues. However, there has been no similar examination of staff resources required to address poverty reduction goals. The EVO should examine whether the IMF staff has the appropriate mix of expertise given its new objectives and whether staff resources are appropriately allocated between departments. In particular attention should be paid to collaboration with Bank staff and whether there are effective processes which enable the two staffs to work together and share analysis. Consideration should be given to what extent the IMF staff should develop their expertise to understand the linkages between social outcomes and economic policy to ensure appropriate policy design.
• Building capacity and enhancing ownership. A shift to an ownership approach suggests at least two issues for staff expertise and deployment. The first is that there is an inverse relationship between staff who have country experience and staff who have decision making power. As more negotiation on programme design moves to the country level it suggests that the authority to agree and make changes to programmes should also shift to the country level. The EVO should examine to what extent this is happening and what further action needs to be taken to ensure that staff have appropriate country experience. Another aspect is to what extent more staff should be decentralised to the field. The World Bank has been undertaking such an approach for the past several years and it would be useful to examine to what extent this has improved country relations and programme performance vis a vis IMF relations and performance.

A second issue concerns whether the IMF staff are effectively able to monitor countries and whether staff resources are best used in this capacity. Particularly as little benefit is transferred to the country in terms of expertise or increased capacity. An alternative which the EVO should investigate is whether the IMF should adopt an approach similar to the OECD‘s whereby governments evaluate each other and what impact this may have on building understanding about the policy reform process and commitment to it.

A third and related issue is the role of technical assistance. Governments are crying out for more technical assistance to help them to assume “ownership” of programme design. In addition, the IMF‘s conditionality review has identified the need for more technical assistance to replace micromanagement of the reform process. However, several poor country governments have commented that IMF technical assistance in its current form is not contributing to capacity building. The EVO should examine to how technical assistance must be reformed to ensure that it is an effective resource for governments.

We look forward to hearing your response to our proposals for appropriate operational mechanisms and suggestions for evaluation topics. If you would like any further details on these proposals please be in touch with us.

Yours sincerely,

Angela Wood
Bretton Woods Project, UK

Signed on behalf of:

Professor Danny Bradlow, American University, US

Simon Burrall, One World Trust, UK

Karen Joyner, BothEnds, The Netherlands

Phil Twyford, Oxfam International, US

David Beckmann, Bread for the World

Jo-Marie Griesgraber, Oxfam America, US

Henry Northover, CAFOD, UK

Robin Robison, Quaker Peace and Social Witness, UK

Rob Mills, Eurodad Secretariat, Belgium

Miriam Walter, WEED, Germany

Carol Welch, Friends of the Earth US, US

Damanjit Singh, WWF UK, UK

Pamela Foster, Halifax Coalition, Canada

Cc IMF Executive Directors; Robert Picciotto, OED.