In late December, the World Bank quietly released an independent and rigorous evaluation of all the Bank’s research between 1998 and 2005. The findings contain something of interest for just about every student of development economics, international relations academic, and foreign affairs practitioner.
The Financial Times characterized the 164 page report as “a scathing critique” of the Bank’s selective use of unscientific data to advocate policies and projects that are ideologically in vogue, such as pension reform, aid effectiveness, and poverty mapping. Some Bank critics will undoubtedly try to use the report to attack the credibility of the institution or to support certain anti-globalization views. This would be a mistake. All research should also be subject to the same rigorous scrutiny. The World Bank, as the preeminent development research institution, should certainly be held to the highest standards of impartial research. Most research economists, however, assume their colleagues engage in data mining to support their theses and to tease out causes and effects that they will be called upon subsequently to defend. In this report the evaluators call the World Bank to task.
World Bank Chief Economist Francois Bourguignon commissioned the Evaluation of World Bank Research: 1998-2005, and the final report was agreed to by the Bank’s senior management and board after several months delay. A distinguished four-person panel carried out the review: Angus Deaton of Princeton, who chaired the panel, Kenneth Rogoff of Harvard, Abhijit Banerjee of MIT, and Nora Lustig at UNDP. An equally distinguished team of twenty-four thematic evaluators assessed a random sample of 186 research projects and reports from the more than 4,000 journal articles, books, and databases produced during this seven-year period. The sample also included fifty of the best outputs chosen by the Bank’s Development Research Group, and a built-in bias toward more recent and current projects. The panel interviewed staff, management, developing country policymakers, NGOs, and other users of Bank research. The objective was to assess whether Bank research had generated new knowledge on development and contributed to the broadening of understanding of development policy—this in keeping with the growing recognition that the Bank should be a “Knowledge Bank” rather than a lending institution.
The report is an invigoratingly good read, charting a course with recommendations that can strengthen the integrity of Bank research and stimulate productive development policy discussion within academia and development agencies.
There is praise for much of the research work the Bank does under its existing resource and management constraints. The evaluators scored 61 percent of the reports they read to be of superior or above average quality with respect to increasing knowledge of development. Special credit was accorded to the collection of data contained in various World Bank surveys: Doing Business, Investment Climate, Business Environment and Economic Performance, the Living Standards Measurement, and household survey project with the inter-development bank, MECOVI. The panel noted that the World Bank Indicators is the most important single database for development research, but suggested that the Bank’s “Flagship” reports, such as the World Development Report, be reduced to maintain quality.
Some of the panel’s findings, however, are indeed candidly severe and “substantial.” The panel believed that much of the Bank’s advocacy research was designed not to increase understanding or advance new knowledge, but to “proselytize on behalf of Bank policy, often without taking a balanced view of the evidence, and without expressing appropriate skepticism.” The panel endorsed the Bank’s right to defend its policies but argued that Bank leadership “selectively appeals to relatively new and untested research as hard evidence that its preferred policies work…” This, the panel held, impaired learning about what would promote development. The annexed individual thematic evaluations summarized in the report provide plenty of detailed examples to support this claim.
The panel’s criticism, however, pales in relief against the value of transparency the evaluation engenders. This was the Bank’s first evaluation of research in seven years, and only two similarly comprehensive exercises have occurred in the history of the Bank, in 1978 and in 1983. More regular evaluations could stem future problem areas.
Among the panel’s recommendations are that the World Bank should:
- Allocate sufficient funding to high-level research and researchers, including the creation of an endowment fund, which would help ensure independence from the Board and management. (The report noted that research accounts for only 2.5 percent of the Bank’s administrative budget, less than half that devoted to supporting the Bank’s executive board.)
- Consider ways to enhance the role of the chief economist in quality control management to ensure a) research is separated from advocacy, b) policy relevance, and c) proper platforms for testing, evaluation, and learning.
- Consider the creation of a chief statistician position, as an additional check and balance, to strengthen and centralize the Bank’s data function in the collection and generation as well as the revision and dissemination of data.
- Foster better capacity building and collaborative relationships with academia, especially with developing country researchers in client states, as essential sources of new ideas and data sources and venues of dissemination of data and ideas.
In their responses to the report, both the current chief economist, and two former chief economists, Nicholas Stern and Joseph Stiglitz, acknowledge the value of the report for improving ongoing and future research. They agreed with some recommendations— consensus reigned on the advocacy-research divide—and they expressed reservations with others, e.g., methodological preferences and what constitutes “undistinguished” research. Most interesting was their identification of additional “gaps” in the Bank’s research that the panel failed to uncover in this evaluation.
The report, its annexes, and the responses may be published in book form later in the year but can be found now at the World Bank’s website, which everyone seemed to agree “is of poor quality and difficult to use.”
See An Evaluation of World Bank Research, 1998-2005 at http://econ.worldbank.org/WBSITE/EXTERNAL/EXTDEC/0,,contentMDK:21165468~pagePK:64165401~piPK:64165026~theSitePK:469372,00.html