home > by publication type > articles > An Evaluation of World Bank Research: 1998-2005
Email
| Author: | Jessica LeCroy, Former Visiting Senior Fellow for Geoeconomics |
|---|
January 2007
Council on Foreign Relations
In late December, the World Bank quietly released an independent and rigorous evaluation of all the Bank’s research between 1998 and 2005. The findings contain something of interest for just about every student of development economics, international relations academic, and foreign affairs practitioner.
The Financial Times characterized the 164 page report as “a scathing critique” of the Bank’s selective use of unscientific data to advocate policies and projects that are ideologically in vogue, such as pension reform, aid effectiveness, and poverty mapping. Some Bank critics will undoubtedly try to use the report to attack the credibility of the institution or to support certain anti-globalization views. This would be a mistake. All research should also be subject to the same rigorous scrutiny. The World Bank, as the preeminent development research institution, should certainly be held to the highest standards of impartial research. Most research economists, however, assume their colleagues engage in data mining to support their theses and to tease out causes and effects that they will be called upon subsequently to defend. In this report the evaluators call the World Bank to task.
World Bank Chief Economist Francois Bourguignon commissioned the Evaluation of World Bank Research: 1998-2005, and the final report was agreed to by the Bank’s senior management and board after several months delay. A distinguished four-person panel carried out the review: Angus Deaton of Princeton, who chaired the panel, Kenneth Rogoff of Harvard, Abhijit Banerjee of MIT, and Nora Lustig at UNDP. An equally distinguished team of twenty-four thematic evaluators assessed a random sample of 186 research projects and reports from the more than 4,000 journal articles, books, and databases produced during this seven-year period. The sample also included fifty of the best outputs chosen by the Bank’s Development Research Group, and a built-in bias toward more recent and current projects. The panel interviewed staff, management, developing country policymakers, NGOs, and other users of Bank research. The objective was to assess whether Bank research had generated new knowledge on development and contributed to the broadening of understanding of development policy—this in keeping with the growing recognition that the Bank should be a “Knowledge Bank” rather than a lending institution.
The report is an invigoratingly good read, charting a course with recommendations that can strengthen the integrity of Bank research and stimulate productive development policy discussion within academia and development agencies.
There is praise for much of the research work the Bank does under its existing resource and management constraints. The evaluators scored 61 percent of the reports they read to be of superior or above average quality with respect to increasing knowledge of development. Special credit was accorded to the collection of data contained in various World Bank surveys: Doing Business, Investment Climate, Business Environment and Economic Performance, the Living Standards Measurement, and household survey project with the inter-development bank, MECOVI. The panel noted that the World Bank Indicators is the most important single database for development research, but suggested that the Bank’s “Flagship” reports, such as the World Development Report, be reduced to maintain quality.
Some of the panel’s findings, however, are indeed candidly severe and “substantial.” The panel believed that much of the Bank’s advocacy research was designed not to increase understanding or advance new knowledge, but to “proselytize on behalf of Bank policy, often without taking a balanced view of the evidence, and without expressing appropriate skepticism.” The panel endorsed the Bank’s right to defend its policies but argued that Bank leadership “selectively appeals to relatively new and untested research as hard evidence that its preferred policies work…” This, the panel held, impaired learning about what would promote development. The annexed individual thematic evaluations summarized in the report provide plenty of detailed examples to support this claim.
The panel’s criticism, however, pales in relief against the value of transparency the evaluation engenders. This was the Bank’s first evaluation of research in seven years, and only two similarly comprehensive exercises have occurred in the history of the Bank, in 1978 and in 1983. More regular evaluations could stem future problem areas.
Among the panel’s recommendations are that the World Bank should:
In their responses to the report, both the current chief economist, and two former chief economists, Nicholas Stern and Joseph Stiglitz, acknowledge the value of the report for improving ongoing and future research. They agreed with some recommendations— consensus reigned on the advocacy-research divide—and they expressed reservations with others, e.g., methodological preferences and what constitutes “undistinguished” research. Most interesting was their identification of additional “gaps” in the Bank’s research that the panel failed to uncover in this evaluation.
The report, its annexes, and the responses may be published in book form later in the year but can be found now at the World Bank’s website, which everyone seemed to agree “is of poor quality and difficult to use.”
See An Evaluation of World Bank Research, 1998-2005 at http://econ.worldbank.org/WBSITE/EXTERNAL/EXTDEC/0,,contentMDK:21165468~pagePK:64165401~piPK:64165026~theSitePK:469372,00.html
To order Task Force reports, Council Special Reports, and Critical Policy Choices, please call, fax, or order online from our distributor, the Brookings Institution Press: phone +1.800.537.5487, fax +1.410.516.6998.
For information on other reports that are not for sale, or for general publications information, please call +1.212.434.9516 or email publications@cfr.org.
Start-Up Nation addresses the trillion-dollar question: How is it that Israel—a country of 7.1 million, only sixty years old, surrounded by enemies— produces more start-up companies than large, peaceful, and stable nations like Japan, China, India, Korea, Canada, and the UK? With the insights of geopolitical experts and investors, the authors examine this nation’s adversity-driven culture to answer this question and offer prescriptions for a global economy on the rebound.
In Forces of Fortune, Vali Nasr presents a paradigm-changing revelation that will transform the understanding of the Muslim world at large. He reveals that there is a vital but unseen rising force in the Islamic world—a new business-minded middle class—that is building a vibrant new Muslim world economy and that holds the key to winning the cold war against Iran and extremists.
In Cuba: What Everyone Needs to Know, Julia E. Sweig presents a remarkably accessible portrait of Cuba's unique place on the world stage over the past fifty years, including its internal politics, its often fraught relationship with the United States, and its shifting relationship with the global community.
Complete list of CFR Books
Copyright 2009 by the Council on Foreign Relations. All Rights Reserved.