How to Fix Climate Science Reporting

The UN’s Intergovernmental Panel on Climate Change (IPCC), which issued a series of reports on climate science meant to inform governments worldwide, shared the 2007 Nobel Peace prize for its work. But the panel came under fire in early 2010 over questionable findings--including the projection that Himalayan glaciers could disappear by 2035 instead of 2350--calling into question the quality and neutrality of the IPCC’s methods.

In this CFR roundup, four experts examine the degree to which the IPCC needs to change its reporting process.

CFR’s Michael Levi says the IPCC has some minor flaws that have opened it up to attack. He suggests the panel should encourage exposing mistakes and IPCC leaders should avoid taking "positions on matters of policy." IPCC Lead Author Kevin Trenberth notes that given the scope of the scientific-consensus process, lapses "appear to be fairly few," but he suggests eliminating the parallel writing of reports to prevent redundant and possibly conflicting scientific reviews.

Roger Pielke, Jr., a critic of the IPCC, argues the panel should establish procedures for dealing with conflicts of interest for members, create a mechanism for addressing allegations of errors, and honor its mandate to be policy neutral. And John Christy, lead author of the IPCC’s 2001 report, wants the IPCC restructured so that a panel of policymakers and scientists develops "the important questions," and a separate set of lead authors, "chosen by learned societies, not governments," collects answers from a wide range of contributors to appear in the text, which he likens to Wikipedia. --Toni Johnson, Staff Writer, CFR.org

Michael A. Levi

The IPCC process for reviewing climate science clearly has some minor flaws--but on the whole, the broad thrust of its reporting still appears to be sound. It’s very worrying, though, that a series of relatively small errors has already had major political consequences. Those who are distorting climate science in order to defeat action on climate change deserve the bulk of the blame for this. But some of the IPCC leadership, along with many of its political allies, deserve blame for overselling the panel’s findings, and for blurring science and politics, leaving themselves--and, as a result, efforts to deal with climate change--unnecessarily vulnerable to attack. That said, the technical problems with the IPCC have metastasized into a much bigger political problem for climate action--and that problem will ultimately need to be dealt with on political, rather than technical, grounds.

Three fixes make sense. First, the IPCC leadership and its supporters need to extremely careful to avoid being reflexively defensive. Any report as long and detailed as the IPCC Assessment Report Four is bound to include some mistakes. Efforts to find those, including after publication, should be actively and openly encouraged. Supporters of action on climate change have too often presented the IPCC as bulletproof; the result has been to turn the presence of a few minor holes into a major development.

Second, the IPCC leadership, and its chairman in particular, need to be careful not to take positions on matters of policy. They have certainly not been so circumspect in the past. This is not to suggest that policy preferences have colored their scientific stances. But avoiding even an appearance that that might be the case is essential. To that end, at a minimum, the top IPCC leadership should be employed full time by the panel, rather than performing their duties alongside jobs that entail other demands.

That said, most participants in the IPCC process inevitably perform their service as an additional burden to their full-time jobs, and that will not change, given the number of people involved. Clear conflict-of-interest guidelines should be welcomed both by those skeptical of the IPCC process and by those who consider it to be important: skeptics should be reassured that the IPCC is free of taint, and supporters should welcome some protection for the many scientists who volunteer their time for the IPCC.

Kevin Trenberth, Head of the Climate Analysis Section at the National Center for Atmospheric Research; Lead Author of the 1995, 2001, and 2007 IPCC Scientific Assessment of Climate Change Reports

The IPCC process is very open. Two major reviews were carried out in producing the report, and climate "skeptics" can and do participate, some as authors. All comments were responded to in writing and by changing the report. The process is overseen by two review editors for each chapter. The summaries for policymakers were approved line by line by governments. The rationale is that the scientists determine what can be said, but the governments help determine how it can best be said. Negotiations occur over wording to ensure accuracy, balance, clarity of message, and relevance to understanding and policy. The strength is that it is a consensus report, but the process also makes it a conservative report.

In Working Group (WG) I, there were eleven chapters, and the report was 996 pages plus supplementary material online. There were 140 lead authors, hundreds of contributors, and two or three review editors for each chapter. There were also over seven hundred reviewers. For Chapter 3, in the expert scientific review there were 2,231 comments and another 1,270 comments in governmental review, for a total of 3,501 comments. Every comment and the writer were entered into a huge spreadsheet, along with the response and actions taken in terms of changing the text.

The IPCC review and oversight process is very rigorous. Clearly there can be and have been some lapses, but they appear to be fairly few. I do not think the system is broken and needs further change; it simply needs more attention to adhering to the process already in place.

In my view, the bigger problem is why the errors occurred in the first place. The role of WG I is to assess the science, while Working Group II should assess the impacts and adaptation options, based on the findings of WG I. Because WG I and II work in parallel with only slight lag, the WG II authors often carry out their own assessment of the science, and it can be somewhat at odds with that of WG I, as the expertise differs. This was the case for the Himalayan glacier problem. Data and information on impacts is generally not as long or homogeneous as for physical variables, and one recommendation is to tie the two together more where this makes physical sense. This would mean more integration of the WG I and II activities and doing away with the parallel writing of reports.

Roger A. Pielke Jr., Fellow, Cooperative Institute for Research in the Environmental Sciences, University of Colorado

The IPCC has fallen short of what should be expected of it in at least three ways.

First, establishing procedures for dealing with actual and perceived conflicts of interest are essential to maintaining the credibility of advisory bodies. For instance, the U.S. National Academy of Sciences takes this issue very seriously, explaining that "no individual can be appointed to serve (or continue to serve) on a committee of the institution used in the development of reports if the individual has a conflict of interest that is relevant to the functions to be performed." U.S. and international federal agencies have similarly rigorous guidelines for dealing with conflicts of interest. The IPCC needs to implement conflict of interest guidelines.

Second, the IPCC has no formal mechanism for dealing with allegations of errors, mistakes, or bias in its reports. When the Himalayan glacier issue was first alleged, the IPCC chairman called the allegations "voodoo science" (Guardian) and dismissed it as the work of politically motivated skeptics. This misdirected response increased the problems of the panel when it later had to admit the reality of the glacier error. I have criticized the IPCC for including a very misleading graph on disaster costs and increasing temperatures that appears nowhere in the literature. The author of the graph explained that the graph was "informal" and should not have been included. Yet, rather than investigate the complaint, the IPCC issued a strongly worded press release.

Third, while the IPCC has a mandate to be "policy neutral," its reports, and its leadership frequently engage in implicit and explicit policy advocacy. IPCC leaders often take public stands in support of or opposition to certain policies on climate change. The IPCC reports, particularly Working Group III, reflect a particular policy orientation that is decidedly not "policy neutral." To cite one example, the IPCC has concluded that the world has all the technology that it needs to achieve low stabilization levels. However, this conclusion ignores a significant body of work suggesting that the work does not in fact have all the technology that it needs. The panel’s mandate to be "policy neutral" is an admirable goal, but has not been met in practice.

Unless the IPCC brings its institutional policies and procedures into the twenty-first century through a wholesale institutional reform, it will continue to come out on the losing end of challenges to its legitimacy and credibility.

John R. Christy, Director of the Earth System Science Center, University of Alabama; Contributor (1992, 1994, 1996, and 2007) and Lead Author (2001) for the IPCC Reports

Many of the IPCC problems are rooted in how these reports are developed.

At present, each government nominates hopefuls who are then placed into a pool from which the IPCC bureaucracy selects a set of lead authors. Both steps in this process are clearly influenced by the political views of the governments and the IPCC itself. These selected lead authors are given powerful control by being vested with final review authority and thus are able to fashion a report that supports their own opinions while marginalizing countervailing views. This is not how the real uncertainties and difficulties of climate science may be established and communicated to policymakers.

There are significant climate questions that should be addressed, but done so in a true scientific structure that allows the range of views to be seen by policymakers. A fundamental problem here is that so much (in terms of unassailable facts) is not known about climate science, that opinion and arguments from "authority" have unfortunately carried the day.

To deal more honestly with this situation, a panel of policymakers and scientists could construct the important questions. Most will be derivatives of the following: (a) what is the climate doing; (b) why is it doing what it is doing; (c) what may the future hold; (d) what are the impacts of possible changes (natural and human-induced); and (e) are there effective measures that can be taken to reduce possible harmful impacts?

The current process is not only biased but slow. To provide honest and timely answers to such questions, a type of Wikipedia-IPCC is needed.

Lead authors, chosen by learned societies, not governments, would be referees in the sense they would construct summaries of information to the questions posed. However, the details and analyses provided by a full range of contributors would appear in the text. In this way, and without the page constraints of a book, policymakers may determine which questions have robust answers and which are still subject to large uncertainties. Advocacy groups may complain that such a venue will inhibit "action," but the uncertainties are such that "no action" may be the best route to take.

Room here is not sufficient to describe the full structure and operating rules for such a system, but it would better expose to the world the difficult uncertainties that our fledgling science faces in trying to answer questions for which there are no simple and agreed-upon facts.