The UN’s AI Panel Could Shape Global Governance. Can It Balance Science and Politics?

Prior to joining CFR, Oweke represented a bloc of 134 developing countries in negotiations on artificial intelligence (AI) at the United Nations. 

Science–policy interfaces are built on a fragile equilibrium. Governments require scientific assessments that are attuned to political realities to make policy, while scientists require freedom from political influence to produce credible assessments. Climate change and biodiversity demonstrate, when this balance between political legitimacy and scientific credibility is achieved, collective action can be galvanized to address humanity’s most pressing challenges. When balance is not achieved, or when it is not carefully managed over time, institutions lose authority and wither. The success of the United Nations’ Independent International Scientific Panel on AI (IISPAI) in equipping governments to grapple with the most transformative technology of our time will depend on its ability to find its equilibrium.

Governments confer political legitimacy on institutions they perceive as responsive to their priorities, representative of diverse national contexts, and attentive to the political circumstances surrounding policy formulation. Scientific credibility, by contrast, depends on methodological rigor, independence from political influence, and the freedom to present evidence even when it is uncomfortable for governments. The central dilemma for any science–policy interface is that excessive government involvement can compromise scientific credibility, while complete isolation from governments can render scientific findings irrelevant to policymakers.

The Intergovernmental Panel on Climate Change (IPCC) and the Intergovernmental Science-Policy Platform on Biodiversity and Ecosystem Services (IPBES) navigate this tension through highly institutionalized relationships between governments and scientists. Governments help define the questions these institutions examine, review draft reports, and approve summaries for policymakers. Experts, however, retain responsibility for assessing evidence, drafting reports, and preserving the integrity of scientific conclusions. Both institutions have multilayered structures that designate spaces for governments, government-appointed leadership bodies, and experts, and manage the interactions between them. As a result, reports take years to complete and, in the case of the IPCC, approval procedures have become increasingly difficult given the politicization of climate change.

The IISPAI, while belonging to the same institutional lineage as the above institutions, departs from its predecessors in several areas that are consequential for assessing its legitimacy and credibility. First, the Panel is mandated to synthesize existing research to produce annual reports covering the impacts, risks, and opportunities associated with AI to keep pace with the evolution of the technology. In addition to these reports, the Panel can produce thematic briefs—which are subject to a lower level of scientific scrutiny—to inform the public about relevant AI advancements between annual reporting cycles. By contrast, assessment cycles for the IPCC and the IPBES take between six and seven years and three years, respectively. While these report cycles are infamously long, they enable an extensive review of drafts, ensuring that final products stand up to scientific scrutiny.

Second, the Panel was designed to be independent, operating free from undue influence from governments, the private sector, and the United Nations itself. During the negotiations to establish the Panel, the notion of independence from governments proved contentious. The negotiating bloc of developing countries argued for a tenuous link for governments to provide administrative and governance guidance to the Panel and greater involvement in the nomination of experts. The Global North, however, fervently opposed any channel for potential government influence.

In contrast, the need to safeguard the Panel’s independence from big tech garnered widespread agreement. AI research is heavily funded by industry, thereby creating an incentive structure that is overwhelmingly skewed [PDF] toward commercial applications at the expense of research on societal implications. The Panel is the first global institution seeking to address this gap, and governments want to safeguard experts from any undue influence from big tech. Whether this can be achieved is questionable, considering that four of the panel’s forty experts hold primary positions at U.S. big tech firms, some of which have been accused of suppressing unfavorable research by their employees.

Governments also sought to insulate the Panel from the United Nations. The UN system comprises entities with mandates on digital governance and AI that often clash, as the shrinking pot of donor contributions generates perverse incentives for competition. These incentives manifest in a range of actions, including fighting over mandates across entities, lobbying governments to negotiate positions that strengthen their mandate at the expense of other UN entities, or providing advice based on self-serving interpretations of rules and procedure.

For these reasons, governments envisioned the United Nations’ role in the Panel to be confined to administrative and logistical support rather than furnishing advice or driving decision-making on the working procedures, report content, or other substantive activities reserved for the Panel experts. Additionally, unwilling to vest secretariat support in the UN Office for Digital and Emerging Technologies, member states instead sought to leverage the broader expertise of UN entities with mandates on AI, housed in the Inter-Agency Working Group on AI.

Third, the document that formally established the Panel did not outline the process of defining the rules of procedure and working methods. Governments, unable to agree on an expansive set of operative principles to guide the Panel, instead adopted a vague resolution out of political expediency. Both the IPCC and the IPBES [PDF] designated a plenary—a body comprising all member states—to decide on the rules of procedure that cover conflict of interest policies, report preparation and approval, decision-making, experts’ roles, and much more. The plenaries in both institutions are supported by bureaus, which are primarily concerned with administering the rules of procedure. The Panel’s equivalent bureau, consisting of two cochairs and up to three vice-chairs, must determine the rules of procedure and administer them absent guidance from governments or the founding resolution.

In short, the Panel must define its procedures and working methods from scratch and produce an annual report that is deemed scientifically credible and useful to policymakers, all while relying on support from a fragmented UN system. This is a tall order for any group, but particularly for forty geographically dispersed experts who meet primarily virtually and, for the most part, are unfamiliar with UN politics and procedure.

Under these conditions, political legitimacy for the Panel requires transparency in decisions regarding working procedures, demonstrable independence in how those decisions are made and implemented, and representation of diverse perspectives in reports. Scientific credibility, in turn, will depend on establishing rigorous methods for selecting evidence, reviewing drafts, and ensuring that findings keep pace with AI development.

In pursuit of its equilibrium, the Panel is armed with both expertise and agency. Cochaired by Nobel Peace Prize laureate Maria Ressa and Turing Award winner Yoshua Bengio, it comprises an impressive array of experts from various regions and disciplines, enabling the production of reports that can uniquely combine frontier technological expertise with analysis of AI’s societal implications. Unburdened by extensive government involvement unlike its predecessors, the Panel also has broad latitude to chart its own course on its working methods; its relationship with governments, the secretariat, and other external actors; and, crucially, the form of its outputs.

The decisions made before the Panel publishes its first annual report will determine whether it can be the evidentiary anchor for global AI governance. If it fails to command authority, there are no equivalent institutions with the combination of scope, representation, and gravitas.

This work represents the views and opinions solely of the author. The Council on Foreign Relations is an independent, nonpartisan membership organization, think tank, and publisher, and takes no institutional positions on matters of policy.