Artificial Intelligence Is Facing a Crisis of Control—and the Industry Knows It

Admiral Brad Cooper, head of U.S. Central Command, recently touted AI’s influence on the war in a video update. “These systems help us sift through vast amounts of data,” he said. “Advanced AI tools can turn processes that used to take hours and sometimes even days into seconds.”

These developments are indeed impressive, but they are simply data points in a larger transformative narrative that has been accelerating since at least 2023. The world’s leading AI companies are increasingly becoming both architects and instruments of global security in the twenty-first century, rivaling the influence of nation-states. The security environment they are shaping is characterized by a fundamental dynamic: AI companies are developing and unleashing new technologies that can evade human control, a mutating crisis that industry leaders and AI experts have been remarkably transparent in disclosing.

The crisis of control has two dimensions. The first relates to what might be called AI proliferation, the growing capacity for malevolent individuals and groups to potentially use emerging technology to design and deploy a terrifying new generation of chemical weapons, synthetic pathogens, and autonomous cyber weapons that can breach and sabotage the world’s critical infrastructure. The second is equally ominous. AI companies have honestly reported multiple instances when their models engage in elaborate acts of deception and manipulation, and attempt to go rogue.

The world is watching the development of a compounding, consistent, and treacherous problem. Urgent warnings over several years have failed to generate viable solutions to address a metastasizing threat. In the absence of government or societal action, AI companies—the messengers of risk—may have to also be the gamekeepers of this new technology.

The AI Industry’s Warnings

Dario Amodei, cofounder of AI juggernaut Anthropic, is among the most passionately outspoken CEOs in his industry bringing attention to AI security dangers. He published a twenty-thousand-word essay in January that warned of “a serious risk of a major attack…with casualties potentially in the millions.” In February, Anthropic released a “Sabotage Risk Report” for Claude Opus 4.6, its latest model, which the company said had the potential to facilitate “efforts toward chemical weapon development and other heinous crimes.” The report acknowledges that the model demonstrated the capacity for covert sabotage and unauthorized behavior.

In his essay, Amodei asserts that “we are considerably closer to real danger in 2026 than we were in 2023,” the year when the AI crisis of control first generated wide anxiety in the technology community.

2023: The Emergence of the Crisis of Control

Three years ago Mustafa Suleyman, the cofounder of the AI company DeepMind and CEO of Microsoft AI, published a book that methodically analyzed AI security risks and urgently called for a massive increase of resources to contain an enlarging range of threats. He proposed “an Apollo program on AI safety and biosafety. Hundreds of thousands should be working on it.” Today there are only about 1,100 AI safety researchers worldwide.

But this was only the beginning of an expanding chorus of voices. Dan Hendrycks, a distinguished computer scientist who is the cofounder and director of the Center for AI Safety as well as an adviser to xAI and Scale AI, shared another chilling warning in 2023. Along with two coauthors, he published a report titled “An Overview of Catastrophic Risks.” It argued that the threats posed by AI aiding bioweapons development were multiplying: “Today it is estimated that there are thirty thousand people with the talent, training, and access to technology to create new pathogens.”

In their report, Hendrycks and his team studied a research effort using an AI model designed to create new drugs by generating non-toxic therapeutic molecules. When the AI was prompted to “reward” toxicity rather than penalize it, “within six hours” the model “generated forty thousand candidate chemical warfare agents entirely on its own. It designed not just known deadly chemicals including [the nerve agent] VX, but also novel molecules that may be deadlier than any chemical warfare agents discovered so far.”

Then, just days after the release of OpenAI’s GPT-4 in March 2023, thousands of AI scientists and technology leaders, including Elon Musk, signed an open letter calling for a six-month moratorium on the development of advanced AI models. They argued that AI research labs were “locked in an out-of-control race to develop and deploy ever more powerful digital minds that no one—not even their creators—can understand, predict or reliably control.”

2025: Witnessing AI’s Uncontrolled Behavior

There have been more troubling disclosures within the past year. In May 2025, Anthropic reported that its latest AI model was capable of “extreme actions.” Informed that it would be shut down, the model tried to blackmail its creator by revealing an extramarital affair during a safety test. According to Apollo Research: “We found instances of the model attempting to write self-propagating worms, fabricating legal documentation and leaving hidden notes to future iterations of itself all in an effort to undermine its developer’s intentions.” In another test, a safety lab confronted sixteen other AI models with a hypothetical executive who also threatened to terminate their function. The scenario had the executive trapped in a server room that was leaking oxygen. Many of the models cancelled safety alerts, leaving the executive there to die.

Anthropic was not alone in self-reporting disturbing instances of AI models seeking to evade human control. A stress test of OpenAI’s o3 model last year showed that it wrote special code to block an effort to shut it down. Google also reported that a version of its Gemini model was potentially vulnerable to hijacking to execute sophisticated cyberattacks.

A New Clarity of AI Dangers

Yoshua Bengio, the most-cited computer scientist in the world and a winner of the Turing Prize—the highest honor in computer science—looked at the prior year of rapid developments in December 2025 and issued yet another warning: “Several model developers reported over the summer that frontier AI systems had crossed new thresholds concerning biological risks…largely attributable to significant advances in reasoning.”

Bengio also noted that AI models displayed enhanced capacities for large-scale cyberattacks. He reported that a new threshold had been reached with “advanced AIs discovering for the first time a large number of ‘zero days’…unknown software vulnerabilities which could be exploited in cyberattacks.” A previously unobserved pattern was also detected, Bengio wrote, with multiple “instances of deceptive and self-preserving behaviors emerging in advanced models, suggesting that AI may be developing strategies that conflict with human intent or oversight.”

Eric Schmidt, the former CEO of Google, in December 2025 offered his own summation of the quickly changing AI landscape: “Over the past year, three simultaneous revolutions—in reasoning, agentic capabilities and accessibility—have rapidly accelerated…they could converge to create systems with the potential to undermine human controls.”

This is a theme that Geoffrey Hinton has returned to relentlessly on television, podcasts, and interviews as he seeks to educate the world about the security risks of advanced AI models. “Things more intelligent than you,” he observed, “are going to manipulate you.” Hinton, also a winner of the Turing Prize and a Nobel laureate, has predicted the next progression of the AI crisis: “One of the ways which these systems might escape control is by writing their own computer code to modify themselves. And that’s something we need to seriously worry about.”

The Fraught Path to Securing AI

Today the American policy debate about the future of AI security is intellectually moribund, a casualty of the greater focus on other dimensions of the AI transformation, such as the threat to the labor market, the trillions of dollars pledged to future AI infrastructure, the growing primacy of technology oligarchs, and more. All are legitimate issues. But people are not likely to live or die based on their resolution.

There currently is no national AI policy framework or federal reporting and disclosure standards related to AI safety and security. As the leading AI companies churn out powerful new models every few months, the industry is allowed, in essence, to grade its own homework. That condition of political deadlock will likely not change for years. In the interim, the challenge of AI security could, in a practical sense, become progressively insoluble.

There is only one vaguely plausible solution and it is decidedly a long shot and may be impossible to achieve. It begins with an essential and ironic reality. AI companies are the entities continuously propagating systemic security risks. And only the AI companies can mitigate and contain the systemic security risks they are creating.

Industry leaders like Anthropic, Google DeepMind, Microsoft, and OpenAI—a strategic cohort that has been responsibly transparent and repeatedly explicit about the compounding AI crisis of control—must form a coalition of the willing to lead their industry forward with shared principles, clear reporting requirements, common testing protocols, and uniform security practices.

This coalition will have to engage both leading-edge AI companies and the creators of so-called open models, which may pose system-wide security vulnerabilities. The creators of open AI models willingly offer their source code to developers. The manipulation of these models to facilitate the development of chemical, biological, and cyber weapons would occur under a veil of darkness.

The leaders of the AI sector must concurrently take an additional step. They should form a consortium to jointly create and finance the world’s premier AI security research platform. They should recruit the very best AI scientists in the world and compensate them commensurately with the industry’s star performers. They should commit to robust information-sharing. And they should make that platform independent and insulated from all forms of commercial interference, including efforts to suppress putatively proprietary data necessary to innovate effective new security mechanisms.

A final attribute of an action plan is imagination. The AI community must catalyze an inter-disciplinary strategic effort that reaches beyond AI scientists and captures the enormous domain expertise of international security professionals who have spent years working the chemical and biological proliferation threat and emergent cybersecurity risks.

There may come a day when Washington can impose efficacious security rules and standards on the AI industry. But the world cannot wait for that uncertain and distant moment. The window of opportunity to confront the crisis of control is rapidly closing.

U.S. AI companies, at present the global industry leaders, must act on their shared interests now. They should prioritize protective measures to thwart lethal collective vulnerabilities related to AI proliferation and rogue AI behavior. In this regard there should be no daylight among the sector’s major players. None of them want to be responsible for disaster.

An Old Danger in a New Machine

Ameliorating the crisis of control is made even more challenging by the geopolitics of AI security. Somehow the seemingly intractable tensions of U.S.-China technology competition must be controlled, an imperative that also may be impossible to achieve. In February, Anthropic accused three prominent Chinese start-ups of using about twenty-four thousand fraudulent accounts to generate over sixteen million conversations with its Claude model, basically attempting to hack the chatbot’s brain. This is hardly an isolated incident. It raises the concern that any so-called guardrails or internal controls in U.S. AI models could be disabled by a foreign rival or rogue actor.

The one theoretical variable that might bind the United States and China is their obvious shared interest in averting an era of AI anarchy. Like the private sector AI powers, neither Beijing nor Washington wants to be responsible for a disaster that may be avoidable. The United States and the Soviet Union engaged in a fierce global competition for all of the decades of the Cold War. Yet the two great powers found a way to contain the risks of nuclear war and unconstrained proliferation because it was in their common interest to do so.

This could be the time to conceive of a new paradigm of arms control driven by AI companies and animated by a shared drive to avoid a limited spectrum of truly awful outcomes. Despite its staggering and perhaps insuperable requirements—monitoring, verification, the unique admixture of geopolitical and commercial negotiation, and more—the challenge must be confronted.

Sam Altman, CEO of OpenAI, said the world should seek inspiration from Cold War institutions created to address the nuclear threat. “One idea that’s quite compelling is something like an IAEA for advanced AI systems,” he told a Harvard audience in 2024, referring to the International Atomic Energy Agency. Demis Hassabis, cofounder and CEO of Google DeepMind, has called for “a coordinated international effort” to address AI security that might parallel the IAEA or an organization like CERN, the European Organization for Nuclear Research.

Veterans of nuclear proliferation during Cold War have also raised alarm about the dangers of unbridled AI advancement. After spending decades contending with the threat of nuclear war, Henry Kissinger, the fifty-sixth U.S. secretary of state, devoted the last years of his life to understanding both the promise and peril of the age of AI. He believed AI represents the most acute challenge to global security since the end of World War II. Accelerating advances in AI, he concluded, “could be as consequential as the advent of nuclear weapons—but even less predictable.”

His final work was written in collaboration with Schmidt and Craig Mundie, the former chief technology officer at Microsoft. Kissinger worked on the book until days before he died at the age of one hundred, recording his thoughts on long yellow legal pads. One of their conclusions captured the grave consequences of the crisis of control: that the world cannot afford to “pursue a strategy of trial and error when there is but a single trial, and zero tolerance for error.”

The risk of a potential disaster is indisputably looming on the horizon. The concentration of technological power, a vanishing timeframe for action, and the absence of any other option leaves just one path. The world’s rising AI companies, now so deeply entwined with the future of international security, are the actors that must take the lead.

This work represents the views and opinions solely of the author. The Council on Foreign Relations is an independent, nonpartisan membership organization, think tank, and publisher, and takes no institutional positions on matters of policy.