Nvidia’s Hugging Face Deal Highlights the Dangers of Open-Weight Models

This week on The Spillover, Sebastian Mallaby and Rebecca Patterson dissect Nvidia’s nearly $13 billion deal to buy the leading open-weight AI platform, Hugging Face, and what the acquisition could mean for the future of AI safety. As Mallaby puts it: “We’re not up against the Terminator. We’re up against an AI swarm.”

The price tag is the first puzzle, because on paper Nvidia’s bid is “an incredible premium.” Patterson outlines that “Nvidia wants Hugging Face enough that it’s offering about $13 billion for a free library,” even though the company’s annual revenue is “around $150 million.” The logic, Patterson argues, is strategic rather than financial. Hugging Face “is the dominant hub where open-weight models are created and downloaded and deployed,” and “if Nvidia can control the hub, it can integrate its own software tools into the models that are successful,” ensuring that those models run on Nvidia’s chips rather than a rival’s.

For Mallaby, the bid is really about Nvidia refusing to stay a chip supplier. Mallaby argues that for years, “Nvidia was the maker of the picks and shovels that enabled the gold rush,” but now “Nvidia rejects that second-rank designation.” By boosting open-weight models, Mallaby argues that Nvidia aims to commoditize the AI models themselves, doing to the proprietary labs “what Wikipedia did to the traditional encyclopedia business.”

The catch is that open-weight models carry serious safety risks. Both hosts agree on this point, with Mallaby laying out two problems: open models “can be modified,” so that “safety guardrails designed to stop you from, for example, making a bioweapon . . . can just be removed,” and unlike proprietary models accessed through an Application Programming Interface (API), “open weight has no off-switch.” Patterson agrees that the risk is real, a concern sharpened by the reported incident in which AI agents “hacked into Hugging Face.” Mallaby maintains that “open-weight models are simply too dangerous to be allowed,” and that “we need AI regulation.”

Whichever version of this future wins, the consequences of the deal will ripple worldwide. Patterson notes that open weight lets a country “not developing frontier models” adopt AI “in a broad way, very quickly, very cheaply,” without the risk of being “shut off by the U.S. or China.” Mallaby adds that for most of the world, “deploying AI is the whole game” and that “deployment is favored by the open-weight future.” Ultimately, Patterson sums up the episode: “This deal will help shape the future of the AI world.”

Mentioned on the Episode:

Aditya Soni, Anhata Rooprai, and Harshita Mary Varghese, “Nvidia Bets $13 Billion on Open AI Models With Hugging Face Deal,” Reuters

Sebastian Mallaby and Sebastian Elbaum, “The AI Trilemma,” Foreign Affairs

Want to keep up with The Spillover? Sign up to receive an email alert when new episodes are released. 

The Spillover is a production of the Council on Foreign Relations. The opinions expressed on the show are solely those of the hosts and guests, not of the Council, which takes no institutional positions on matters of policy.