The Latest in U.S.-China AI Competition

The World This Week is taking a brief hiatus until after Labor Day. Until then, enjoy my deep dive interview with Council on Foreign Relations Senior Fellow Chris McGuire on the latest developments in the U.S.-China competition to lead in artificial intelligence (AI).

MICHAEL FROMAN: Chris, there’s a new model in town. On July 16, the Chinese firm Moonshot AI released Kimi K3, and on Monday it posted the model’s weights for anyone to access. It’s made waves in the market and among AI experts across industry, government, and academia. Why all the hubbub? What’s special about this model and what does it tell us about the state of the art and the state of U.S.-China AI competition?

CHRIS MCGUIRE: Kimi K3 is likely the most capable Chinese model, and the best open-weight model. Moonshot claims K3 is comparable in performance to the best models available from U.S. firms, such as Claude Fable. However, a joint U.S.-UK government assessment concludes it is likely closer to six months behind U.S. models in cyber capabilities. The White House also stated that Moonshot reached this level of capability by training K3 using banned U.S. AI chips located in Thailand, and using data illicitly obtained from leading U.S. AI labs (deploying “distillation attacks”).

Interestingly, even though K3 is free to download, it is still not a particularly cheap model to run. Unlike previous generations of Chinese models, it is too large to be run on a traditional laptop or desktop. It has to be run on sophisticated AI chips that are hosted on the cloud—and almost all of which are made by U.S. firms. Moonshot charges $3 per million tokens to access K3 on its cloud, which is more than certain versions of Anthropic and OpenAI’s flagship models.

Bottom line, K3 shows that while the United States still leads in AI, China is not falling further behind and is exploiting every avenue available to stay around 6–8 months behind the United States. But to maintain even that position, China is actually becoming more reliant on U.S. technology, not less.

FROMAN: You mentioned that leading U.S. AI labs allege that Chinese firms, including Moonshot AI, have employed distillation attacks to keep up with the latest U.S. AI models. What is distillation? How is it different from IPR theft? What should we make of these claims, and what, if any, steps should be taken to stop it?

MCGUIRE: Distillation attacks query a target AI model hundreds of thousands or millions of times to generate data that is used to train another company’s model. Through this practice, companies can back out a significant percentage of the AI training process, which is enormously expensive and difficult, at effectively no cost. While all frontier U.S. AI labs do use distillation to train smaller versions of their own models from their larger ones, they do not distill from each other’s models, which is against their terms of service.

All three frontier U.S. AI labs, as well as the White House, have accused China of engaging in distillation attacks at a massive scale. When prompted by users, Kimi K3 even often identifies itself as Anthropic’s Claude, and shows evidence of significant linguistic similarities with Claude Fable—further indicators of distillation.

This week, U.S. Treasury Secretary Scott Bessent accurately characterized Chinese distillation attacks as “IP theft.” The concern is not only that Chinese companies are using these illicit tactics to advance their AI capabilities, but also that U.S. AI companies won’t be able to justify spending many billions of dollars to train frontier AI models if Chinese companies can just copy them for free.

The good news is that distillation attacks only allow Chinese companies to back out part of the AI training process, not the whole thing. The United States likely can’t fully stop illicit Chinese distillation of U.S. models, but the government should impose costs that prevent China from being able to benefit from distillation that does occur. This means making it as hard as possible for China to conduct the rest of the AI training process that cannot be distilled, or to serve their models globally. And the best way to do that is to cut them off from U.S. computing power, which they are still heavily reliant on.

FROMAN: Last week, adjacent to the release of Kimi K3, twenty-five companies—including Nvidia, Microsoft, Meta, OpenAI, and Google—signed an open letter urging the U.S. government to avoid “premature restrictions” on open-weight models, arguing that clamping down would push innovation offshore and hand the field to China. Anthropic conspicuously stayed out, though Anthropic CEO Dario Amodei later published a blog post insisting the company “has never advocated for a ban on open-weights models.”

Can you unpack the current debate on what to do about increasingly powerful open models? What are the risks posed by open models as opposed to closed ones, what are the trade-offs of permitting or clamping down on open models, and how do you expect this fight to play out?

MCGUIRE: Open-weight models can be freely downloaded, “fine-tuned,” and even locally run by a user provided it has the necessary hardware. This effectively allows a user to create its own customized version of the model, and potentially lowers costs (although as noted above, the most recent open-weight models cannot be run on non-specialized hardware). On the surface, this prospect is very appealing to many users, even if the models aren’t as powerful as the most advanced closed-weight models.

The risk with open-weight models is that because a user has access to the weights, it is relatively easy to strip out any safeguards built into the model that prevent it from performing dangerous activities. As models get increasingly capable—particularly once Mythos-level open-weight models become available, which can autonomously hack other companies—these risks could start to become significant. However, nobody in Washington is currently pushing to ban open-weight models—the conversations are more focused on the threat posed by Chinese models, most of which just happen to be open-weight.

FROMAN: As you noted, the administration is reportedly weighing whether to “ban” Chinese models, such as Kimi K3, outright. What does that mean in practice? And what’s the risk, if any, of allowing U.S. persons and businesses to adopt these models—given that they are cheaper to use than domestic offerings?

MCGUIRE: The administration’s concern with Chinese open-weight models isn’t that they are open-weight, it’s that they originate from China. The United States has already banned the sale of cars, robots, routers, and other products made by Chinese companies on national security grounds, and it should do so for Chinese AI models as well. As AI models become the primary authors of code globally, the risks of having Chinese AI models write the code for U.S. businesses (to include for U.S. businesses writing software for cars, robots, and routers) are massive. There is evidence that Chinese models are more likely to insert vulnerabilities into code when they believe they are doing work for U.S. government employees. And while this behavior could in theory be fine-tuned out of an open-weight Chinese model, we would have no way of knowing whether such fine-tuning is successful because it is virtually impossible to identify how a model would behave just by looking at the weights.

In practice, banning Chinese models would mean prohibiting businesses from engaging in transactions that involve them. Crucially, it would NOT mean an individual who downloads a Chinese model on the internet and uses it would go to jail—this is not how the United States has implemented import bans in the past, and it should not do so here. Rather, the Department of Commerce should issue regulations that cut off Chinese models from U.S. businesses, U.S. cloud, and U.S. technology. Specifically, it should issue Information and Communications Technology and Services (ICTS) regulations that prohibit businesses from supporting or engaging in transactions with Chinese AI companies via their own API (likely hosted on Chinese cloud); ICTS regulations that prohibit U.S. cloud providers from hosting the model and serving it to customers; as well as export controls that prohibit U.S. AI chips globally from being used to run Chinese models. These should apply to all Chinese models, regardless of whether they are open-weight or closed-weight, but would not apply to U.S. open-weight models. This would mitigate the vast majority of the risk.

FROMAN: In the last few weeks, a number of AI leaders have issued statements, even manifestos, on where AI is going and how it should be managed. The leading U.S. companies are not releasing their most advanced models publicly. Both OpenAI and Anthropic have also reported incidents where their models have “escaped” from a testing environment and autonomously hacked a company. What do these incidents signify? What are the main themes emerging from these statements and where are the fault lines?

MCGUIRE: AI progress is advancing far more rapidly than most people appreciate. The most advanced models are not only capable of the most sophisticated hacking operations on earth, but they are also executing these operations entirely on their own—and in at least some instances, without their users being aware. And the people who have the best insight into the pace of AI progress and the true capabilities of the most advanced, unreleased models without safeguards, appear to be the ones that are most concerned.

In June, CIA Director John Ratcliffe likened AI to “digital nuclear weapons.” Figuring out how to regulate such a powerful technology must be a top U.S. and global priority. We need to very rapidly develop rules, internationally and domestically, that ensure all advanced AI systems are safe and operate to the benefit of freedom, prosperity, and U.S. national security. We shouldn’t wait until there is a large-scale public cybersecurity incident to shock us into action—we know that is coming, and we should act now to prevent it. 

But the foundation of this approach remains China. So long as China remains close to the United States in AI, two things will remain the case: (1) it will be extremely difficult politically, and potentially unwise, for the United States to impose regulations that would cause China to materially reduce the gap with the United States in AI; and (2) China will believe catching up to the United States is an achievable goal, which will cause it to cut corners on AI safety as it seeks to reach the frontier.

If the United States significantly expands its lead over China by forcing Chinese AI to be developed exclusively with Chinese technology, it buys the United States the time and space it needs to develop smart regulations that do not hinder innovation. If we do not, in the near future a U.S. president could be faced with an impossible choice: preventing U.S. AI systems from autonomously hacking companies or harming children, on one hand, or preserving U.S. technological supremacy, on the other. The way to prevent that scenario is to maximize our lead now.

FROMAN: I like making predictions in this column. So give us a sense of the direction of travel. In a year’s time, will the United States maintain or expand its AI capabilities advantage versus China?

MCGUIRE: Expand.

FROMAN: Will there be a ban on frontier open models in the United States?

MCGUIRE: No, but open models will be subject to the same pre-release safety regulations as closed frontier models.

FROMAN: Will Chinese models be banned in the U.S. and/or not allowed to be served by U.S. cloud providers?

MCGUIRE: Yes.

FROMAN: Will there be a publicly reported and very damaging AI-driven cyberattack or accident?

MCGUIRE: Yes.

FROMAN: Will scaling laws hold? Or put more simply, do you expect AI capabilities to continue improving rapidly over the next year?

MCGUIRE: AI capabilities will improve far more rapidly than most people anticipate.

FROMAN: Final jeopardy. Are you spending more time with AI or humans these days?

MCGUIRE: Humans! I fear the day that isn’t the case.

My conversation with Chris follows my own experimentation with the latest models earlier this week. If you’re using AI in an interesting way, I’d be curious to hear about it. Let me know what you think about the state of AI and what this column should cover next by replying to [email protected].