U.S.-China AI Race Escalates + Chip Bans Aren’t Working + A Lesson From Nuclear Proliferation

MALLABY:
I’m Sebastian Mallaby. Welcome to The Spillover. Each week, we consider topics in technology, geopolitics, finance, economics, and we try to connect the dots as new developments arise.

Last week, for example, we discussed the war in Iran and how that spills over in everything from the price of food to the price of energy to monetary policy in the United States. And this week, we are going to take a look at technology policy, the geopolitics of AI, the race between the US and China. And we are going to think about it in a couple of ways.

We’re going to think, first of all, how it seemed when ChatGPT first came out in 2022. We’re going to take a look at how the debate might have changed in 2026. And then we’re going to think about how it might look two or three years from now, when perhaps the Nuclear Non-Proliferation Treaty could be a model for how AI safety could be negotiated between nations.

Now, unfortunately, Rebecca Patterson, my normal co-conspirator, co-host, is traveling this week, and so she can’t be in this discussion. But instead, we’ve got Chris McGuire, who is a senior fellow at the Council on Foreign Relations, my colleague. He covers emerging technologies and China.

The good thing is that I have enormous respect for Chris’s intellect, but I’m not promising to agree with him. So that should make for a fun discussion. Before we get into it, Chris, I think it’s important for the listeners to understand a little bit about your background.

And before you came to the Council on Foreign Relations, you were working in the Biden administration’s White House on technology issues, right?

MCGUIRE:
That’s right. Immediately before I was in the Trump administration, actually, because I was a civil servant. But I was a civil servant for the last 10 years before coming here and served for most of the Biden administration in the Biden White House, initially in the Office of Science and Technology Policy.

And then I spent almost three years at the National Security Council, including as the Deputy Senior Director for Technology and National Security. So I had the broader China tech portfolio throughout the Biden administration, and then kind of covered the directorate that did everything on semiconductors, AI, quantum, biotech, kind of across the board on both the promote protect side. But obviously, yes, I spent a lot of my time on the China pile and on the protect side.

MALLABY:
And I think I’m right. You also worked maybe further back in the past on nuclear security, nuclear arms control, and you negotiated with Russia.

MCGUIRE:
Yeah, that’s right. So I came to that technology because it was the kind of strategic technology of the era when I when I grew up. And I think kind of that was the logical places you were thinking about sort of grand strategy of kind of where to go.

And I think I pivoted partly, I pivoted into emerging tech partly because we became clear like this is the strategic technology of the coming century in the way that nuclear was of the past century, not that they’re the same in any way that but just insofar as the opportunity to think, you know, large scale and deeply about how technology and these kind of sophisticated and technologies impact our kind of broader security discussions.

I think that there’s there’s a lot of opportunity for new thinking and emerging tech in the ways that there wasn’t like the 50s, 60s, 70s, 80s in the nuclear age.

MALLABY:
Right. And the lessons that we can perhaps draw from the history of nuclear negotiations and arms control, maybe are going to be relevant to the future of how we think about AI control. Or at least I hope so.

We’ll get into that. Yeah, I want to start you off there. Here.

Go back to 2022. It’s November, and ChatGPT comes out. And all of a sudden, artificial intelligence goes from sort of the fringe of people’s consciousness to the mainstream.

And there you are, it’s your file, you’re doing emerging technologies. And this one just emerged big time. So talk a bit about how you and your colleagues in the administration, late 22 going into 2023.

How did you how did you kind of rank order the risks from AI? How did you set priorities just kind of paint that picture?

MCGUIRE:
Yeah, I think actually, it’s helpful to take a step back one more year and think back to kind of 2021 and 22. And the reason for that is that we had a theory in the in the Biden administration that basically the scaling laws were going to hold scaling laws, meaning meaning that AI would get way more powerful, would get way more powerful. And also that computing power was going to get way more important.

And basically, the more compute, the more computing power you put into an AI model, the better it’s going to be. Obviously, there are other advances same time. But what that means is that number one, AI is going to get really good really fast and on an exponential, which means it might seem like it’s only okay right now.

But it might not be that long until it’s extremely good. And then number two, the computing power and the chips that go into it. And ultimately, the aggregation of them in kind of large data centers was going to be extremely important in the development of this technology.

And also because this is something where the U.S. dominates in kind of the ability to control the technology. So you asked, Sebastian, what the what we saw is the risks. I mean, if that exponential growth happens, which was our theory at the time, it was just very clear that that the kind of commoditization of intelligence and the ability to build this supremely powerful model that can that can think as well as humans, even at niche tasks, let alone at every task was going to be extraordinarily valuable.

And particularly for national security, that the applications were going to be everywhere. Right. I mean, it was not be hard to have an AI make every kind of back end logistical decision in war for you, make better battlefield decisions and commanders.

Obviously, eventually you’ll have autonomous systems that are piloted by AI. I think the cyber risks were ones that were that were very significant because just you’ll be able to whoever has the better AI is going to be able to have to have the offense defense advantage in the cyber realm. So just it became very the more you think about how this big change is going to be applied to the national security space.

And that’s not even to mention the tremendous economic benefits that come from being in the lead. And then also all the challenges that we face on the kind of economic transition front, like that being as far ahead as possible in this was going to be extremely important.

MALLABY:
I realized I misremembered. I knew that the chip controls were imposed quite some way back. Were they actually before ChatGPT came out?

One month before. One month before. Yeah.

So so you were really I mean, it’s worth just underscoring here that this was quite some foresight because although the scaling laws were believed by people inside the labs, they were not really necessarily believed outside. I mean, people who are building the systems at Anthropic or OpenAI or Google DeepMind or Google and DeepMind because they were separated back then. They saw that this scaling was happening as you added data, as you added compute, the systems just got a lot stronger.

But it wasn’t universally accepted outside. So first, the first thing to underscore here is the administration was kind of going with a trend that not everybody recognized. And you were aggressively stating to where that you thought the puck was going to be before maybe others understood that it was the puck was going over there.

MCGUIRE:
Yeah, I think that’s right. And I will say there were people that were varying levels of convinced of this inside the administration. There were some people who were 100% convinced the scaling laws were going to hold.

There were some people I think I might even put myself in this camp that were thinking it was a strong possibility, but not at all a guarantee. But the way that all of us saw it was the return on investment from this policy is very positive, no matter what.

MALLABY:
The policy being the export controls.

MCGUIRE:
Exactly. So in the world where AI chips matter a lot, this is very obviously a good policy. And I think that’s why we’ve seen a lot of consensus around that.

MALLABY:
So the thing here is you see this technology coming, it’s going to, or at least there’s a high probability that it may well become exponentially more powerful. And you figure out that the bottleneck or the choke point that you can use is the chips because the chips are designed by NVIDIA primarily, a U.S. company. They are built by TSMC, a Taiwanese company.

You need equipment from Europe. The Dutch company ASML provides the lithography. So you think, okay, this is a supply chain, which China does not have.

And if we deprive China of the cutting edge chips, that will give us a huge advantage in this intelligence explosion.

MCGUIRE:
Yeah. And the foundation is really the equipment. Because it’s not just ASML, obviously ASML being one of the probably the most complicated, important part of the supply chain, but also all the other steps in the process and, you know, applied materials, KLA, LAM.

There’s very large companies that are very good at this. There’s five companies in the world, Tokyo Electron from Japan being the fifth, that produced the vast majority of advanced equipment. So the U.S. and our allies have a very strong control over this. And specifically on a regulatory side, we realized that we could say any chip that’s made with U.S. tech is controlled. So it’s actually, even if it wasn’t a U.S. company designing the chips, because it requires U.S. tech as an input, and it is impossible to design out U.S. tech from the advanced semiconductor supply chain, given how integrated we are, then it gives us the regulatory hook to control the chips. And we actually built, this was a device that the Trump administration discovered with Huawei in 2020, when they were trying to combat Huawei’s ability to spread 5G base stations around the world.

And they realized that they could cut them off from TSMC by using this hook and to say, because you need U.S. tech to make the chips at TSMC, we will block you from doing so. And we took that logic and then expanded it. Instead of saying one entity can’t get any chips, we said the whole country can’t get certain types of chips, obviously not every chip to China, but just certain types.

So this was really building off of something that was a pretty bipartisan effort. We just kind of expanded it into a different direction.

MALLABY:
Okay, that’s a great scene setting. So that was 2022. I remember, by the way, writing a very long essay in the Washington Post supporting the policy.

So it made total sense to me at the time. Thank you. But now that we’re in 2026, and I’ve just been to China, although it’s not just because I’ve just been to China, I’m starting to have doubts.

And really, the doubts begin with the beginning of 2025, when DeepSeek, the Chinese AI company, produced what was a pretty impressive model. Now, it was said to be incredibly cheap. I don’t really believe that bit of the story, because I think the cost of the process that was being measured was not the whole process.

I’m going to put that one aside. It’s never mind about the cost. But what I do think was, you know, impressive about it was that notwithstanding the semiconductor export controls, which by then had been in place for what, three years or something, it was possible for China to produce very good AI.

And so by that measure, it felt as though it was a blow to the policy. Tell me how you thought about it.

MCGUIRE:
Yeah, I think that DeepSeek, I don’t think surprised me. And I don’t think it surprised a lot of the people who spent a lot of time working on this, for many reasons. I mean, the first and foremost is, AI is not at the point when you would be unable to make a model that is moderately advanced, but, you know, using relatively small numbers of chips still get to a pretty good model, right?

I mean, I agree with you that I think the numbers of chips that were used to trade DeepSeek that they claimed was way too low. And the actual costs, there’s been private speculation, probably more in the realm of like $500 million. But $500 million is an effort that like a state backed campaign could support.

And the thesis with the controls was always that it was kind of boiling the frog that as the exponential got more and more punishing, right, and as you needed more and more chips, and it was more and more expensive to build and train a frontier model, then the cost of being six months behind was also going to increase commensurately.

MALLABY:
Right. But other labs in China have come out with new models.

MCGUIRE:
They have.

MALLABY:
I think you could read DeepSeek’s failure to follow through as more like, well, you know, this is a hedge fund that built as a kind of side project, an AI model, right? But they were not, you know, the frontier now is with Alibaba and companies like that. Right, Dan?

That’s probably right. That’s probably right.

MCGUIRE:
But I guess I would say just a few things to kind of sum up, like there’s the export controls do get more punishing over time. So the fact that there was the Chinese were able to have something that was still good back in 2025, didn’t inherently surprise us is number one. Number two, there are some loopholes in the controls, admittedly, like there are still ways that they can smuggle chips.

These were things that we’ve tried to close over time. I’m still quite concerned about obviously Chinese smuggling of chips is now very much in the news. And I think there’s ways that we have to crack down on that.

But that doesn’t mean that the logic that is underpinning the strategy is incorrect at all. The third I’d say is that you listen to all the Chinese companies, right? You listen to DeepSeek’s leadership.

You listen to the Chinese national government leadership. You listen to all sorts of quotes from Alibaba. All of these companies say the same thing, which is that they’re compute constrained.

And the world in which they have access to more compute is a world where their AI is better. Now, where is it relative to the US? It’s a little hard to say, and I’m not exactly sure, but it will certainly be better than it is today if they had more computing power.

And the reason they don’t have more computing power, the reason they don’t have the most advanced chips, the reason they don’t have the biggest data centers in the world, China can’t scale this technology to the degree they normally can with basically anything else is because of the export controls. If without the export controls, they would scale massively, they would benefit from massive state subsidies and just a big aggregation of resources into developing some of the largest data center supercomputers in the world. I have no doubt of that.

MALLABY:
They’ve definitely scaled the energy supply, so why wouldn’t they scale the chip clusters? Exactly. Let’s just double click on one thing you said.

You said the smuggling. As I understand it, and I think maybe I understand it because I talked to you in the past, there is also the issue that if you are building a Chinese AI model and you want to train on a large compute cluster, you might be able to just use a cloud, which is not in China. And you can disguise your identity as a Chinese model maker, you have some kind of front account, and maybe you can even train on an American cloud.

You can.

MCGUIRE:
There’s nothing restricting Chinese companies from training on American cloud. DeepSeek could train DeepSeek v4 100% on Microsoft or Amazon or Google Cloud, and no one would be violating the law.

MALLABY:
Was that an understatement? Just wait a second. Isn’t that like a total negation?

You’ve got a loophole, which is sort of bigger than the enforcement bit, no?

MCGUIRE:
At this point, from where we see AI is and how capable it is, I think yes. I think at a certain time, you could make the argument like, well, it’s better for them to be on our cloud than doing it themselves. Maybe there is an advantage to having them on the cloud, because it means that you can shut them off at any given time.

This is the debate of like, well, if we see AI getting super powerful, then we’ll stop. If we were to export them chips, then we could stop exporting chips. There’s actually a lag on that, right?

There’s gonna be a lag of a few years. But if you knew, for instance, that every single thing they were doing on it was on American cloud, and you could press a button and shut that all off, that gives you a little bit more control.

MALLABY:
Yeah, but could you push the button? I mean, if you don’t know which accounts training on your cloud are the Chinese… It’s certainly complicated.

MCGUIRE:
There actually were some restrictions that the Biden administration put in place just before they left on this type of activity that the Trump administration pulled down, and they haven’t issued any kind of replacement for that. So I think that if we do think, and we can get into this when we talk about nonproliferation regime later, but if we do think that this is a technology that it’s important to kind of protect globally our advantages, I think there are ways that you can do that. But I would also acknowledge that the regime has loopholes.

MALLABY:
Just to close out the discussion on sort of the enforceability of the original export controls. I mean, because it seems to me that there’s kind of three classes of objection to the export controls, with the benefit of four years of hindsight, right? One is, well, they’re not really enforceable.

And that’s partly to do with smuggling. It’s partly to do with the use of American cloud compute, which is a sort of different kind of loophole. And some of it feels predictable, that actually, it turns out there are only kind of three people in the US government who are tasked to look at goods going into China that might be breaking some kind of sanction or embargo.

So the point here is that if you create this swing, where a bigger share of the global compute is in the US, even if Chinese model trainers are accessing that compute through these front names, it’s still a win, because it’s better to have the compute in the US where a US cloud provider will obey US law. And if you wanted to clamp down, you could. Yeah, we can say it’s insufficient, but it’s back.

So the compute, the shifting compute is good in and of itself, even if the models in China are pretty good. I mean, you know, we can debate how many months they are behind. But to me, I would say, look, if they are three months behind, six months behind, doesn’t matter.

That’s a pretty short time. And the bigger stretch in kind of AI capability, kind of deliverable AI in the real world, whether that’s in the real economy, or whether that’s in the defense capacity, the delta in terms of how quickly you build the application, how cleverly you build that into a real workflow, may be bigger than three to six months. So being at the frontier in terms of the base model, in AI, may be less important than how quickly you build the application.

MCGUIRE:
Depends on how big the gap is. But I think that’s right. You have to take into account both the lead that we have, and then also how quickly each can go to go to market, so to speak.

I think if you’re at an 18 to 24 month lead, all of a sudden that becomes there’s no way that there’s a kind of applicability gap that can make up for that.

MALLABY:
But we’re in the three to six month world. We’re not in the 24 month world. But I guess you’re holding out some hope that in the future as compute demands exponentially grow, the export controls could bite and then you might stretch the lead to 24 months.

Is that the point you’re making?

MCGUIRE:
I think they are. I mean, they are biting now, right? Like if China is equal to us and also able to apply much faster, we still are, we will be in that world.

We’re behind China, right? Because they’re at the same level of technology and they’re able to deploy it faster. So like that is that is a even taking into account, they might have some advantages on application and we’ll see how much that actually manifests.

But like being ahead of them in the technology actually has inherent value.

MALLABY:
And talk a bit about how you see the Trump administration’s changes to the controls that you put in place.

MCGUIRE:
So I think it’s been incoherent is what I’d say. So on the one hand, the Trump administration initially strengthened some controls on China by controlling the H20 chip, which I think was an oversight.

MALLABY:
This is an NVIDIA chip.

MCGUIRE:
Right. That’s right. In April 2025, there was an NVIDIA workaround chip that the Trump administration that the Biden administration had not blocked, the Trump administration did block.

And then they then about faced on that in July 2025. And then in December 2025, agreed to sell the H200 chips to China. And then subsequently you’ve seen a lot, there’s been a lot of noise on this policy and not a lot actually happening.

I don’t think there’s any H200s that have gone. It seems they have approved some, probably fewer than people think or than the companies would wish rather. I have no doubt that there will be H200s that go, which is unfortunate.

I don’t think that there’s any upside to that, particularly as you’re seeing even this whole debate aside, which we could have ad nauseum. The fact is all of our companies are extremely compute constrained. The compute supply chain is stretched incredibly thin.

Every single US company says that this is the number one thing they need. It’s at the point when it’s actually impacting consumer prices because there’s so much memory that’s going towards AI chips that actually there’s no memory capacity left for other devices like smartphones and TVs and computers. So the prices of those things are going up.

So at this level, selling to China seems much more difficult to justify. So they’ve kind of swung back and forth. And I think where you are right now is there has been a move to sell very advanced chips to China, which would be much, much more advanced than anything that the Biden administration was willing to sell.

You probably will see some of those. That will be a massive boost to China’s capacity. I mean, we’re talking probably as much as they could make in each year domestically with just that single amount.

But it depends on how much they order. The bigger thing is just there doesn’t seem to be a kind of coherent strategy behind why we are doing this and what our semiconductor export policy is. We haven’t seen any updates to the controls on equipment, which the Biden administration updated regularly.

The Biden administration also put forward this global approach for how we’re going to regulate, how we’re going to make sure that AI data centers globally are trusted and making sure that they’re not covert Chinese cutouts. The Trump administration has decided not to enforce that, but they have not replaced it. So it is actually still technically on the books, but it is not clear what they’re going to replace it with, if anything.

So I think that ultimately there is not clarity on what the policy actually is. Also, the AI action plan specifically says AI compute is a strategic resource that we must prevent our adversaries from obtaining. And at the same time, we’re selling chips to China.

So my personal belief is that we are in this weird period of flux, but as the technology gets more and more important and it just becomes viscerally obvious that this is the most important technology in the world, the viability politically or even kind of logically of selling AI chips to China is going down every single day. And it probably will get to the point when I think we are going to see even more separation of our supply chains in the near future. The sales to China of H200 is not withstanding.

MALLABY:
Well, we definitely agree that if the technology continues to exponentially improve in strength, the visceral obviousness of needing to do something about it is going to be very strong. So we agree on that. I just mentioned that when I was in China, I went to see Huawei.

The first object of US technology sanctions in China. And I think basically the firm lost access to advanced chips and their phone business was pretty much knocked out for a bit. But they bounced back.

And it’s true that today’s Huawei phone is probably less powerful than the four years ago because they lost access to imported chips. They had to build their own chips. Their own chips aren’t as good even with four years to try to get that right.

They’re still not back to where they were before, but they have tricks. Like for example, you take a photograph of the Huawei phone and it’s not beautifully sharp and color balanced initially. But you take your phone out an hour later and the photo has been uploaded to a cloud and edited on the cloud because they don’t have enough chips to put in the phone to do it beautifully immediately.

But they can have the phone communicate with cloud backup. And I interviewed Ren Zhengfei, the founder and CEO of Huawei. Not sure interview is quite the right word.

It was sort of a meeting. It went on for two hours and he went on at some length about how stacking chips in a data center can get around not having as many cutting edge chips. You can’t stack them in a phone because that’s a small space.

But in a data center, you’ve got plenty of space. Is there anything, any truth to the idea that the bottom line is they still produce a phone which people want?

MCGUIRE:
I think I would completely agree that the of the stacking point and that ultimately what matters is how many chips you can get together, especially in the AI point. But I think there are fundamental differences between the phone analogy and the AI analogy. With the phone, it is easier to get to a good enough.

I think that especially at seven nanometers, they’re not as far from the frontier as they are with an AI chip. The reason is actually phone chips are much smaller. They’re much easier to make.

The quantity of AI chips that they could make at seven nanometers is much more constrained because their failure rates are going to be much higher. And these are just much more sophisticated chips and devices. I agree, though, that I think they can do it and they can do some.

And the question ultimately is how many. And even given that, because of their quality differentials, and they do face constraints on quantity, they can’t make unlimited quantities of seven nanometer, 14 nanometer chips because there are constraints on the tools. The estimates that I put out are that for Huawei to even assuming the most generous assumptions for their production capacity this year, they’re probably going to be at four percent of Nvidia’s production in terms of AI compute.

So even if they do stack all the chips together, they’re just not making enough of them.

MALLABY:
And the reason is because the quality is not that good. Just to clarify, you’re making the Huawei to Nvidia comparison because Huawei kind of aspires to be the Chinese Nvidia.

MCGUIRE:
Yeah, that’s right. I mean, actually, those are the two leaders of both countries. But just for the sake of simplicity, if you just compare Huawei and Nvidia, they just can’t make anywhere near enough chips.

MALLABY:
And the reason is because their chips aren’t as good. So you said four percent, so 25 times bigger in the US.

MCGUIRE:
That’s right. And the gap is getting bigger because their best chip right now is about five times worse than Nvidia’s best chip in two years based on Huawei’s own internal roadmap. That’s going to be 17x.

So the gap is widening significantly in quality. And China just is not able to ramp up production quantity enough to the point where they can be competitive. And then the other exponential that’s working against them is the amount of AI that it takes to train and also run, or the amount of chips that it takes to train and also run AI models.

It’s not impossible that by 2027, 2028, China will not have enough compute in the country, assuming we don’t give them a lot, to be able to conduct a frontier training run in any reasonable amount of time. Obviously, if you ran something for two years, then you could theoretically aggregate enough compute. But the constraints on this as the amount of compute needed for advanced AI models is going up and up and up.

And our companies are facing this to some degree, but the Chinese are facing it to a much, much more punishing degree because they just are not as good at making the chips domestically. Their supply is just nowhere near as good as ours.

MALLABY:
Right, right. Let’s draw a line under this part of the discussion. I mean, bottom line, we’ve got a situation where the Chinese labs are producing decently good models.

They may be, you know, three to six months behind the US, but they’re decently good. And yet, there is at the same time, if you look at just the chips, not the models, it feels like that’s biting. Okay, so but I want to get to a second, I think, interesting debate, which, to me, wasn’t obvious in 2022, when the export controls were put in place.

But as I’ve, you know, thought about AI more, and, you know, written a book about AI, I’ve come to think of this as part of what I might now second guess about the policy. And that is that I feel like there was a sort of implicit assumption, maybe not in the administration, you can speak to that. But I think certainly in American AI labs, that what really matters in this race is to get to a point of kind of recursive self improvement in the models, meaning you use AI to write code to improve your AI.

And that if you could get to that, you’d have a sort of intelligence explosion. And whoever got to the point of explosion first, where your progress just goes vertical, game over, you’ve won. And I mean, I talked to one of the heads of the major labs, I won’t say which one, who said to me, look, you know, even if the controls don’t last, even if they leak, it doesn’t matter, we just have to slow the Chinese down for like three or four years, and we’re done.

This will be over. And he was saying this, you know, some, maybe two, three years ago. So that was the sort of expectation.

And if you now evaluate that projection, in some ways, it’s true, right? We do now have AI writing the code for AI, you know, all of the leading labs are using their own internal models to produce a lot of the code that’s generating the next generation. But I’ve come to think of this idea of an intelligent explosion as sort of just way too simple.

In the real world, in order for AI to do stuff, you have to deploy it into a certain environment. You know, let’s say it could be a law firm’s workflow. And you need to get the clients of that law firm to agree to using AI on their data, you have to make sure it’s not going to be hacked, you have to, you know, sign off a whole bunch of complicated agreements.

And it’s just going to take a long time. The intelligence explosion story where once the code is improving itself, it’s done. No, it’s not done, because there are all these other real world constraints on the deployment of AI.

And so I just, I just feel like the understanding of the race, that, at least in Silicon Valley, encourage people to support the export controls was wrong. What do you think?

MCGUIRE:
I think that the way that we thought about it was actually never that kind of pin to the intelligence explosion, or some people say it only makes sense if they’re you think about an context or things like that, that actually is not exactly how we thought about it. I agree, there are some people who think about that way. But our thinking was, if you’re going to have very powerful AI models doesn’t matter exactly, you know, AGI or not, whatever, like, it’s just going to become increasingly important.

We want to maintain this is exactly what Jason said, we want to maintain as large of a lead as possible over China in AI. And that should be the enduring US policy and our kind of North Star on on what we’re trying to do. And that I think they’re, that is hard to argue against, in some ways.

No matter how far ahead we are, it is better than if we were if we were closer. And if there’s things that we can do to slow them down, we should do those things to slow them down. I mean, if there’s, if there’s clear reasons why slowing them not by slowing them down actually hurts us, then we can talk about that.

But I think I would dispute most of those arguments. And it’s actually not really something we’ve been discussing today. So I think that regardless, it’s it’s helpful.

And yeah, I would agree with you that that the intelligence explosion point is actually obviously more complicated. I mean, like one example that we’re seeing now is this idea of distillation, right? So the Chinese can actually use our advanced models to make their models better in ways that allow them to be less reliant on compute, and also leverage the fact that our models are so advanced, which is this like factor that actually inherently works against the benefit that we would get from recursive self improvement.

Because even though our models are going up at an exponential pace, the ability of the impact that they’re getting from distillation is also increasing exponentially. So yes, the gap might widen between us as recursive self improvement kicks off, which I think is what we’re seeing now. But it’s not as if they’re going to stay flat, and we’re going to kind of immediately go to the stratosphere, which is all the more reason that you want to tighten the rest of the regime.

MALLABY:
Right. And just so listeners understand the distillation idea is that if you’re trying to train an AI model, on some particular type of skill, the old way was you had to go find data to train on. And so if you were teaching it to write or something, you might download a bunch of books and train on that.

But you need, you know, as the models became generally good, but needed to have more advanced specific abilities, let’s say in, you know, complicated physics or something, you were actually paying PhD physicists, you know, real amounts of money to put together problem sets, show equations, show how they would be solved. You needed all that stuff to train on. And once that’s, once you have a model that can do all that complicated physics, then the follower, in this case, the fast follower being China, can simply come to the good physics model, and query it and get a bunch of outputs that provide all of the training data that you need.

You don’t have to go hire a PhD physicist or even an army of them. And so you’re, the ability for the follower country to stay close to the frontier is actually higher than one would have expected without distillation, right?

MCGUIRE:
Yeah, potentially, there’s many factors that go into it. But this is one that allows them to fast follow a little more easily. There’s other elements of the process through where this actually doesn’t provide benefit.

It’s not as if the entire training and certainly not inference process you can solve with distillation. But there are elements that you can kind of fast track with this. So that’s important as we’re thinking about how do we kind of comprehensively slow China down?

MALLABY:
Yeah, yeah. So let me move now to what I think is the third objection to the controls. And I think this is actually the most interesting part of the discussion and probably the one where we’re going to disagree the most.

Let’s say that the chip controls are making some impact on China. They do produce great models or pretty good models, notwithstanding these controls, but still there are constraints in their ability in the future. You know, all the points that you’ve made.

Okay, that’s all good. But what if there was a cost and you raise this point yourself, what if there was a cost of these controls, and then you have to think about the cost benefit of the policy and the cost it would seem to me and this is, you know, I don’t think you’re going to love this kind of counterfactual, but I’m going to give it to you anyway. The cost is that look, when you were sitting there in 2022 and thinking to yourself, okay, we see the scaling laws, we see AI is accelerating in power, we want to try to make sure that AI doesn’t threaten US national security.

I think there were two kinds of national security threats you could have prioritized. One was, you know, well, if China gets it, that’s bad. And that was the one you chose.

Second possibility might have been what about rogue states, terrorists, general proliferation of this technology? The more people who have it, you know, the more likely one of them is going to be crazy and attack us with it. The key point here is that you kind of needed to choose because if you make China the enemy, and you say we’re going to deprive China of chips, then obviously, they’re not going to collaborate with you on a non-proliferation regime.

If you had alternatively gone to China and said, look, you have amazing technology companies, we have amazing technology companies, we’re both going to get this technology. But we should jointly control how it diffuses around the world. You could, I think, perhaps have recruited China into an alliance where you controlled the other kinds of risks from terrorists and so forth.

And you didn’t choose that path. I think you had good reasons, but maybe you leave them out for us.

MCGUIRE:
Yeah, so this is a fair point and one that I think a lot of people, especially in the AI safety community, I think talk through. And we thought through this and I’ll give you the exact reasons why we decided not to pursue that. The world where we are jointly managing this technology with China is profoundly more risky than the world where we have a significant US advantage.

First of all, we thought we had the ability to get to that world. And have enduring US advantages, which is good. And second, I don’t think that we thought that the Chinese would, in good faith, work together with us up to that point.

The Chinese see arms control as something that the United States did to the Soviet Union to get them to lose the Cold War. And it is a way for the Americans to kind of keep other countries down. And it’s not a way to pursue joint interests here.

And that is incompatible with this idea of, we’re going to kind of work on this together. And fundamentally, the United States and the Chinese government would probably both use AI to advance their vision of the world. And we just see the degree to which we see, especially on the inside, how the Chinese government views the United States as their number one adversary.

And the degree to which they are not just competing with us, but kind of actively combating us in cyberspace, especially in intelligence operations, in kind of gray area conflicts around the world, is diametrically opposed. So this idea, and especially in conjunction with the attitude where they’re not going to be receptive to this arms control pitch, there just was no chance of that. And I think that we would just end up kind of giving away our technology and then losing in the end, like we have with so many other technologies.

So we chose a different approach.

MALLABY:
Yeah. I mean, look, as a student of history, I’m attracted to the argument that, you know, historical experiences influence how people behave and how countries behave. And so, you know, the US invented, you know, the atom bomb, used it, saw how devastating and terrifying it was.

And, you know, the Cuban Missile Crisis, as you say, crystallized that, the downside risk of nuclear weapons. And China just didn’t have any of those experiences. In the 60s, it was focused on its own politically generated internal catastrophes, you know, the Great Famine and so forth, and the Cultural Revolution.

And those are the muscle memory things for China and to them, technology to the contrary, is what kind of got them out of, you know, being essentially an unsuccessful state. And technology has been intrinsic to the acceleration of Chinese growth in the last 30-40 years. And so they love it instinctively.

So I get all that. But as I went on from Huawei, and I went to Beijing, and I talked to partly academics, but also other lab leaders, and I went to Hangzhou and saw some, you know, company CEOs, I got a very different picture, where, you know, alignment of the models with human intentions was a big debate. So what I’m saying is that I didn’t experience the Chinese debate as being quite as closed towards the importance of governance as the historical muscle memory story would imply.

MCGUIRE:
When the US had the, we developed a permissive action link technology, which was basically something that you could put on a nuclear device. And that you could put in the code or something to arm it, such that that would make it less vulnerable to theft. That was something that we did have some quiet cooperation on with the Russians, because it’s in both of our interests to make sure that our weapons are only used as intended, and that they aren’t stolen, and rogue actors inside the system can’t execute a nuclear strike.

So even though we were in kind of diametric opposition on many fronts, on certain safety related things, we were actually able to work together. And I think there’s, you know, ideally, we could get to a point with the Chinese in a similar point with AI. I think the question is really like, what’s the best path to get there?

Is it to just kind of join hands and work together?

MALLABY:
I mean, maybe this is not an analogy that you think sort of is that relevant. But it strikes me that, you know, if I think back to debates in Washington on climate change, you know, let’s say, in the 2000s, and a bit after, the story was always, you know, the US shouldn’t bother with climate change. I mean, at least the story for one half of the political spectrum.

Because frankly, the Chinese building all these coal pad, you know, electricity generators, and they’re building so many of these things, it just swamps whatever we’re going to do. And so, you know, just look at China, it’s just revolting what they’re doing. And so why should we do anything?

And now the truth is, you go to Shanghai, as I just did, and the air is clean, because you’ve got electric vehicles all over the place, the electrification, use of solar power, all that stuff is quite successful. No doubt, they’ve got, you know, they’ve pushed some of the pollution to other cities and so on. So I don’t want to overdo this claim.

But it’s not like China has done zero on climate change. They’ve moved their energy for all kinds of reasons, energy independence being one of them, they have done quite a bit. And if you count the value of the exports of solar panels, as a contribution to global reduction in emissions, it’s quite significant.

And if the Europeans buy EVs, by the way, that’s also significant. So I think the reading of China on climate as being a total renegade feels wrong in retrospect. And in the same way, the assumption that China is impossible to talk to about AI safety also felt wrong when I was there.

MCGUIRE:
I completely agree with you that like, no one’s saying we shouldn’t talk with the Chinese about these issues. And obviously, we should, it’s a question of how willing they are to engage. But on the energy point in particular, like, obviously, yes, the percentage of Chinese energy production that is renewables has gone up over time.

But in absolute senses, the China’s coal energy burn, the percentage, the amount of coal powered energy that China has online each year is still going up. So yes, they are producing solar panels, but it’s just part of an all of the above strategy. They’re also burning more coal this year than they did last year.

And they’re just trying to ramp energy as high as possible. So I just completely agree with you, they’re doing some but like it shows you the kind of broader intentions. Like they’re actually still the problematic player on on coal and climate in some ways as well, even with their contributions.

This is why it’s hard to figure out ways to kind of get to the these kind of shared goals. I think that’s an interesting example.

MALLABY:
Yeah, no, I obviously I get that. And I agree with that. I’m just I suppose I’m reacting to the common discussion I hear in China about China in Washington, right, which is basically more nuanced.

This is, you know, a soul crushing dictatorship, a total monolith. There’s only one line there. And when you go there, you realize it’s actually, you know, a complex big country with lots of people with different opinions, and caricatures don’t quite work.

So we’ve talked about the, you know, how things felt in 2022, how things might feel today in 2026. And then now looking forward a bit to whether a non-proliferation treaty is a useful mental model for how we might think about AI, because I am concerned with proliferation. And I’m especially concerned with open source.

I mean, I think we haven’t talked enough about open source so far. But basically, for the listeners, I would say that if you have models that can be downloaded by users, there are two things, and then they’ve got those models, you can’t take them back. That’s the nature of open weight models.

There are two consequences from open weight models that are downloaded by the users. One is that they can modify them, meaning they can take the safeguards off, that’s bad. But the other thing is that they can do bad stuff, start a cyber attack or something.

And if you figure out a cyber attack is happening, you can’t switch it off. Whereas there was a massive cyber attack recently in Mexico, with a hack on I think the electoral records in Mexico. And Anthropic figured out that Claude was being used for that cyber attack.

And so they did shut off access to Claude. And in fact, OpenAI did the same for ChatGPT, which was also being used in that cyber attack. So there’s two very important advantages you get from banning powerful open source models.

And it seems to me that an important US objective, which has never been embraced really by either the Biden team or the current one, would be to get serious about limiting open source. I guess the threshold question is, do you agree with that?

MCGUIRE:
I do. I think that look, the open source debate is also very difficult. Politically and also constitutionally, in some ways.

We have to acknowledge that you do get to some freedom of speech issues when you’re regulating the proliferation of open source models, that you have to be very careful. And obviously, there’s significant actors in the United States, in particular, that have a lot of vested interests in open source and have promoted a campaign to push it. Yeah.

MALLABY:
I mean, the existence of a major lobby is not a very persuasive policy argument there.

MCGUIRE:
No, but it is a higher hill to climb.

MALLABY:
I get it.

MCGUIRE:
Right. And I think that the first order problem is being ahead of China, for no other reason than because the best open source models in the world are coming from China. So right now, actually, China is a bit of a proxy for the best open source models.

So if you’re concerned about the capabilities of open source, the number one thing you’d want to do is constrain China’s AI capabilities. Now, that could change at some point over time, admittedly, but right now, that’s what it is.

MALLABY:
Well, constrain their abilities is the stick policy. But what about a carrot policy where you persuade them that it’s in their interest?

MCGUIRE:
I mean, this gets to the kind of fundamental questions we were having before. And I think you would hopefully be at a point when you could work together on things that are in our mutual interests, even regardless of the environment. Nobody wants to have an AI model get out there that is able to make biological weapons very quickly and easily.

That is not in any government’s interest at all. And I’m very concerned about how we’re going to regulate that. And I think that if we can’t get our hands around the question of, should AI infrastructure globally be in entrusted actors?

And in my opinion, although you might disagree, the question of should we be providing our chief adversaries with the key inputs into this technology? If we can’t answer those questions clearly, then I think the open source regulation question is profoundly more difficult. But it is one that we’re going to have to solve.

And I think that the first step is making sure that we maintain the lead and that the countries also that are developing those open source models struggle to do so. But then we have to ask ourselves, what does it mean? What do we want to do independently?

There’s a question even separate from China, right? Because regardless of where Chinese models are, if we believe this is actually such a big international security issue and also domestic security issue, then we should have our own laws to regulate that domestically. To regulate open source.

Yeah. And it’s a hard question. But ultimately, as capabilities get better and better and better, and as algorithmic improvements actually push the amount of compute that you need to deliver those capabilities down and down and down, to the point where you in theory could have a small open source model that’s running on your computer that could do any of the really concerning things that we’re talking about, especially in an absolute sense, that’s a really scary world. And what do we want to do about that problem?

Frankly, I don’t think we have good answers to that. I think that that’s something that the American political system has yet to really grapple with. But I think it’s probably coming pretty soon.

MALLABY:
If we agree that open source is dangerous, then having the U.S. ban it internally, but the Chinese continue to provide good open source models, which the rest of the world can use, is a very bad outcome. I mean, there’s no point the U.S. restraining itself if this danger is proliferated by somebody else. So it has to be a deal where the U.S. and China both agree that open source is a bad idea. And I think that a key part of the argument here is that if you had the two AI superpowers agree, then you could pretty much impose constraints on open source, open wake on other countries, because basically most of the ecosystem needs the U.S. Because the U.S. has the clusters, it has the chips. Only China has any hope at all of developing a ecosystem. And so if you had the U.S. and China agreeing that open source is a bad idea, because it’s not in any government’s interest to have some terrorist organization developing a devastating bioweapon, then it could be done. I mean, I think the nuclear analogy is good, because you know, civilian nuclear power was permitted to the non-nuclear weapon states on condition that they didn’t use it to build a weapon. And in the same way, I think one would help to supply AI to all countries that want it, provided that they do it in a way that is not open source and dangerous. And that would be the bargain.

And that was a bargain that worked not badly in nuclear policy.

MCGUIRE:
Yeah, I mean, important to note that China didn’t actually accede to the NPT until 1992, right? So there’s actually a long period between when the NPT was ratified, I think, in 1968, and when China actually ultimately joined. There are tenets of this idea that I think could be applied.

I think that they also could be applied while in fierce competition, though. It doesn’t necessitate us to provide our chief adversary with the most advanced technology in the world in order to reach that deal. Because as you said, this is something that’s also in their interest, right?

And the world where we are enabling our adversary to have the technology that they could use to undermine our interests, and particularly if they’re very good at certain elements of this that we’re not, right? If they are better at deploying quickly than we are, if they have, you know, more people than we do, where does that mean for our kind of algorithmic advantages that we have now? But how confident are we that those are enduring?

They have access to more data than we do. More energy than we do, right? So like, there’s worlds where if we don’t take this approach on compute, we start to fall behind.

And then at that point, how much do we trust the Chinese to keep their end of the deal and actually make sure like, okay, we’re gonna make sure that you guys can make up your part of the bargain here too. Like I’m not confident in that. I don’t think any state would be.

MALLABY:
I think it’s a wrap. All right. So it’s been a great debate.

So just to sum up, I’d say we discussed how things seemed in 2022. We discussed how things look in 2026 and how plausible a non-proliferation treaty to deal with open source might be in the future. Huge thanks to Chris for joining.

But before we let you go, Chris, we have these tidbits at the end where, you know, we raised something maybe a little amusing, maybe unexpected. So mine is going to be, I don’t know if you guys can see this box here. For those listening, I’ve got a kind of gray box, which says on the outside, SuperHexa AI Eyewear.

And I was given this in China. I went to a dinner and there was a venture capitalist who said that one of his portfolio companies is making these AI glasses. So I’m going to get them out of this box.

I’m going to put on these glasses here. I look fairly geeky now. And if I switch these glasses on and I look at a Chinese newspaper, it’ll be rendered into English text and I’ll be able to read, which would be fantastic for my next trip to China.

But Chris, I need your advice as a national security expert. Should I risk using them?

MCGUIRE:
I maybe might not bring any of your personal devices close to them or link them with your phone on Bluetooth or something like that. But I think as an experiment, when you’re in China with a burner phone in order to help you read a newspaper, I think that’s probably all right. If you’re going to use it here in the US to read sensitive documents, maybe shy away.

MALLABY:
Okay, great. I’m putting them right back in the box and away from all my other machines. Great.

Okay. They’re not switched on, I promise. What’s your thought of the week?

MCGUIRE:
So there’s a story circulating about the degree to which AI is kind of permeating every element of industry and culture and even kind of untold ones. So actually, there’s AI enabled cattle herding. It’s now extremely lucrative industry.

So Peter Thiel is apparently valuing a company at $2 billion that develops things that go around cow’s necks to automatically geo-fence them into certain areas and then also push them towards food at particular times. So the idea that we’re actually using AI to get rid of cowboys and cattle dogs. So nothing will not be touched by AI.

Even the sheep herding dogs are going to be a thing of the past these days, thanks to technology. I love that.

MALLABY:
Actually, I love the $2 billion valuation for the following reasons. So Peter Thiel was the anchor investor in the original AI lab, DeepMind. He wrote a check in 2010.

He wrote a check for $2 million and the valuation of DeepMind, which was going to be the creator of large numbers of amazing models, was $4 million. So we’ve gone from a $4 million valuation on DeepMind to a $2 billion valuation for something to do with cattle.

MCGUIRE:
If cow herding is valued at $2 billion, imagine what the labs are, right?

MALLABY:
Right, right.

MCGUIRE:
We’ll find out this year, maybe. We’ll see.

MALLABY:
Okay. So Chris, it’s been such a pleasure. Thank you for coming on The Spillover.

I’ve really enjoyed debating this with you.

MCGUIRE:
Thanks for having me, Sebastian. I don’t think this issue is going away. So maybe sometime again in the future, we’ll reevaluate.

MALLABY:
And for those of you tuning in, we’ll see you next week on The Spillover. And don’t forget also, that we have a giveaway, giving away copies of my book on AI, which will be coming out in a week or so. And you have to submit questions.

And the questions we pick will be rewarded with a book. In our show notes. If you have an idea, or you just want to chat with us, email us at podcasts at cfr.org.

Be sure to include The Spillover in the subject line. This episode was produced by Molly McAnany, Gabrielle Sierra, and Jeremy Sherlick. Our video editor is Claire Seaton.

Our sound designer and audio engineer is Markus Zakaria. You can subscribe to the show on Apple Podcasts, Spotify, YouTube, or wherever you listen to podcasts. Research for this episode was provided by Liza Jacob.

This transcript was generated using AI and may contain errors or inaccuracies.