The Real AI Race Isn’t About Models. It’s About Compute.

This transcript was generated using AI and may contain errors.

MALLABY:
Suddenly, just in the past month, the progress of artificial intelligence, the sheer speed of the developments, the thrill, and frankly, the terror of what these developments portend for the rest of us, all of this has gone into overdrive. First up, OpenAI has confessed that two of its models escaped from their testing environment, advanced out onto the internet, and penetrated the defenses of another AI company, Hugging Face. Meanwhile, Hugging Face, the targeted company, announced that in order to figure out what the heck had happened, it had used a Chinese AI model.

So China, often accused of undermining Western cybersecurity, actually helped to maintain cybersecurity in this instance. Also, meanwhile, each of the three Frontier Labs and a couple of non-Frontier ones have released new models in the past month. China has announced a string of new AI models that compete with Western ones in capability.

Various Western AI leaders, including Demis Hassabis of Google DeepMind, have proposed safety ideas. The Trump administration denounced China for reverse engineering American AI models. And China’s president, Xi Jinping, delivered a major speech on China’s plan to be the top AI provider for the world, announcing what one might see as an AI version of the Belt and Road Initiative.

So in short, the models are getting stronger. They are getting more dangerous. And politicians and lab leaders are floating a bewildering slew of policy ideas in an attempt to get on top of the chaos.

I’m Sebastian Mallaby. Welcome to The Spillover. Today, my Spillover co-host Rebecca Patterson is on the road.

So I’ll be discussing AI progress, AI safety and US-China AI rivalry with Jordan Schneider, the founder of the China Talk podcast and newsletter. I’m super excited to have Jordan on the show for two reasons. First, he’s incredibly smart, especially on the Chinese side of the AI race.

And second, he has vouchsafed an earnest promise that he will disagree with me strongly. Jordan Schneider, thanks for doing this.

SCHNEIDER:
Oh, so happy to be here. I’m an enormous fan of yours. If only I could write books like you do.

I’m just a lowly podcaster out here. So honored you’ve decided to jump into the medium.

MALLABY:
Well, you know, I’m a beginner. You’re not a beginner. We were joking before that you’re an overnight success after 10 years of hard slog building your channels.

But just tell us, you did history at Yale. You studied in China. You went to Bridgewater, the hedge fund.

Tell us a little bit about your journey before we get started.

SCHNEIDER:
Oh, man. Well, I mean, it’s still Odyssey Week. So we can start off with the Yale thing since you brought it up.

I did a lot of ancient Greece and Rome, Latinist back in the day. And it was so fun to just get to like, come back to all that stuff.

MALLABY:
Are you more fluent in Latin or more fluent in Mandarin?

SCHNEIDER:
Oh, I mean, yeah, I’m like down to 10 words of Latin at this point. It’s a little embarrassing. But I think one of my colleagues who’s in this building, Eddie Fishman, took all those courses with me.

What can I say about my journey? You know, the big hinge was in 2017, I was sick of Bridgewater and applied to every fellowship I could find on the Internet. And this was before ChatGPT.

So you actually had to like look for them to send me to Asia, like broadly. I think it was like 10 different countries or something. And Peking, Yenching Academy at Peking University is the one golden ticket for free grad school I got.

So I showed up in China in 2017, spent three years there, which is also when I started China Talk. And I’ve been back in the US since COVID. And now China Talk is not only a podcast and newsletter, but also a little think tank with a small but mighty band of five or so researchers.

MALLABY:
Well, congratulations. I know you’ve got a lot of people who follow a lot of fans. So well done.

So let’s get into the AI topics. I mean, given your background and my obsession, you have expertise, I have just sort of, you know, compulsive infatuation for the China side of this story. We’ll do a lot of China in a minute.

But let’s start with the US side of AI and maybe the sort of safety debate around AI. I’ll give you my sort of capsule framing of how I see the recent history. And perhaps you can fill it out, tell me where I’m wrong and so forth.

I’d say that, you know, you get ChatGPT coming out in 2022. Pretty quickly, in other words, within a year, by the end of 2023, you’ve had the Bletchley Safety Conference in Britain, with countries sending people from around the world. You’ve got the US AI Safety Institute, the UK AI Institute.

That’s end of 23. Then during 24, this builds, and a lot of other countries set up national AI safety institutes. Of course, these have limits, but the infrastructure is being created.

And at the end of 24, you get this meeting of all the different safety institutes in San Francisco, where there’s some talk about combining perceptions on what makes a model safe, what safety, how you define it, et cetera. So there’s this kind of fairly decent progress through the end of 24. Then, you know, Trump comes in early 25.

France goes to Paris for the AI summit, which is in that line of meetings created by Bletchley. And he flips the script and says, we’re not interested in safety. We want to do rollout, enough of this woke stuff.

Let’s just deploy, deploy, deploy. And so 25 is kind of a dead year for safety. And as late as March of 26, if you had said to the Trump administration that they would be trying to suppress an American AI model, decelerate the progress in the name of safety, they would have said, you’re nuts.

But actually, April comes, Mythos comes from Anthropic, and they do a 180, and they do suppress a model release, and they do care about safety. And so we’ve had this kind of helter skelter, like gradual rise of safety, a collapse of the interest in it. And then a return.

Is that how you see it? Or would you add to that change it or?

SCHNEIDER:
Yeah, well, I mean, it kind of comes back to like Demis reading sci fi in the in the 90s, 2000s and 2010s. Because the creators in of this technology in the West, in order to be interested in this stuff, in the 2010s, had to be a believer and had to internalize that this was like a deeply weird thing, which would ripple through society and like, very uncomfortable, I mean, exciting, but also very uncomfortable ways. And I think the Trump administration, pre Mythos didn’t believe it.

They just and I think the Chinese wouldn’t get to them later, also still don’t really believe it. They think this is a normal technology, which is exciting and has a lot of growth. But like, you know, we don’t need the we’re we’re, you know, Republicans, we’re anti regulation.

We took we have money from a16z and Marc Andreessen’s advising us on any and on everything. And he’s telling us it’s OK, so it’s OK. And then there’s been this fascinating dance that you’ve that you’ve written about of the model makers, like kind of understanding that they’re going to need to get regulated, but like not in this way, not in a way that is going to stop me and my first thousand employees from having nine figures to our name.

Right. So we have this interesting dance. And yeah, like was could you have priced in the Trump administration flipping at some point?

I mean, I think if you’re Demis Hassabis, the answer is yes, because you understood that we were on this curve where at some point, you know, you get to a point where governments are just going to have to step in and say, no, we’re going to have a say in how this technology is going to be developed. But yeah, it’s been really dramatic. And on the China side, it hasn’t happened yet for them.

They’re still if you look at or actually, you’re the interviewer.

MALLABY:
But that’s the next question. We can talk about something else for the next question. I was actually going to stick on the US a little bit more.

So, you know, we’ve kind of taken the story up to that mythos moment in April. The Trump administration changes position. You have Glasswing, the gradual rollout of mythos and the kind of, you know, back and forth about, you know, is fable allowed and is it not allowed and so forth.

And then most recently, you know, just this week as we’re recording, you have this episode with two OpenAI models that for the first time ever, you know, autonomously jailbreak, get out of the sandpit where they were supposed to be confined, you know, find their way onto the Internet, hack their way into another company’s site, hugging face. This is not a sort of amateur, you know, small company. This is a serious cutting edge AI company with lots of capability.

They get hacked anyway. I mean, this is pretty unprecedented.

SCHNEIDER:
They think it’s the Chinese. Right. And they’re freaking out.

They’re calling the FBI. And then two days later, they’re like, oh, wait, actually, it was OpenAI testing their models. So that’s how serious it was.

Like they were they were convinced this was a nation state level threat. Right. But it was just like a test gone awry.

MALLABY:
Right. Right. It’s the nation state version of the Turing test.

SCHNEIDER:
This is wild. It’s wild. Right.

I mean, like and you’ve also had this discourse over the past year and a half where people keep saying, oh, we think alignment is more and more under control. It’s less of a problem. And then something like this happens.

MALLABY:
Right. Right. Right.

So how do you see this impacting that arc of safety? We’ve got to the point where, you know, there’s there’s some desire to kind of clamp down on or control the release of soul from OpenAI mythos fable from Anthropic. Does this just like intensify the same thing or do you see some fundamental gear shift?

Well, look like someone’s going to have to do this work.

SCHNEIDER:
Right.

MALLABY:
Like being the work being figuring out the policy response.

SCHNEIDER:
No, like making sure that the models, as they get more powerful, don’t do crazy things like this. And I mean, and someone needs to be able to be to like understand whether this stuff can get a thumbs up. Right.

And if OpenAI, who has every financial incentive in the world for this not to happen, right, isn’t doing the work, doesn’t know their models well enough, can’t control it. Is CAISI with it’s like 10 million dollar a year budget going to be able to do it? Is the UK AI Safety Institute with its 50 million dollar a year budget going to be able to do it?

It’s I think it is just another underline to the fact that like we’ve really crossed a threshold with these models, like having the potential to do really dramatic harm just on their own, because we like can’t even physically watch them.

MALLABY:
Right. So the CAISI that you just mentioned is the renamed version of the Biden era, US AI Safety Institute now called CAISI, whatever that stands for, which which has 10 million dollars a year and has gone through two leaders in three months.

SCHNEIDER:
And they had a guy who showed up for a day who was fired because like apparently no one realized that he used to work at Anthropic or something. So I guess like like we’ve now reached a point where this administration and I think increasingly the world is going to be turned on to the fact that there are real downsides and there are real scary things that could happen. Yeah.

But no one has the answer yet about how to do this in a smart way.

MALLABY:
So let’s talk about one proffered answer, and that’s from Demis Hassabis, the one that he put out last week. And I think one of the things he says in that is let’s have a self-regulatory institute. And I think I see this as a end run around the budget problem that the existing AI Safety Institute has.

I mean, you said $10 billion budget that ain’t going to get you anywhere. But if you had a self-regulatory body, you could not depend on appropriations from Congress, which slows you down and limits your resources. You get, you know, some sort of middle ground between something which is a government thing under-resourced.

But, you know, you don’t want to rely either on OpenAI to police itself because we just did that and it didn’t work out so well. So you need a serious independent body, but maybe funded by industry, not by the government. How do you see that?

Do you think that’s going to work? It’s just really tricky, right?

SCHNEIDER:
Because there’s so much money on the line and it’s like it could be companies on the line. It’s like, all right, if these firms are going to be public soon, how much market cap are you losing because you’re slower to market by two months because, you know, some obnoxious independent AI safety person says your model isn’t safe, but you actually think it is safe. So, I mean, yeah, it’s nice, but there are no clean solutions.

Demis, of course, like he tried to have his own independent safety board, or I guess this was Mustafa with his… There was the two of them together. Yeah, and it was with their NHS medical stuff and it just like got really obnoxious really fast because I think the problem is the incentive structure is like if anything bad with AI safety happens with the board is like they’re blamed for the lights going off in Denver, right?

But we also have like an economy to keep afloat by not sort of taking off technological advancements from the future.

MALLABY:
So you mean the bias of any AI safety body is to be too restrictive because you’ll take the blame if you are not. And then it slows it down too much and that’s bad for the economy and China gets ahead and…

SCHNEIDER:
It’s just a really hard design problem. I mean, I have more faith in you doing it than like the companies themselves. Well, I think it’s just like…

Shoot, we’re really in trouble then. I know, right? I need a little more tech than just three cameras to get us the answer to that.

No, I don’t have a great answer.

MALLABY:
Yeah. And is there any other idea floating around proposed by somebody else that you favor as a sort of like better thing or you just view this as such a difficult problem?

SCHNEIDER:
You know, I think a week ago, had you asked me this, I would have figured that the model makers understood. I think the… I would have thought that the model makers had enough incentive and had enough kind of technological capability.

I mean, it’s only two companies, but they’re like I mean, I guess two and a half, three, sure. Well, it can be nice. They have enough technologists where like, okay, if the capabilities are exceeding our ability to control them, then you just like shift over compute and you shift over human beings until your ability to control and understand what the models are doing catches up to how powerful they are.

But if that’s not happening, someone’s going to need to make it happen.

MALLABY:
Yeah, exactly.

SCHNEIDER:
I don’t actually know if like market force, if like the competitive dynamics is, you know, if the cost of this, like how much, like, I don’t know, you’ve written a lot of markets books, like how much, if OpenAI was public and that news story came out, like what percent do you think their stock would have gone down?

MALLABY:
Wow. Yeah. 15.

I mean, I don’t think it would have been a zero, but I think, you know, yeah, it would be a hit to people’s confidence in the technology, the confidence that they could serve technology to customers and customers would believe in the safety of it. Yeah. Yeah.

So it would be a problem.

SCHNEIDER:
So maybe you actually just need the company, you need the companies to be public for them to really internalize. That’s a nice idea.

MALLABY:
The cost of this. Yeah. But I think the incentive, the incentive distortion is a little bit different.

You could frame it that, you know, maybe to give them the benefit of the doubt, the lab leaders of Google, DeepMind, Anthropic and OpenAI all would quite like to spend more on safety, but they can’t unless the other guys do as well. And we’ll get to China in a minute, which is obviously a big part of this ecosystem. But there’s a collective action problem, which can only be solved by some sort of body that’s regulating all of them.

And so if they all were slowed down in the same place, they might be cool with that.

SCHNEIDER:
Yeah. But if the body slows them down and one is in first and one is in second and one is in third, and the cost between being in first and second is like, I don’t know, 250 billion dollars of market cap, like it’s a problem. Right.

So I agree with this, but I think in the abstract, but like if you freeze everyone in place and someone’s ahead of someone else, then you’re the incentives all of a sudden.

MALLABY:
You could have a World Cup soccer game with a referee, but both sides will doubt that the referee is really fair. For sure. Yeah.

Yeah. Yeah. Yeah.

Yeah. All right. Let’s switch to China.

And I want to split this into two parts. One is to talk about internal actions within China. And then after that, we’ll get to sort of U.S. policy to China, the kind of, we’ll bring it all together. So taking the China internal story first, we’ve seen a slew of recent models and looking at Kimi K3 and the rest of them. How do you see the gap? How big is the gap between U.S. and China at this point?

SCHNEIDER:
So I think the, I remember having conversations even like five years ago about the idea that algorithmic innovations and data would kind of end up netting out between the U.S. and China. And the thing that was ultimately going to matter in the long run was compute.

MALLABY:
So in other words, just to clarify, so the data, China would have an advantage because there’s… So data, awash. I see.

SCHNEIDER:
Algorithmic innovations, how smart your scientists are. Also, you know, in the, over the medium term, awash. Got it.

But when you’re talking about like how much your nation, your state, your economy can gain from AI, it would come down to how much compute you have. Because you can steal weights, you can copy what other people are doing on the model side, you can steal dating trainer sets, you can make your own dating trainer sets. I mean, it’s not like there aren’t people in China to label data, right?

They have lots of PhDs and whatever, just like Mercor can pay people. And at the end of the day, like, yeah, it’s nice to have that model advantage. But what’s gonna matter over the long term is like how much AI can you actually use.

And, you know, we’ve run the numbers here at China Talk. The ratio of what the Chinese ecosystem has access to versus the rest of the world is still like 10, 12 to 1 in terms of both production and access to production of like raw chips, as well as access to cloud compute. And so even though, you know, over the past, since ChatGPT came out, we’ve gone from 18 months to nine months back up to 12 months, maybe now down to six months.

That is the thing that I think is, you know, probably a more important lodestar than like, how is the best model in China today relative to the best model available to the US?

MALLABY:
In other words, I shouldn’t be asking about how far is China behind in terms of the quality of model. It’s more a question of like, how much can China deliver the AI to users within China, given their lack of computational resources? Within China and around the world.

There is, though, still a question about the power of the models in the sense that one significance of a strong Chinese model like Kimi K3 is that it might be used by non-Chinese companies, and it is being used, and served on an American cloud. And to that extent, the Chinese lack of compute resources doesn’t matter, right? What does matter is that, you know, Kimi K3, if it’s really good, cannibalizes the revenues of the US frontier labs.

SCHNEIDER:
Yeah. So point A is like a 10 year net assessment thing. Yes.

The fact that Chinese models today are, you know, only five or six months behind, as opposed to two months ago, where they were more like 9 to 12, is absolutely very relevant for OpenAI Anthropic, in deep mind, if you’re trying to make a business on selling API access. I think it’s debatable, like, as long as you have a lead, it’s debatable. And I think the point that you just made of like, it is like it requires so much less compute to train a model than it does to deliver it at scale.

And like having a business requires delivering it at scale, right? That, yeah, there’s a lot of knots to unpack there, which maybe we should.

MALLABY:
Yeah. And just quickly, one last thing I think worth maybe putting on the table, because I think there’s some misperception about it, just because the tokens generated by Chinese models are often priced at a very low rate. It doesn’t follow that they’re better to use, because the capability, you should really price the cost, what’s the cost of the AI for each task you get done?

Yeah. And if the, as I understand it, Kimi K3 is cheap on a per token basis, but not cheap on a per task basis.

SCHNEIDER:
Yeah, well, I mean, it’s, it’s kind of like, we’ll see, right? Because it hasn’t been open sourced yet. That’s supposed to happen in three days.

And like, whatever they are selling, their token, whatever the token, the tokenomics of like…

MALLABY:
That is a new use of the word tokenomics, by the way, it used to be a crypto term. You’ve just…

SCHNEIDER:
Oh, no, no, no. This has been used for a while now.

MALLABY:
This is a shout out to SemiAnalysis.

SCHNEIDER:
They started with this. The, yeah, the tokenomics of whatever their business model is of like selling it via AliCloud or whatever, is going to be very different from once it’s on OpenWeights, and then it’s on Azure, right? Because presumably they’re taking some slice, or maybe they’re not, and they’re just selling it at a cost because people want to like have access to it.

But also like the economics of like Azure is more well run than Alibaba, and they have better access to cheaper chips. So I don’t know. But I think there’s like the, this is like the more like CFR-y question, right?

Is if you have sort of what does it do? And this is the thing that the US labs are all freaking out about is like, what does it do to your long term business model? If the, if you’re trying to have this like cash cow of selling these front, these close front tier models, that is then going to feed back into your R&D.

And insofar as this continues, and it actually turns out that we get Chinese models for years that can kind of prove that they can continue to sort of OpenWeight, like not quite best in class, but like pretty close for like a lot of your stuff, then there are a few other independent variables of like, to what extent is like the marginal, you know, having like Claude Mythos 6 versus Claude Mythos 5, like actually how many business cases do you actually need for that?

But yeah, I think it’s a significant long term threat to the business. There’s, there’s a significant like security questions that I think are that come out of that. And I don’t know, it’s also a big, big mess.

MALLABY:
So, so, so I want to, you know, make an observation, we’ll put a pin in it and come back to it. But the observation is simply that, you know, just like the US has chip export controls, which are a way of sort of damaging the Chinese AI ecosystem, trying to slow it down and so forth. I think of Chinese OpenWeight as their kind of counterweapon in the sense that they can sell cheap models, which is sort of good enough to American consumers, American enterprises.

And this will damage the economics of the frontier labs and it will slow them down a bit.

SCHNEIDER:
So to be clear, now they’re not selling them. They’re giving them away right now.

MALLABY:
They can give them away such that they are then sold to the Western users.

SCHNEIDER:
Yeah. It’s, it’s fascinating because this was an emergent strategy, right? Which has now been kind of adopted by, you know, Xi Jinping on a speech, like at a speech at an AI conference, like talking for 10 minutes about this thing.

But this originally started because the Chinese models were not that good. Sure.

MALLABY:
So you had to open way.

SCHNEIDER:
The only way anyone would ever pay attention to them or even like try them out was by making them free. And we’re still there, right? Where like, no one is paying for this stuff, really, because as you said, on the sort of like cost capability curve or whatever, you can still get better stuff.

And I think like, you know, OpenAI and Anthropic are very aware of this and are going to be pricing their stuff accordingly such that you don’t end up having a big incentive to go over. And it’s, you know, it’s, it’s hard for those Chinese AI labs, right? Because they are working with less compute.

They’re, they’re, they have, they’re smaller companies, they have fewer resources. And to try to like, and I guess to date selling, selling software in China has been really, really difficult. So when you look at the revenues of a Zhipu or a MiniMax, and then you look at a revenue of OpenAI and Anthropic, it’s like a joke.

It’s like, it’s like they made like $150 million last year or something like, like truly pathetic. So, you know, do they need to end up having a business model at some point? Maybe, maybe not.

I mean, I sent you this, this, the DeepSeek CTO interview who, who is raising billions of dollars, but is basically telling all of his investors sort of the vision that Demis Hassabis was pitching back in like 2015, saying, look, I’m running a science lab. It’s going to be like the Manhattan Project. If it works out, it’ll be the biggest thing humanity has ever seen.

Don’t you talk to me about stupid business model stuff. And, you know, he ended up having to make AI overviews for search at a certain point. Right.

And, but on the downside of that, like now he has to manage 6,000 people and deal with Satya and public markets and whatever else. So you have a guy like Liang Wenfeng in DeepSeek who’s trying to build like a financial structure to make the bet of, no, just like give me 300, mostly dudes, dudes, and just let us make science. And we’re going to do a better job than these like giant high bound organizations that are now stuck in an innovator’s dilemma because they have business models.

MALLABY:
He basically says in that thing, products are for losers. Yeah. You know, you know, the real winners, we just look at AGI and we’re not gonna be distracted by this nonsense of having a customer.

SCHNEIDER:
Yeah.

MALLABY:
So it’s the anti-business model business model.

SCHNEIDER:
Totally. And look, um, you can go really far not having a business model if the public markets believe in you, right? Like Amazon lost money for 30 years.

Tesla was like the biggest short for 20 years. Right. And it’s now worth the tribute is now worth, uh, well, I mean, who knows what it is today, but look, look like, like, it’s not totally crazy to think you can sell people on an idea, especially if, you know, you’re not Demis Hassabis in 2015, like you’ve proven that you can do worlds.

Um, uh, you can, you can deliver technological change, which like shakes the world. Right. So I wouldn’t put it, which DeepSeek has proved, which DeepSeek has proven.

So I wouldn’t put it past him at that perspective. There’s also the layer of like, yeah, if the Chinese government ends up believing in this thesis, that open weight is like, you know, kind of screws over America and it’s good for diffusion is good for soft power. Then, um, you know, having the state just like keep cutting you checks is another way to not have a business.

MALLABY:
So let’s get to that. So, so, I mean, we’re talking here about, you know, DeepSeek on the one hand, raising money on the other hand, telling its funders that it’s not going to make any revenues. Yeah.

Um, which, which gets to this issue, which applies to actually all of the Chinese labs. Um, how are they economically sustainable if they are producing open weight technology, which means they give it away. The revenues may accrue to the cloud provider that serves the model, but they don’t get revenue as the model designer.

The algorithms are given away. Yeah. So what the heck is going on here?

How do, how does the whole Chinese AI sector carry on existing when they don’t seem to have a revenue model? Is this government subsidies? Is it billionaires cross financing this, like the DeepSeek guy makes money from his hedge fund and he pours it into AGI for fun.

What’s how are they, how are they surviving?

SCHNEIDER:
Well, no one has a good answer yet, but like basically, um, what we’ve seen so far is investor interests, you know, people buying into a dream.

MALLABY:
And these are Chinese domestic investors or who?

SCHNEIDER:
Um, uh, yeah, well, yeah, the foreign money is basically gone. Right. So it’s, it’s, it’s Chinese domestic VCs.

It’s, um, and then it’s the current hyperscalers and like large companies in China. And by the way, it’s like, these are like 15, 20 companies that are building models. Like, um, uh, Kling, uh, uh, came out of a, a short video app that I used to work for called Kuaishou, um, that make some of the best video generation models in the world.

We have ByteDance, we have a Meituan, which is a food delivery app. So imagine like DoorDash is having world leading models. I mean, this is basically because the CEO is rich and he thinks it’s cool.

So like, yeah, we’re going to do it and we’ll, we’ll, we’ll figure it out. Right.

MALLABY:
Um, I mean, I want to click on Kling. You worked there. There’s got to be a story.

SCHNEIDER:
Oh man, it was just really dumb. They had me like open. I was like doing, I was like the third foreigner there.

There were already 3000 people. I, they had me doing, um, Brazil. And I was like, guys, I studied Spanish by the way, in high school.

Like I don’t speak this language. I basically spent my whole time recording China talk, uh, podcasts. And then they caught up to me at some point.

Um, uh, but okay. So coming back to the, how does this work in China? I mean, it’s not that expensive.

It’s pretty expensive. Um, uh, uh, look, it’s not cheap. There is capex involved, but like the cost of training the models, especially if you can, you know, fudge around with distillation or whatever, these aren’t like it’s, it doesn’t cost you like 10, 20, $50 billion.

We’re not at those like God run things that we were talking about and, uh, in, in, in 2023. Right. Um, there’s another dynamic of the Chinese domestic market.

Like you don’t, um, over the medium term, like you are going like right now you still have a lot of Chinese consumers, like using VPNs and these transfer stations to actually just like get Claude and, um, uh, touchy BT tokens. But I mean, that’s a protected market. So there’s like revenue there, right?

You may not have to open source everything, or you can, um, you know, if you are the one who is supplying it because you’re Alibaba and you also own Ali cloud, like, you know, that’s good for business. So I think it’s, it’s, um, it’s not obvious like people, but also look two, three years ago, people asked, uh, uh, uh, I remember there’s a famous quote from Sam Altman from like 2021 being like, how are you going to make money? He’s like, I don’t know.

We’ll, we’ll figure it out. Um, and I feel like that’s the same thing.

MALLABY:
Yeah.

SCHNEIDER:
It’s a, it’s a, if you can look over the pond and see trillion dollar companies being created, like, you know, you will, you will have a real story to tell.

MALLABY:
Right. Right. Right.

Okay. So, so we’ve described, um, sort of the Chinese AI private ecosystem, but let’s talk about the government angle on this. And there was this speech just last Friday as we’re recording, uh, by president Xi Jinping.

Um, and it, it sort of, uh, I forget whether your paper when China gets a mythos level model. Yeah. Uh, did you do that right before the speech right afterwards?

We’ve done like versions of it a few times.

SCHNEIDER:
Um, yeah, we’re updating it. We’re slowly updating. I mean, it’s fat, like coming back to that, like Trump pre mythos, post mythos thing, right?

Who would have, you, you could have had all the exquisite intelligence of the world have read every single email that’s coming in and out of the white house. Um, which I do, of course. Yes.

There would have been nothing in there to indicate that, um, you know, once, um, a model as good as mythos comes out, they’re going to flip on a dime. And Scott Bessent is going to call up all of the heads of the banks and say, you know, this, this stuff is crazy. You got to lock it down and whatever.

So, um, given that we’ve seen that movie before, um, uh, I do want to like put a bit of, uh, um, like I still want to keep an open mind at the, as to the future of Chinese policy, because, um, what we’ve seen, what we’ve seen, uh, you know, from 2018 to 2025 was kind of like benign neglect. We only really care if these models talk about Tiananmen. Um, starting in 2026, there was a little more focus on it.

Um, at the highest levels, like you had a, um, uh, you had a Politburo study session about AGI and, you know, maybe they’re like cluing in a little bit, but the, the, they intervened to stop people downloading open core at one point.

MALLABY:
That was a small sign.

SCHNEIDER:
Yeah. But it’s still, it’s still kind of like, okay, this is like a fun, cute thing. And then we have a big declaration, um, from Xi at this speech saying, we think openness is good.

And we think it’s like positive for humanity. It’s positive for, you know, China’s role in the world. We want to keep doing it, but there are outs, which you can read in that speech where he’s like, we want this technology to be safe and controllable.

And then he, he quotes, um, uh, uh, a lion from this like famous Chinese debate in 81 BC about like, whether or not they should nationalize the salt industry, um, uh, debates of like salt and iron, where the quote was basically like, look, you gotta be smart about this. Like we can’t really know you gotta be like flexible and nimble to base depending what’s going on. So I, because they haven’t hit the, because these, these models, which are very good, have not crossed that, um, April, 2026 mythos, this stuff can hack anything.

Um, if you just like press go and don’t look at it over a weekend, um, I could see them getting freaked out.

MALLABY:
Right, right. So they could just like with COVID, they went from lockdown to opening up super fast. They could change.

SCHNEIDER:
Well, they went from, uh, yeah, they went from this thing is fake to lockdown super fast. Right. And that’s, that’s kind of the direction we’re going on.

I mean, you made a point in a, in a prior edition of this show about like, won’t the Chinese get freaked out when, you know, Xinjiang separatists and the Falun Gong get access to open weight models.

MALLABY:
I mean, yeah, it doesn’t seem so yet. Right, right, right, right, right, right. Um, so I think I said at the top that, um, the speech, uh, could be seen Xi Jinping speech could be seen as a sort of declaration of an AI version of a belt and road policy.

And in the sense that, um, the attempt is to provide, um, you know, AI for the world by China, they want to be providing open models because that’s very cheap and useful for, for both the global South, but that matter for Europe. Uh, and if you pick this moment when much of the world mistrusts United States because of the quality of its political leadership, I’m thinking of Europe, particularly, you know, the Europeans are very keen to use Chinese models right now because they don’t want to be fully dependent on American ones, given that the American ones were both were first, you know, allowed with fable and then disallowed when the Trump administration changed his mind. And it just made, you know, people felt, Hey, I need an alternative to relying on the United States. And, you know, Chinese open models seem like, you know, a good hedge.

Um, so, I mean, do you think it’s correct to see the speech? Like is, if, if you just read it on its face, kind of stipulating that they might change their mind later for now, the policy seems to be, you know, try to just get the China stack out there, make as many people as possible, use it, keep it open, put safety second. That’s the policy.

SCHNEIDER:
Yeah. Well, we, we should specify the China stack because it comes back to my first point, um, which I think is the most important one about how much compute does everyone have? Because, um, you know, you can debate the curves, right.

But like, there are a few things I actually have a lot of confidence thinking about three to five years out, but you can really just count up the fabs and the tools and, and have an understanding of how much TSMC and, uh, Samsung and Intel are going to be able to print out relative to what Huawei and SMIC are going to be able to, um, uh, to take out over the next five years. And that ratio of somewhere between 10 to 13 to one is going to continue in the next five years. So, um, even if the AI stack, uh, like, like, like, like the AI stack that China will be able to export in the future is only going to be the model layer, not actually the, um, the sort of like cloud, uh, the cloud layer that compute under it, because like that, those chips are just going to go to not Chinese providers.

MALLABY:
So, so with that proviso, it’s not really the China stack, but it’s the China models. The idea is to disseminate them around the world as fast as possible, keep them open, you know, not withstanding that language in Xi’s speech around, you know, switch off switches and so forth. There ain’t going to be no off switch.

It’s going to be open way.

SCHNEIDER:
Yeah.

MALLABY:
Yeah. So I guess the next question is, um, China is pursuing a belt and road strategy on AI, but is that a smart choice? I mean, um, there are questions about how belt and road has worked out for them.

Um, you know, if the idea was to make friends and influence people, it turns out that, you know, you put Zambia in debt and afterwards they hate you. Um, and it, you know, you wonder slightly about how much leverage does China really gain by spreading its models around the world. People might use their models, but they’ve downloaded them.

They have them on prem. They have them on a local cloud. What leverage does it really give China?

SCHNEIDER:
Yeah. I mean, that’s why the, the, the belt and road analogy is kind of weird to me, right? As like, okay, like if you’re building a train and you’re, or a road and you’re going to like charge a toll on it, you’re going to build a port and then that’s going to be where your boats do shipping from.

And it leads to more like the whole point is the whole point of belt and road was to lead to more economic integration with China over the longterm. Right. It was just like, like configuring trade routes and creating connections.

Um, but if all you’re doing is like giving away a technology and yeah, you can like redistill it, you can do whatever you want with it. It doesn’t seem like a particularly sticky thing. And then again, like, um, making the models themselves, it’s hard, but it’s not that hard.

And like, if you buy the thesis that we can’t trust America, like it seems weird to go to that from that to, okay, so we’ll trust China. Like the, the answer is no, we need to do this ourselves. Right.

Um, and maybe it’s just as too much like collective action problems, like there’s talent gap, whatever. Um, all our good engineers are going to go work for these American labs that can pay them more than like the sad, um, uh, the sad domestic startup, but it just, um, I don’t know. It doesn’t seem to me to be like a stable, um, equilibrium in the way where, uh, kind of giving up on, on, on Nokia leads everyone to just Huawei’s arms in a way where you’re locked into Huawei, as opposed to this world where you can just kind of keep trading things in and out.

MALLABY:
Right, right, right. So in other words, it’s not a very good strategic move by China to give away these models in the hope that it buys them leverage. What it might do is work as an offensive weapon for just reducing the profits of the frontier U S models.

SCHNEIDER:
Well, again, you’re, you’re playing with a tough hand, right? Because you have less, you have fewer chips. Um, and, uh, by the way, like all those, uh, whatever chips, uh, Huawei are going to produce, uh, have more value to the PRC being deployed to run AI domestically than they do running AI and some like, um, you know, 40 chess, like let’s build a data center in Zambia.

So then Zambia would like us. Right. So I just don’t think that there’s going to be a lot of like excess compute.

Um, that’s in going to be in the offing for, um, for the rest of the world. I mean, Europe may be the one exception because like, if you can earn more money serving AI there than you can domestically, like that, that might have an economic rationale, but, um, you know, Belton road was exciting. Uh, uh, as a, I mean, in it’s like the dream version of belts road was like to bring developing countries closer to China.

Like, I don’t, I don’t see how, um, I just, I don’t, I don’t quite see the vision yet.

MALLABY:
Yeah. So let’s, um, try to put the two together. Now we’ve talked about the U S we’ve talked about China.

We’re going to talk about us policy towards China. What strikes me is that, um, you know, the Trump administration having done its U turn on AI safety with respect to domestic models, you know, the logical next step is if you care about safety within the U S you should also care about the safety of models coming out of China. Uh, because if you hobble us frontier labs and say, you gotta be very safe and very careful.

And then there’s a bunch of open weight stuff coming out of China, which isn’t safe or careful, you haven’t really made anything safer. And you know, you’ve just like, slowed yourself down to no avail. And that is indeed the argument that sort of the deregulatory side of the, of the tech debate in the U S um, that’s the argument they make.

So, you know, David Sacks, it’s precisely, he says, you know, I don’t believe we can slow China. Therefore, I don’t believe we should try to slow America, but the administration has apparently chosen to slow America. That seems to be where they’re going.

So the logical corollary is you have to try to slow China. Um, and yet that’s not quite being said out loud. There’s some, you know, press reporting about a September summit between Bessent and his counterparts, Scott Bessent, the treasury secretary and his counterparts in, in China to get a discussion about US AI safety cooperation started.

This is sort of rather saying, I think it’s fairly characterized as rumor at this point, but, but there is no strong vision from the administration about how they’re going to extend that domestic safety initiatives to China, notwithstanding the fact that unless you do extend it to China, the domestic stuff makes no sense. Do you agree with that?

SCHNEIDER:
Well, okay. So I don’t know if the domestic stuff makes no sense, right? Because like the, the theory of the case you have now is we are, you want to have a lead and model quality in order to stamp out all of the bugs you can find, um, such that when, um, you know, the Chinese open weights or whatever the Chinese national security state can get its hands on, they are, um, you know, using those capabilities against much more hardened targets.

Right. So, um, look like if mythos six is 10 times better, if like the next mythos is better than this mythos and it can find bugs and patch them even faster than, you know, that lead is still really useful.

MALLABY:
Yeah. But you’re articulating the David Sacks view here, which is a completely coherent view. Either I’m saying you race, um, you try to get mythos six as fast as possible.

And then you don’t care about Chinese open weight models because you think you can harden your own systems and protect yourself. That is a coherent view. What is not coherent is to be slowing down us frontier development whilst not slowing down the Chinese.

SCHNEIDER:
And that’s what makes you think they’re not. Oh yeah.

MALLABY:
Because right now they are, they’ve told both OpenAI and Anthropic to chill it in terms of who gets their models. Uh, they are apparently, you know, on the cusp of announcing pursuant to the executive order from a while ago, um, some sort of upgrade of the national regulatory apparatus. It seems like we are moving into a world of slowing release.

SCHNEIDER:
Oh, I understand. Okay. So like, so, okay.

In the, like your vision of like rational one is you’re pulling every lever to make a Western cyber the most, you know, Western AI do the most defense it can around, uh, uh, this ecosystem as well as pulling every lever to slow down, uh, the Chinese model development.

MALLABY:
Yeah. I think you, you either want to go slow and safely everywhere or slow and safely nowhere because, because if you just try to be slow and safe whilst not making the other guys be slow and safe, then the, the, the threat from the other guys is increasing, but your own defenses are not increasing.

SCHNEIDER:
Well, I mean, in this administration’s defense, which is a phrase I say very rarely, um, uh, look, I, I think you’ve, you’ve seen just over the past few weeks, uh, an acknowledgement. I mean, we have a few months, an acknowledgement that, or a recognition that they do have levers to pull to slow down Chinese AI development. And obviously there’s like, we’ve had, uh, uh, a lot of hand-wringing about, should we export the chips not export the chips?

This distillation thing seems to really have pissed them off. Um, the idea that there is like US IP getting stolen, that they’re sort of like tail-ending or piggybacking off what we, um, have developed. You saw some tweets out of, uh, uh, Kratsios and, uh, secretary Bessent saying, we know, Kimi, we’re onto you.

Like we’re in your systems. We saw you distilling.

MALLABY:
Um, but where does that go in policy terms?

SCHNEIDER:
I mean, it’s, you know, I mean, I think we’ll see by the time we’ll, we’ll, we’ll see how fast this production team gets, uh, gets this out. But I got a feeling over the next few weeks, I mean, there’s, there’s, there’s lots of there’s, there’s leverage.

MALLABY:
Everybody distills everybody, right? Elon is admitted to selling, uh, one of the frontier labs, um, you know, thinking machines distilled the Chinese model.

SCHNEIDER:
I mean, my, my brother can take $20 out of my wallet, but like someone on the subway can’t. Um, I mean, I, I think, I think like, look, you can put them on sanctions lists. You can put them on enemy lists.

You can file lawsuits. You can make sure that they get no revenue from the U S by making it illegal to do business with these, with, um, uh, you know, with the likes of, of Kimi or whatnot. Like I gave Kimi 20 bucks cause I wanted to try out the model this week.

Like if that’s illegal, that’s a big thing.

MALLABY:
That’s a, that’s a, um, you know, maybe, but we were saying earlier that their game is not really commercial anyway.

SCHNEIDER:
Yeah. Look, um, again, we’re, we’re, we’re, we’re, we’re feeling towards the, the, the world in which you want to do, uh, the most cyber hardening America and make the Chinese models as bad as you can, or like slow their curve. Right.

I think we are, we are, we have, we, we haven’t quite, uh, or this administration hasn’t quite pulled all the levers to live in that.

MALLABY:
But let me put it to you like this. Okay. So, so you’re, you’re saying there is a, there is a coherent policy which consists of the U S wins and China is like behind.

And, um, I guess I’m doubtful about that because at least in terms of frontier capabilities, as you said yourself earlier, the gap has shrunk.

SCHNEIDER:
Yeah. Well, well, the idea is you, it’s two things, right? You, you have this structural lead and how many chips you have.

Right. So, um, look, if I, if we have the same, if, if, if, if, if the models flattened now, right. And Kimi’s, Kimi’s newest model, uh, Kimi K4 is exactly as good as what Claude has.

Um, the MSS will have a, uh, an order of magnitude or probably two orders of magnitude, fewer chips to try to do hacking.

MALLABY:
That’s what the MSS is.

SCHNEIDER:
Uh, uh, ministry of state security, Chinese NSA. Uh, well, there are lots of other things, but like, uh, Chinese, Chinese, uh, hackers, um, uh, you know, gangs in, in, in, in Russia, the North Koreans will just have less firepower to shoot their models into hacking things than the world. And yes, there’s like a balance and we’ll see how that plays out.

But, um, I still think like, it’s not hard. You talk to cyber people and they’re just like, look, it’s going to be real, be a really awkward two or three years, but we’re going to end up at a point where you’ll have, you’ll have, um, uh, the ability to sort of like monitor this stuff really well. And your code will get so good that’ll actually be like, you know, provably cleared.

Um, right. So I think that is a potential world you can live in where you don’t end up having an accord and you don’t have cyber, cyber.

MALLABY:
Okay. So, so, so one question about that vision is, um, maybe with cyber hacking, uh, you can imagine a world where American models are so fantastic that we harden everything, by the way, you’d have to harden, not just in the U S you’d have to harden globally because you have things called multinational companies, U S ones, which do business in Korea and France and wherever.

SCHNEIDER:
Intelligence too cheap to meter. Well, we’ll get there.

MALLABY:
Sure. Why not? So we’re going to harden all the French banks too is the point.

And that kind of America first vision here doesn’t work. We do. We agree on that.

SCHNEIDER:
Well, yeah. Or, or, um, uh, the, the bank, the American banks are the only ones that you can trust to keep your money safe. So then we have even the, the, the dollar is just, um, uh, is backed by the, the, the shield of, um, uh, American models.

MALLABY:
So my, my sense from talking to American chief executives is that when they, you know, hear the administration’s policy, what they point out is, Hey, we do business all over the world.

SCHNEIDER:
I was being sarcastic.

MALLABY:
Okay. Okay. Okay.

SCHNEIDER:
Um, so, so one issue here is that, you know, I mean, well, staying on that for a second, like, I, like the, the, the, the, the countries I would be worried about is the, like, you know, the Nigerian central bank and the Kenyan central bank where like maybe the financial, you don’t quite have as much sophistication. There are still like an, you can still extort a lot of money. You can, um, there is a piece about how Boko Haram was like using these models to figure out how to like, you know, set up their guys to like rate a village or something.

MALLABY:
Nobody’s going to sleep.

SCHNEIDER:
Um, but yeah, like once you, once like the, the, I think like larger, more functional governments will end up getting to the other side of this, but, um, but, but look, so, so, so one set of issues around this vision of let’s have America win and harden systems is that you got to harden all the systems.

MALLABY:
Okay. Not easy. Second thing is there might be other kinds of threat coming down the pike where the defensive option of hardening systems doesn’t exist.

And I’m thinking about bioweapons, right? The, the, the individual wetware human beings who wandered through the shopping mall, you can’t really harden them. So if the bad guys have used AI to build a bioweapon, there’s no defense.

And I said, but the pushback I get on this point is, well, actually AI is not going to help build bioweapons.

SCHNEIDER:
Is that your position or maybe I don’t, I, I don’t have a good answer on this. I think like, I think the, the, the, the cyber risk is the one that is like so obvious that it’s staring everyone in the face that it is like re like, I feel like it is possible to have mental models about, um, the, the virus one is, I, I’m, uh, you should, you should have another guest to, to, to dive, to dive deeper in it. I mean, it’s, it’s interesting, like, to what extent, again, because China, uh, the Chinese system has like not really bought in or, or, or they, I mean, as of today, they see the calculus of, of, of the cyber stuff as something that they can deal with.

Right. Um, will they get to a point where they see the calculus of the bio stuff as something they can’t deal with? Yeah.

I don’t know. And then also on a, on a sort of like, uh, when you’re talking about a negotiation, right? Like either they’re going to believe it or they’re not going to believe it.

And if they don’t believe it, then like, okay, we’re like giving them chips to do something to save humanity. I don’t know. It’s, it’s just, especially given how, sorry, I’m going to take us down.

You’re the interviewer.

MALLABY:
So I guess, um, just to back out, what we’re discussing here, um, is whether there is a plausible strategy that makes America safe, which consists of saying America is going to win this AI race, uh, because we have more compute. Um, and the Chinese will not be able to damage us and we will live a happy and secure life, um, without needing to talk to them about closing down their open weight production cycle. And I’m just skeptical of that.

I’m skeptical partly because there are certain kinds of capabilities that a future open source model might have like bioweapons, where there is no, you know, however strong our AI is, I’m not sure it enables us to protect their civilians who wander through the shopping mall. I’m also skeptical because just on cyber, um, you know, you got to remember the famous, um, you know, terrorist line to Margaret Thatcher, which was, you know, when, when they blew up her bedroom and she happened to be in the bathroom, she survived, they put out a statement saying, you know, you have to be lucky every time we only have to be lucky once the attacker in most of these threats scenarios can try multiple times.

It only has to work once the defender has to succeed every single time. And so there’s a fundamental asymmetry in these cyber warfare situations, you know, where, where the threat attacker may well have the upper hand. So I worry about that as well.

So I’m just skeptical that we can move forward to a future where, unlike in the nuclear scenario where we had a non-proliferation regime, um, where, you know, happy as Larry, uh, without anything equivalent.

SCHNEIDER:
Yeah. Two things. I would, I would like bound my comments on this of like over a two to three year, uh, window.

Like I, I would prefer the worlds in which, um, the, uh, well, I mean, excuse me, over, over a 10 to 20 year window, I prefer the worlds in which the U S has more compute to just do whatever it wants, as opposed to in many futures, that is just like a better hand to have, um, on the cyber piece. One like interesting nuance of that is like capex of cyber used to be basically nothing. And now there’s real capex, um, uh, involved in, in, in doing this for the attacker and the defender.

Yeah. Well, I mean, in, in, in our, in our example for the, for the attacker in particular. So you are going to need a data center.

You’re going to need, you know, multiple millions of dollars of these chips and to be able to keep them running. And I think that kind of like sort of at a certain point in, in, in, as this evolves and the, the, the models keep getting bigger and bigger, such that like your table stakes for being a cyber hacker, keep going up, you know, hopefully we get to a point where it actually comes back to the nation States. And then, all right, we’re, we’re, we’re crossing our fingers that North Korea and Putin and Xi don’t want to like, uh, do a, uh, you know, kill a hundred million people with a, with a bioweapon or something.

So, um, that’s a, that’s a, that’s a hopeful thing, but, um, I don’t know, I think we’ll like the dynamic that needs to happen. And this is sort of what happened with climate change over the course of the, um, uh, 2010s really is, uh, the Chinese government just convinced themselves that it was a problem. Um, and we’re not there yet.

And there may be futures where Xi is right. And the, the, all the doomsday scenarios that people have been freaking out about since they’ve read, you know, the Culture series like are not actually real, but you’re referring to the science fiction series, the Iain Banks series. Okay.

Just, just for people who don’t know. But, um, uh, uh, but if he’s wrong, like, look, this guy’s changed his opinions on things. And, um, I, it’s much more, um, uh, it’s much easier to come to some settlement when both sides agree on the problem.

Right. Um, as opposed to like, you know, looking at Iran right now, it’s like someone wants to straight open. Someone wants to straight close.

Like how you, it’s not that easy to solve. Yeah. Yeah.

MALLABY:
So, so you would, but you’re basically advising me to be patient. You’re saying, um, you know, I’m feeling anxious. I feel like we need to get a deal on controlling open weight models out of China sooner rather than later, because we’re getting to a point where these capabilities are getting scary.

Um, and you’re telling me just call it, you know, we’ll see the Chinese may come around to the idea that this is scary stuff. If it is objectively scary, they will see that they’re not dumb. And at that point we can talk to them, but don’t force it yet.

SCHNEIDER:
Look, once Kimi deployed in China does the thing that OpenAI models did this week, that’s going to free people out. There’s no way it won’t free people out once it, you know, like, oops, I just like released some, you know, secret party documents or something. Um, there’s going to be, there’s going to be some response function to that.

If, if it doesn’t, because alignment capabilities have advanced to the point that when they release a model that this is good, like you don’t get those problems, then maybe we’re in a world where it’s okay. But, and so that’s whatever, give that a 30% chance in the 70% chance where that’s not the case. Like the Chinese state is going to respond in some sort of way.

And once they kind of internalize the fact that these things are dangerous and need to be controlled, then like you may not even need your global summit. You may not even need to give any chits because they recognize that there’s, that there’s risks to this technology or it’s just a much easier conversation. And it’s the sort of conversation that, you know, you have about global financial crisis where it’s just like, yes, we’re all on the same side.

We want, you know, we don’t want another great depression. Let’s like, see how we can use, uh, use our tools together. Um, and I think you’ll, you’ll, you’re more likely than not to just get there now, naturally, if it’s something that’s really needed.

MALLABY:
Okay. Well, we should wrap up. I could talk to you forever, but this has been great.

Uh, I’ve ended up feeling somewhat reassured that we’ll get to a deal with China, um, when we need to, or even without a deal that we’ll have a mutual agreement on being safe. Uh, thank you for joining us on The Spillover. Okay.

Thanks. Want to stay up to date on the latest episode of The Spillover? Sign up to receive an email alert when new episodes drop at cfr.org slash newsletters or click in our show notes. If you have an idea or just want to chat with us, email podcasts at cfr.org. Be sure to include The Spillover in the subject line. This episode was produced by Molly McAnany and Gabrielle Sierra.

Our video editor is Claire Seaton. Our audio producer is Markus Zakaria. Special thanks to Justin Schuster and Todd Yeager for their recording assistance.

You can subscribe to the show on Apple Podcasts, Spotify, YouTube, or wherever you listen to podcasts.