The Open-Weight Clarification
Dario Amodei has never been shy about his worries on AI safety. But last week, the Anthropic founder and CEO felt the need to draw a line in the sand — or at least, in a blog post — because the industry was starting to misrepresent his position. Rumors had been swirling that Anthropic somehow backed efforts to ban open-weight AI models, or open-weight models of any origin. Amodei shut that down hard.
"Anyone who has read my past writing should know that I don't regard such bans as a useful measure, but let me state it clearly so that there is no doubt: Anthropic has never advocated for a ban on open-weight models," he wrote. Emphasis was his own.
The timing wasn't accidental. Nvidia CEO Jensen Huang had just published an open letter on X — his very first post on the platform, incidentally — rallying a broad coalition of companies including Hugging Face, Meta, Microsoft, and Mistral around the idea that policymakers should not impose "premature restrictions" on open-weight AI. The letter didn't name China. But everyone in the room knew exactly who it was talking about.
Here's the thing: Amodei isn't arguing against open models. He's arguing about which open models, and who ends up wielding them. It's a distinction that matters, and it's one that keeps getting lost in the noise.
The Distillation Problem
The real tension in this debate isn't philosophy. It's practice. Chinese AI labs have been demonstrating rapidly growing capabilities, and American researchers have been raising eyebrows at how. One method, called distillation, works like this: a Chinese lab takes an open-weight model, bombards it with prompts, and listens carefully to how it responds. Over time, the lab trains its own model on those responses. The result? A model that's far more capable than it should be, built on the intellectual property of American researchers who released their weights freely.
Amodei called this out directly. He's not worried about businesses in China using open-weight models — those are, in his words, "a public good" that "don't cost anything besides the compute needed to run them, and they provide value to businesses, developers, and researchers." He's worried about state-level actors using distillation to close the gap on American AI, potentially leapfrogging U.S. models entirely.
His proposed countermeasures are straightforward: restrict China's access to powerful chips (this has been longstanding U.S. policy), and launch a formal crackdown on distillation. The U.S. has already threatened sanctions against China if it determines that IP theft involving U.S. models is happening. Amodei is backing that hardline approach.
For context on how these distillation allegations have escalated, see our coverage of Anthropic's allegations against Alibaba and the White House accusations against Moonshot AI.
Biological Weapons and the Open-Source Dilemma
Amodei's concerns don't stop at military applications. He's also worried about biological weapons — and here's where the open-weight debate gets genuinely uncomfortable. Open-weight models, by their very nature, are harder to guardrail. They're harder to monitor. Once they're released, they can't be withdrawn.
He cited a U.K. AI Security Institute report to make this point. The message is stark: you can't take the genie back once it's out of the bottle. And if someone — whether a state actor or a malicious individual — uses an open-weight model to design a biological attack, there's no mechanism to stop it. The model is already everywhere.
This directly challenges what open-source advocates argue: that having access to powerful open models helps defenders protect themselves. Amodei disagrees. Or rather, he disagrees about the high-end models. He thinks models without dangerous capabilities are fine — even beneficial. But the most capable ones? Those are a different story.
A Global Testing Framework
Here's where Amodei's position gets interesting — and maybe, just maybe, a bit hopeful. He supports creating a global model safety testing organization. And he wants it to include China.
"I think this idea is actually close to a consensus," Amodei wrote. He's been "heartened" by the Trump administration's recent moves in this direction, as well as by industry proposals that would apply such testing to the most capable models regardless of country of origin or whether they're open or closed. Less capable models — think startups and academia — would be exempted entirely.
But there's a catch. For this to work, it has to be global. That means even the CCP needs to be on board. Amodei thinks that's possible, though. He argues that "limited cooperation around preventing AI biological weapons may be possible because it is in China's interest too."
It's a pragmatic take. And it's worth noting that Amodei doesn't just worry about the CCP. He says authoritarian governments broadly are his concern, but the Chinese Communist Party is "the most capable." That distinction matters — it means he's not making ideological arguments here. He's assessing capability, and China is winning that assessment.
The Bottom Line
Amodei's position is nuanced, and he's clearly trying to push back against a narrative that never really existed: that Anthropic wants to ban open-weight models. They don't. They want open-weight models for what they are — useful tools for businesses, developers, and researchers. But they also want guardrails on the most powerful models, especially when those models might end up in the hands of authoritarian regimes capable of using them for permanent military superiority or internal repression.
The debate will continue. It should. But Amodei has at least made his position clear: open-weight models are fine, until they're not. And when it comes to China's rising AI capabilities, he's not taking any chances.
Related reading: How export controls have shaped Anthropic's frontier model strategy | How national security mandates are reshaping AI model releases
