Is Open-Source AI Really the Dangerous Path?

Raffi Krikorian is Chief Technology Officer at Mozilla.

In Washington, AI is increasingly being treated as something that needs to be controlled. The government believes that AI is, first and foremost, a national security asset, meaning that it must be sequestered to prevent enemies from gaining an advantage. On the other side of the world, in Beijing, the approach is moving in the opposite direction. China is reducing barriers, encouraging adoption, and using open-source AI as a way to spread Chinese-developed technology across global markets.

There is now a fundamental divide. The United States is betting that control is the path to preserve its lead. China, instead, is betting on diffusion. The country whose technology is adopted most widely may ultimately shape the future of AI. Questions over open source and open weights sit at the center of that contest.

Beginning on July 24, high-profile support for open source moved what is often a debate behind closed doors into the public sphere, where it belongs: Nvidia’s Jensen Huang’s first-ever post on X linked to an open letter signed by 35 companies — including Palantir, Andreessen Horowitz and Microsoft — warning Washington not to over-restrict open source software. “Open models strengthen safety and cybersecurity, accelerate innovation and diffusion, and enable sovereignty,” Huang wrote, leading the likes of Elon Musk and Mark Zuckerberg to post their support.

Mozilla signed it because, despite being a very different company from many of the signatories and not always seeing eye to eye, we believe in the spirit and substance of the letter, particularly that 'openness may be one of the most important paths to AI safety and security’. The letter was followed by the announcement of the Open Secure AI Alliance for AI Safety and Security, which aims to “build and share open tools that promote responsible use of and trust in AI.”

The battle is on. Here’s what’s behind it.

Following the money

Open models now do about a third of the world's AI work, but they collect only about four percent of the money.

Those two numbers, taken from Mozilla's new “State of Open Source AI” report, provide more context than the entire AI safety conversation. The report tells a story of a performance gap between open models — the ones whose weights anyone can download, run, and adapt — and the best proprietary systems. While the capabilities are still a jagged frontier, on average, the performance has narrowed sharply over the past year. Costs keep falling. Seventy-nine percent of developers now build with open models. And yet the money hasn't followed the usage.

This is where the battle lies. A third of the work, but only four percent of the money — that gap is the prize.

The battle started almost exactly three years ago, when Anthropic's Dario Amodei told Congress that advanced open-source AI is on a "very dangerous path." On the surface, his argument is pretty simple: once a model's weights are public, no one can monitor abuse or revoke access. Once released, an open weights model can't be “unreleased.” But let’s ask ourselves what that revoke switch actually does. Revocation is not a feature of the model, it is in the API contract. That means it only applies to a lab's own customers — and the people in Amodei's threat model were never customers. The labs shipping frontier-class open weights — such as DeepSeek, Alibaba, Mistral, and Moonshot (with its just-released, 2.8T-parameter model Kimi) — mostly sit outside Washington's reach anyway. Two million open models already sit on Hugging Face; many run on a laptop. That means that in the real world, there's no single kill switch to throw.

This distance between what such a regulatory switch claims to control and what it actually does is what's missing from the debate over which models are "safer." It's also the key to the fight over who captures AI's value.

To the companies that built the proprietary models, value is about maintaining a privileged position and using everything at their disposal to protect it — policy, pricing, and technology. For everybody else, value means the ability and power to shape, audit, and improve the systems we all depend on. And increasingly, that power doesn't live in the model at all.

For instance, right now, two developers can take the identical open model and ship completely different products: a scam-call operation or a nurse-advice hotline. The model doesn’t know (or care) about the difference. What is making the actual decisions is the layer of software built around it — the agentic harness — that sits between users and the model, determining what the application can access, remember, and act on.

As models get cheaper, not to mention more interchangeable, that harness is where the power is actually going. And it's being quietly locked up by the big labs. Farmers know how this story goes. They bought their tractors outright, but the manufacturer kept the keys to the software, making the farmers owners on paper but renters in practice. It took years of lawsuits — and, just this month, the Federal Trade Commission — to start prying that lock back open. A similar arrangement is now being built for the software that reads your email, books your travel, and remembers every detail of your life.

This isn't an accident of engineering; it's a business model. A closed wrapper makes money by making itself expensive to leave. An open one can't lock the door, so it survives only by staying worth using. Same underlying technology, opposite incentives. It's the reason the value captured by open models sits at four percent while their usage sits at a third. The real question for all the builders right now isn’t which model you’re using. Rather, it's whether you could leave for a different one.

Guess who’s deciding the future?

The debate that matters isn't really which models get released or which get regulated; it's who controls the layer wrapped around them. That's being decided right now — mostly by developers who don't realize they're the ones responsible. For a glimpse of the future, we can look to the internet: it exists as it does today because, when the architecture was still up for grabs, developers chose HTML and HTTP over proprietary walled gardens like AOL. AI is at that same juncture now, and the fact that two million open models already exist suggests plenty of builders have shown up early. That window doesn't stay open on its own, and it doesn’t stay open forever. It stays open because people keep choosing it.

For developers, four habits matter most in ensuring an open future:

1. Build on open harnesses, not just open models. The orchestration layer above the weights is where capability is concentrating, and closed labs are already welding it shut. Keeping it open takes deliberate effort.
2. Own the memory layer. Store accumulated context in portable controllable formats, so it's retrievable if a vendor changes its terms rather than trapped inside one.
3. Keep a second model warm. Integrate an open model and keep it production-ready even while running primarily on a closed API, so switching is cheap if it becomes necessary.
4. Don't assume all open stacks are equal. Open models skew toward particular regions and providers; keeping this layer genuinely open means actively supporting a geographically distributed set of options, not defaulting to whichever model is cheapest this quarter.

None of this requires believing anyone is acting in bad faith. It's worth noticing, though, that the loudest safety arguments arrived right around the time models got cheap enough for the real competition to move up a layer. That's not evidence of a conspiracy — it's just where the incentives point, and it's why so much of the current debate is aimed at the wrong target.

More evidence is in our report, and most of it is good news: performance gaps closing, costs collapsing, millions of developers building. The question in front of developers isn't whether AI is dangerous — it's whether they'll hold the keys to the machines they're building. The question for governments is whether the keys they’re reaching for turn anything at all. For now, that door is still open. Let’s work together to keep it that way.