Supporting Openness & Safety — Lessons From Recent State Laws

This summer, Nvidia, Microsoft, and more than 250 other organizations signed a letter making the case for “open weight” AI. The letter argued that downloadable open-weight models can expand access, strengthen competition, give users greater control, and contribute to security—and urged policymakers to address genuine risks through targeted measures rather than sweeping restrictions.

The letter punctuated a long-running debate over how emerging safety rules should accommodate open source and collaborative technology development. For longtime Tech Policy Press readers, the point is familiar. Articles published here have highlighted not only the importance of openness, but also that openness is not binary. It exists on a spectrum and tends to advance collaboration, competition, autonomy, privacy, and user control.

Policy proposals to mitigate the risk of harm to users and society abound, especially in AI. And all too often, those policy proposals implicitly assume a centralized model: one company develops a product, controls its distribution, and can govern what happens downstream. But open source development often does not work that way.

Fortunately, in recent California debates, open source developers and community members were able to identify those mismatches, engage with lawmakers, and secure changes that preserved openness without abandoning safety goals. These examples show that responsible solutions are possible, but they also expose a capacity gap: achieving them requires technical and legal expertise to engage in the legislative process.

That need will only grow, as policymakers at every level continue to consider AI, child-safety, and other technology bills that may affect open development. The lesson is not that “open” should be exempt from regulation, but that obligations should reflect an actor’s actual knowledge and control rather than broadly requiring developers to monitor and control downstream uses.

The what and why of openness

Traditional open source development does something simple yet powerful: it gives downstream users reliable permission to use, study, modify, and redistribute software.

That enables collaboration and improvement. A developer can build on someone else’s work, fork it, adapt it for a different community, or continue maintaining it if the original developer changes direction. Widely used open source software also provides durable permission: the original developer generally cannot simply pull the rug out from under everyone who built on the software; permission persists for all those who downloaded it.

That architecture also supports autonomy. Tools that can be downloaded, modified, and run locally allow people and organizations to operate without continuously sending information to, or seeking permission from, a centralized provider. Users can customize software to their own needs and values and, in many cases, keep sensitive data on their own machines. As Mozilla chief technology officer Raffi Krikorian recently framed the question, one important choice in AI is whether users will ultimately own or rent the technology on which they depend.

Sometimes, people mistake openness—and the ability to modify software or an AI model—as indicative of safety or security, or lack thereof. But open systems can incorporate safety testing, documentation, secure development practices, and other precautions. And many harms depend not simply on a model’s characteristics, but on the context in which it is deployed. As Arvind Narayanan and Sayash Kapoor have argued, AI safety is not solely a model property: deployment and surrounding systems matter too.

The policy question, then, is not binary—whether to regulate or not—but instead what obligations make sense for which actors in which contexts.

Case study #1: Age attestation

California’s Digital Age Assurance Act illustrates the challenge.

Enacted in 2025, the law requires operating-system providers to obtain a user’s age or birth date during account setup and make an age-range signal available to applications. That makes intuitive sense for something like iOS or Windows: a centrally maintained product with an identifiable provider, an account-setup process, and infrastructure capable of supplying an API.

But those assumptions do not map neatly onto much of the open source ecosystem. Linux distributions and other community-developed operating systems may be maintained by nonprofits, small teams, or volunteers, and may not have centralized user accounts at all. Requiring every such project to build age-attestation infrastructure could force open projects to fundamentally redesign how they operate.

What’s more, the law also appeared to treat contributors of software code incorporated into an operating system or application as responsible for implementing the requirements. Open source organizations, developers, and users warned that the resulting legal and technical burdens could make it impossible for smaller projects to continue serving users in covered states.

Legislators took these concerns into account. They revised the bill to make clear that the law applied to stand-alone consumer applications and operating systems, not merely software components incorporated in building those tools. The revised bill also provides that an “operating system provider” does not include an entity distributing an operating system or application under license terms permitting recipients to “copy, redistribute, and modify” the software. Colorado adopted a similar approach in its age-attestation legislation.

These changes preserve the core intent of the bill while recognizing that requiring decentralized software projects to construct a centralized age-monitoring mechanism is a poor match between means and ends. The age rules can still operate at layers where companies possess the infrastructure and relationships necessary to implement them.

Other contexts may call for different approaches. The European Union’s Cyber Resilience Act, for instance, largely excludes free and open source software supplied outside commercial activity while creating a lighter, tailored regime for certain “open-source software stewards.” The precise solution will depend on the problem, but the underlying logic is similar: distinguish actors by their actual role rather than treating the software ecosystem as homogeneous.

Case study #2: AI and provenance

California’s AI provenance law offers a second example.

The California AI Transparency Act, originally enacted as SB 942, requires covered generative AI providers to include machine-readable provenance information in certain AI-generated content. That is a legitimate policy objective: provenance can help platforms and users understand where digital content came from and whether AI played a role in creating it.

But the original law created a problem for open source development. If a third-party licensee modified a covered AI system so it could no longer produce the disclosures required by SB 942, the original developer was required to revoke that party’s license to the underlying system.

That remedy assumes that the upstream developer retains continuing control over downstream permission to use the software. Open source licenses intentionally reject that premise. A coalition including GitHub, Mozilla, Hugging Face, and Black Forest Labs argued that mandatory revocation of permissions was incompatible with the terms on which collaborative software development depends.

California legislators worked to address this concern in this year’s SB 1000, which also expanded the reach of the AI Transparency Act to smaller providers. Instead of requiring revocation as the only response, the revised language would allow a provider that learns of a downstream violation to notify the user. The downstream party must then bring the system into compliance or stop using or distributing it; if it does neither, the provider reports the violation to the Attorney General. The bill also expressly says providers have no duty to monitor or investigate downstream modifications.

The provenance requirement survives. What changes is the enforcement mechanism. Rather than forcing the upstream developer to maintain a technical or legal kill switch over everyone downstream, responsibility can fall on the actor actually violating the requirement.

Looking ahead

These sorts of questions will become more important, not less. States are considering frontier AI safety rules, pre-deployment testing requirements, product and tort liability, child safety measures, and other regulations that may reach increasingly far down the technology stack. In fact, Illinois passed an age attestation bill that mirrored California’s original version, while lacking the clarifications related to openness.

Without sustained participation from developers, community organizations, and experts who understand decentralized development, lawmakers may not know when a proposal rests on assumptions that do not fit the technology.

The California examples are encouraging because intervention worked. Developers and advocates helped identify alternatives that preserved the policy objective while avoiding unnecessary surveillance, control, or incompatibility with open source development.

Doing that consistently in Sacramento and beyond will require more capacity and resources than the open source community has historically devoted to state policy. There will be many more AI bills, and many more opportunities either to get these details right or miss them.

The goal should not be less responsibility. It should be responsible regulation: rules aimed at the actors who can actually prevent harm, designed with enough understanding of open systems to preserve the benefits policymakers were trying to protect in the first place.