The Power of the Collective Purse: Open-Source AI Governance and the GovAI Coalition

In the absence of substantive AI regulation, procurement might be a promising pathway to bridge the AI governance gap. Typically, when we think of the “power of the purse” in AI procurement, we think of federal procurement because setting standards for AI vendors through conditional spending requires a combination of institutional expertise and significant capital. Conventional wisdom suggests that local governments, and even state governments, do not possess the requisite bargaining power to compel vendors to comply with AI governance requirements. But that is only true when localities stand alone in their efforts to influence the private sector’s AI practices. GovAI seeks to change that.

Launched in October 2023, the GovAI Coalition was spearheaded by US cities, including San Jose, Cleveland, San Antonio, San Diego, St. Paul, and Long Beach, along with the County of San Diego, the Tri-County Metropolitan Transportation District of Oregon, and the State of Colorado Department of Revenue. Finding strength in numbers, the coalition represents a collective effort to standardize and improve AI governance across various levels of government, leveraging pooled resources and collective bargaining power to level the playing field with AI vendors. Now, with over 300 members spanning federal, state, county, and municipal agencies, the GovAI coalition is poised to help reshape the landscape of public sector use and procurement of AI, fostering efficiency, transparency, and, ideally, responsible governance.

Collective Bargaining Power: Power Through Pooled Resources

In 2023, collective bargaining took center stage when workers and unions used their combined strength to negotiate protective measures in the face of AI’s existential threat to the arts. Now, GovAI is using a similar playbook.

Collective bargaining lives and dies by the maxim: “The whole is greater than the sum of its parts.” While local and state governments may lack the financial leverage to demand compliance from AI vendors independently, they can achieve significant leverage in the aggregate. In fact, when viewed collectively, state and local procurement expenditure significantly outweighs that at the federal level. In 2019, state and local governments spent approximately $1.3 trillion on public procurement, compared to the federal government’s $586 billion. This disparity highlights the untapped potential of pooled capital among local and state entities, particularly as the demand for AI solutions in public administration grows. With the rise of "smart city" initiatives, investors expect global smart city AI revenues will surge from $693.3 million in 2023 to $6.5 billion in 2032.

The GovAI Coalition capitalizes on this collective strength by determining standards that vendors must meet to access the substantial procurement capital managed by coalition members. For example, the Coalition developed the Vendor AI FactSheet to improve the transparency and explainability of vendor products. Functioning as an AI “nutrition label,” the FactSheet requires vendors to provide details about their AI systems, such as the model’s training data and update procedure. At first, Coalition members struggled to independently demand vendors to complete the FactSheet. However, as a collective, members have achieved greater success and minimized duplicative efforts by utilizing the vendor registry, a shared repository of FactSheets. The Coalition openly and democratically develops standards like the Vendor AI FactSheet, and members, and even non-members, benefit in kind. While only government bodies with formal approval from their jurisdiction’s leadership to join the Coalition possess voting rights, policy negotiations occur in meetings open to anyone. GovAI even invites vendors to the weekly meetings—what’s more, the vendors want to participate.

It behooves vendors to support GovAI’s development of new, open, and standard procurement policies. The Coalition’s gatekeeping function benefits vendors by reducing the regulatory burden of navigating a nationwide patchwork of requirements. By aligning with Coalition standards, vendors ensure compliance without sacrificing efficiency. GovAI’s vendor registry helps streamline this standardized approach to procurement policy. By maintaining a central database of cooperating and compliant vendors, Coalition members and uninvolved government bodies can benefit from an unofficial pre-qualification list that reduces vetting costs. Inclusion on such a list would be a competitive advantage for vendors. Enticed by these benefits, vendors actually participate in the coalition’s open meetings. Although primarily passive observers and forbidden from marketing their products or services, vendors do chime in to provide feedback on proposed requirements from the industry’s perspective or to rectify incorrect assumptions about the technology.

Now that state and local governments have the industry’s attention, the Coalition’s grassroots origin improves the odds that the public sector’s AI usage will internalize the public’s best interest. This ground-up approach to AI standard-setting embodies the call for AI Localism to fill the governance gaps left at the state, national, and international levels. Local agencies’ proximity to their constituents better positions them to identify and address specific concerns related to public sector AI use. Greater proximity also allows for more efficient public feedback loops that drive improvements in AI deployment and governance. By meaningfully involving local governments in setting AI standards, the coalition ensures that these standards are both practical and responsive to the needs of diverse communities. The Coalition really has heard from voices across the size spectrum: San Jose (pop. 971,233) might lead this initiative, but major working groups such as LLM use cases and small government adoption are run by Nederland, Colorado (pop. 1,500) and Lebanon, New Hampshire (pop. 15,044) respectively.

Open Source: Leveraging Collective Knowledge for Greater Impact

Collaboration and transparency often go hand in hand. One of the most significant outcomes of the GovAI Coalition’s work is the development of open-source resources that benefit not only coalition members but also vendors and uninvolved governments. By pooling resources and expertise, the coalition is creating a shared repository of guidelines, contracting language, and best practices that any government entity can adapt to their specific needs. This collaborative, open-source initiative greatly reduces the transaction costs for government agencies, particularly those that are understaffed or under-resourced. While the more expansive budgets and technological needs of larger state and local governments sometimes lead to outsized roles in Coalition standard-setting, this allows smaller local governments, which may lack the capacity to develop comprehensive AI governance frameworks independently, to draw on the Coalition’s collective institutional expertise. This crowd-sourced knowledge ensures that even the smallest agencies can implement robust AI governance policies without having to start from scratch.

The benefits of open-sourcing extend beyond the public sector. For vendors, open-source standards provide insight into the public sector’s AI needs and the governance standards they expect vendor solutions to meet. This transparency allows vendors to align their product development with the expectations of multiple government agencies simultaneously, leading to more efficient product design and a smoother compliance process. Additionally, by participating in the development of these open-source resources, vendors can influence the evolution of these standards in ways that make compliance more manageable.

For the public, open-source practices allow voters to hold both their government and AI vendors accountable. Citizens can scrutinize the AI tools being adopted, the governance frameworks in place, the incident response plans established, and the effectiveness of these systems in delivering public services. This accountability loop ensures that AI technologies are not only effective but also ethical, fostering greater public trust in both AI systems and the governments that deploy them. Moreover, the pooling of open data among coalition members helps improve the performance of AI tools for public sector use cases, with smaller government bodies benefiting from the self-interested actions of larger ones. Everyone wants better AI, but better AI requires a lot of data, and larger jurisdictions will have more data to donate than smaller ones. For example, when a large city donates multilingual data for the purposes of improving AI translation models, the improvements benefit everyone involved—vendors develop better tools, agencies receive more effective solutions built for their unique needs, and citizens experience better public services. Home to the largest Vietnamese population in the United States, San Jose utilized its own Vietnamese language data from public queries to improve its Vietnamese translation model by 5-10%—gains they hope to share with others. Open-sourcing this pooled data is one of the Coalition’s key goals for its next phase.

Incentive Alignment: A Path to Mutual Benefit

The GovAI Coalition exemplifies how public and private sector incentive alignment can drive mutually beneficial outcomes. “Alignment” has been a buzzword of late—AI’s white whale. Technologists refer to the chance that humans may build an AI system with one set of goals in mind and end up with a system built to achieve different goals. But, in the context of AI governance, incentive alignment means that both government agencies and AI vendors have a shared interest in standardizing and streamlining procurement processes.

For government agencies, the incentive is clear: by banding together, they can wield collective bargaining power to level the playing field with AI vendors and ensure that procured AI technologies meet ethical and operational standards. In theory, the coalition’s gatekeeping force helps mitigate the risk of large tech vendors’ negotiating power overwhelming any single agency. Communicating with one voice, agencies can demand AI tools that suit the needs of their unique constituents.

For vendors, the alignment is equally beneficial. Complying with a unified set of standards across multiple jurisdictions reduces the complexity and costs associated with navigating a patchwork of regulations. This efficiency not only makes it easier for vendors to scale their products but also enhances their ability to innovate within a stable and predictable regulatory environment. Moreover, the coalition’s open-source resources provide vendors with insights into the public sector’s AI needs, enabling them to better tailor their offerings to the market.

Most notably, efforts like the GovAI Coalition may shed light on how to align the market’s goals with the public interest. Given state and local governments’ greater proximity and accountability to their constituents, they are better positioned to represent the public’s perspective. Harnessing the power of collective bargaining, they can demand procurement standards that go beyond cost and functionality—they can demand safer, more ethical AI. As vendor alignment with the standards set by the GovAI Coalition increases, so too does the potential for these responsible AI practices to spill over into the private sector. Companies uninvolved with the coalition may still feel pressure to comply with these standards, either to remain competitive or to preempt future regulatory requirements. This could lead to a broader shift towards more responsible AI practices across the industry, benefiting society as a whole.

A Nascent but Promising Path Forward

The GovAI Coalition represents a nascent but promising approach to AI governance in the public sector. By leveraging collective bargaining power, developing open-source resources, and aligning incentives between the public and private sectors, the coalition has laid the groundwork for a more equitable and efficient AI procurement process. However, it’s important to recognize that this approach is still in its early stages, and its long-term success depends on several empirical assumptions that have yet to be fully tested.

One key assumption is that the collective power of the GovAI Coalition can genuinely balance the entrenched power of AI vendors. While the coalition’s pooled capital and standardized requirements are significant, the influence of large tech companies in the AI market remains considerable. Vendors may still have more leverage in negotiations, particularly if they hold proprietary technologies that are in high demand. The market for AI is especially concentrated, with a handful of already large companies taking the lead in AI offerings. Given the intensive data and resource demands of AI development, there is good reason to think these titans will maintain their market dominance. The assumption that vendors will readily align with the coalition’s standards for the sake of efficiency could be overly optimistic, especially if those standards are perceived as too stringent or costly to implement.

Another assumption is that the open-source nature of the coalition’s resources will lead to widespread adoption and accountability. While open-source practices offer many benefits, they also rely on active participation and engagement from all stakeholders. If agencies or vendors do not fully buy into the open-source approach, the potential for transparency and collaboration may not be realized. Moreover, the effectiveness of these open-source resources in fostering public trust will depend on how well they are communicated and how accessible they are to the average citizen.

Despite these challenges, the GovAI Coalition offers a compelling model for how governments can work together to navigate the complex landscape of AI governance. By watching and supporting this initiative, stakeholders across the public and private sectors can help ensure that AI is developed and deployed in ways that are not only effective but also ethical and responsible. As the coalition continues to grow and evolve, its success could pave the way for broader adoption of similar models, leading to more responsible AI practices across the board.