A Plan to Fund Independent Assessments of General-Purpose AI

In 1865, a catastrophic steam boiler explosion at a brewery in Germany resulted in one death and several people injured. The tragedy was preventable. A qualified technician could have spotted the fault, but regular inspections were not yet common practice. In response, over 20 entrepreneurs came together and founded an association for inspecting and insuring steam boilers. This was the beginning of what would become the first TÜV. Over time, this initiative grew into a globally recognized system of independent safety inspection, ranging from vehicle certification to medical devices.

Today, we face a far more complex and consequential challenge: the development of increasingly powerful general-purpose artificial intelligence. Unlike steam boilers, general-purpose AI carries the potential not just for isolated accidents, but severe systemic risks and national security threats. To avoid learning safety lessons the hard way, we believe the success of the TÜV model offers a compelling guide: create trusted third-party institutions before the first disaster forces our hand.

Recognizing these risks, several leading AI companies have developed frontier AI frameworks, outlining how they manage the safety and security of their most advanced models. Many have also signed the Frontier AI Safety Commitments announced at the 2024 AI Seoul Summit, which include pledges to support external red-teaming and third-party model evaluations.

These independent assessments have become recognized as an essential element of effective AI governance. Several studies have argued that assessments made by companies themselves are not enough, and a recent expert survey highlights third-party assessments as one of the most effective ways to mitigate AI risks. Regulators have also acknowledged the value of third-party assessments; notably, the EU’s Code of Practice of the AI Act specifically outlines scenarios under which independent assessments are required. Similarly, the recent US AI Action Plan highlights rigorous evaluations as a critical tool for measuring AI reliability.

The current assessment landscape is unsustainable

However, the current funding landscape for independent assessors is brittle and unsustainable. Many assessor organizations are relatively small and thus vulnerable to direct financial dependence on the AI companies whose products they evaluate. This financial dependence creates inherent conflicts of interest, risking the objectivity and credibility of assessments.

Other factors can also compromise independence. For example, an assessor might become overly reliant on regular contracts by a single AI company. In other cases, assessors may hold equity stakes or receive bonuses linked to an AI company’s financial performance. This is particularly likely given that some of the technical talent for assessments may come from AI companies and they usually cannot cash out their equity until after a certain period. Another risk involves assessors later accepting jobs at companies they previously evaluated, similar to regulators who move into positions at firms they once oversaw. Any one of these can erode critical scrutiny even when outright fraud never materializes. Empirical work on financial auditing shows such incentives systematically bias findings toward management-friendly outcomes.

History offers a clear warning. In the early 2000s, Enron’s auditor, Arthur Andersen, earned millions in consulting fees from the same client it was supposed to audit. When signs of fraud appeared, Andersen looked the other way rather than risk upsetting a valuable client. The result was one of the largest corporate collapses in history. Enron’s failure wiped out thousands of jobs and pensions. The lesson is clear: financial dependence undermines oversight.

At the same time, governments and the public are calling for external oversight of general-purpose AI models. Coupled with increasingly advanced AI models entering the market, there is a growing demand for independent assessments, yet still no suitable funding mechanism. Philanthropic support has helped to fill the gap so far, but it remains unpredictable and insufficient to support a robust, scalable ecosystem. To ensure the integrity and sustainability of third-party evaluations, a more stable and independent funding mechanism is urgently needed.

Pool the funding

To close the funding gap and maintain independence, frontier AI companies should pool their resources into a common AI Assessment Fund. Given its mandate to advance the science of AI safety and facilitate information sharing across the ecosystem, the Frontier Model Forum is well-positioned to seed this initiative by convening the founding members and providing the first tranche of capital (e.g., by expanding its AI Safety Fund).

Contributions to the AI Assessment Fund would primarily come from top AI companies and could be linked directly to their investments in general-purpose AI development. For instance, companies might contribute 0.05% of their total capital expenditures related to general-purpose AI development, ensuring that the fund scales automatically with the pace of development. This figure would correspond to approximately $10 to 50 million (as top companies’ capital expenditures range from $19 to $100 billion), which is in the same ballpark to average auditing fees paid by S&P 500 companies. Additionally, philanthropic contributions could bolster the fund, mirroring the collaboration model of the existing AI Safety Fund.

Crucially, the fund (not individual companies) would select and pay assessors. This “pot” model prevents the kinds of financial entanglements that have historically undermined oversight in other sectors, such as with the case of Enron. By severing direct commercial ties between companies and assessors, the fund helps preserve the credibility on which the oversight ecosystem depends.

Governance and implementation

To ensure effectiveness and transparency, the fund would need to include several key mechanisms. First, the fund should be administered by an independent non-profit organization that actively incorporates input from industry stakeholders and civil society representatives. This structure helps address potential antitrust concerns that can arise when competitors pool resources for shared infrastructure. There is precedent in the management model initially adopted by the AI Safety Fund, which was independently operated by Meridian Institute.

Second, the fund must clearly define what it takes to qualify as an independent assessor. This should include baseline eligibility criteria, such as a demonstrated track record in red-teaming and strong internal security practices. It should also outline a set of approved evaluation methods, with room for these methods to evolve as the technology advances. These requirements should be developed with input from industry, academia, and non-profits to ensure they are credible and balanced.

Funding alone is not enough; profession-wide rules, robust governance and other structural checks—like the four-eyes principle—are equally indispensable. Furthermore, assessors must have sufficient access, while guaranteeing the integrity of intellectual property and model weight security. Such access could be obtained with the help of encrypted analysis techniques and secure sandbox environments, among others. Finally, it is crucial to design and maintain a protected channel for assessors to report attempted interference and sanction companies that retaliate.

While long-term partnerships can offer efficiencies, they also risk creating dependence between assessors and the companies they evaluate. To mitigate this, assessors working with a single AI company should be rotated after a set period, which would mirror current obligations for companies listed in the United Kingdom. Similar practices exist in other fields, such as those used by TÜV organizations in the automotive industry.

Long-term sustainability: standards, insurance and liability

In the long run, creating a sustainable and trustworthy ecosystem of independent assessors requires robust standards, insurance incentives, and effective liability frameworks.

First, clear standards must be developed to guide how independent assessments are performed and what constitutes sufficiency of evidence. These standards should outline essential criteria such as assessment methodologies, transparency requirements, and safeguards for impartiality and accuracy. A formal accreditation process, managed by recognized accreditation bodies, should then be established based on these standards, providing assurance to regulators, companies, and the public that accredited assessors meet high standards of quality and independence.

Second, the insurance industry can play a pivotal role in incentivizing rigorous general-purpose AI assessments. If insurers offer substantial reductions in premiums for companies undergoing credible third-party evaluations, companies would have a clear economic incentive to support comprehensive and frequent assessments. However, realizing this incentive structure is challenging due to the systemic and global nature of potential AI risks, which differ significantly from more localized risks in industries such as the automotive industry. Therefore, robust liability laws are necessary to align incentives effectively. Liability frameworks should clearly assign responsibility for harm caused by advanced AI systems, motivating developers to invest in rigorous independent evaluations.

The technical foundation for an ecosystem of independent AI assessments is already taking shape, but success depends on three sets of actors. What is needed now is institutional commitment from AI companies, industry bodies such as the Frontier Model Forum, governments, and insurers.

If we wait until after the first disaster, we will have lost the chance to build it thoughtfully.