AI Safety Met the Guillotine in Paris. Good Riddance

Mark MacCarthy is adjunct professor at Georgetown University in the Graduate School’s Communication, Culture, & Technology Program and in the Philosophy Department.

On February 10-11, French President Emmanuel Macron hosted an AI Action Summit in Paris. It was the third major international conference seeking a consensus on global governance of frontier AI models.

The supposed triumph of AI innovation over AI regulation received most of the media coverage. But while that’s part of the story, it’s not the real news. The real lasting outcome of the Paris conference is the opportunity to once and for all marginalize the so-called existential risks of AI.

To understand this development, let’s go back to March of 2023 when the Future of Life Institute issued an open letter asking AI labs to “pause giant AI experiments.” The animating concern was: “Should we develop nonhuman minds that might eventually outnumber, outsmart, obsolete, and replace us? Should we risk loss of control of our civilization?” In May of that year, hundreds of prominent people signed a one-sentence statement on AI risk asserting that “Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.”

This was a clear call to go slow on AI development in order to avoid the extinction of the human race. Policymakers were embarrassed to be seen validating these science-fiction fears, but they were also afraid to dismiss them completely. So they embraced the malleable term “AI safety” as a way to talk about real AI model risks relating to national security, cybersecurity, and disinformation and to mollify the AI doomers.

But the phrase “AI safety” has always connoted preventing AI from spinning out of human control and threatening the existence of the human race. The Bletchley Park AI Safety Summit in November 2023 focused significantly on AI safety, and its output document specifically warned that today’s AI models raised “issues of control relating to alignment with human intent” and threatened “catastrophic” harm. The Seoul AI Summit in May 2024 continued this emphasis on AI safety to the exclusion of almost all other AI concerns.

The Paris AI conference, however, largely ignored these far-fetched risks. US Vice President JD Vance warned the attendees that “the AI future will not be won by handwringing about safety.” As the final statement from the conference made clear, the priority was on “making innovation in AI thrive by enabling conditions for its development.” The US and the UK could have cemented the marginalization of speculative AI risks by signing on to this document. They chose not to, and it was a missed opportunity.

Downgrading existential risks outraged AI safety advocates, as the Financial Times reported. Stuart Russell, a leading professor of computer science, said, “You cannot have innovation without safety.” Max Tegmark, head of the Future of Life Institute that organized the 2023 open letter, complained that the process to address existential AI risks, begun so promisingly at Bletchley, had been “guillotined” in Paris.

Turning attention away from speculative AI risks is all to the good, however. They have always been a distraction from the real and urgent risks that advanced AI systems pose. For years, AI researchers have warned that AI foundation models create serious new risks that have to be managed at the model level. These include: non-expert development of chemical or biological weapons, producing and distributing multi-modal disinformation and unprecedented offensive cybersecurity challenges.

The AI Safety Institutes set up in both the UK and the US in late 2023 focused on these real risks. Under voluntary arrangements with the frontier AI labs, they issued informative joint assessments of the latest models, including in late 2024 pre-deployment evaluations of the capabilities and risk mitigation measures of OpenAI’s o1 model and Anthropic’s Claude 3.5 Sonnet model. But the name suggested to many that these institutes were hot-beds of concern about existential risk.

After the Paris conference, the UK government renamed its AI testing agency to ensure its public perception matched its actual focus on real risks. The new name? The AI Security Institute. “This new name,” the UK government explained, “will reflect its focus on serious AI risks with security implications, such as how the technology can be used to develop chemical and biological weapons, how it can be used to carry out cyberattacks, and enable crimes such as fraud and child sexual abuse.” The announcement was accompanied by a new agreement with Anthropic, whose CEO Dario Amodei said his company would continue to “work closely with the UK AI Security Institute to research and evaluate AI capabilities…”

Whether this new name and direction mean less attention to algorithmic bias, as some have alleged, remains to be seen. It would be unfortunate if bias is downgraded along with existential risk. It is a real AI risk that has to be addressed at the model level rather than pushed down for business users to deal with. It needs to be assessed by any AI testing institute.

So, the real story from Paris is not an end to AI regulation and a heedless rush into AI innovation, but a refocus on assessing and mitigating genuine AI risks. AI testing institutions, under whatever name, are needed to ensure AI companies keep up the hard work of assessing and mitigating real, foreseeable risks of their AI models. The next thing to watch for is whether the US AI Safety Institute survives the Trump Administration’s review of the Biden Administration’s policies on AI.