Why Fear Matters in Governing Military AI

In May, officials from the United States and China returned to the negotiating table to discuss artificial intelligence—an issue as consequential this century as nuclear weapons were during the Cold War. But AI poses a fundamentally different challenge. Its capabilities are difficult to measure, evolve at extraordinary speed, and are increasingly intertwined with economic competitiveness, military power, and the strategic interests of a handful of technology companies. The emergence of two AI superpowers facing shared risks without shared rules presents one of the most urgent governance challenges of the twenty-first century.

Among these risks, the growing use of autonomous AI systems in military decision-making deserves particular attention. Unlike traditional weapons, such systems may operate at speeds and levels of complexity that exceed human understanding and effective oversight. As machines take on greater roles in surveillance, targeting, and battlefield operations, the risk of errors, accidents, and unintended escalation increases. The prospect that critical decisions about the use of force could be delegated to algorithms raises profound questions about accountability, human control, and international security. In short, extreme military AI risks include high-consequence outcomes arising from autonomous systems operating beyond effective human oversight.

History suggests that governance often emerges in response to perceived extreme risk. Nuclear arms control, environmental regulation, and public health institutions gained support only as societies recognized the scale of the threats involved. Fear of nuclear catastrophe helped mobilize public backing for arms-control agreements, while international pressure from governments, scientists, and public health organizations led to the 1972 Biological Weapons Convention (BWC), which prohibits the development, production, acquisition, transfer, stockpiling, and use of biological and toxin weapons.These examples show how scientific concern and international cooperation can establish global norms against high-risk technologies.

Similar concerns about autonomous military AI may create the political conditions for new forms of oversight and international cooperation. These concerns were on display last week, when United Nations Secretary General António Guterres called for a ban on “killer robots.” Without a shared understanding of the risks, efforts to build safeguards may lack the urgency needed to keep pace with technological change. We argue that philosophy can help clarify the ethical foundations needed to govern military AI before its risks become irreversible.

When rivals start talking

Both Washington and Beijing recognize the risks of advanced AI, yet neither is prepared to slow its pursuit of technological leadership. Both worry that restraint would confer a strategic advantage to the other. But despite what some argue, AI is not a race with a finish line. Framing it as one may intensify tensions and increase the very risks both sides seek to avoid. As the nuclear era showed, rivalry does not eliminate the need for cooperation to prevent catastrophe. The race for advanced AI leadership is intensifying. Mirroring the Cold War, mastery of this technology and being the first to achieve “superintelligence” is now seen as a matter of survival and deterrence. However, growing recognition shows this rivalry could spiral out of control. We urgently need risk-reduction mechanisms to prevent accidents, cyberattacks, and autonomous system failures.

Perhaps the most revealing fact is this: when rival powers begin discussing crisis-management protocols for AI, the issue has clearly moved beyond the realm of science fiction. The possibility that advanced systems could be used in extreme geopolitical conflicts has become part of the real calculus of international security. Yet how can we ensure that errors do not occur—or that they are not transmitted and amplified through systems that remain only partially understood even by their creators? How can we guarantee the responsible use of technologies whose inner workings we do not fully comprehend? Is it possible to control the use of AI on the battlefield at a safe level?

Joseph Weizenbaum, the MIT computer scientist who created ELIZA in 1966, was among the first critics of AI and its military applications. For him, AI was too important to be left solely in the hands of computer scientists and required the participation of philosophers, social scientists, and scholars from other disciplines. Weizenbaum warned that AI was a dual-use technology and that civilian advances would inevitably be adapted for military purposes. He feared that systems designed to maximize efficiency would weaken human judgment, dilute responsibility, and lower the barriers to the use of AI technology.

Recent conflicts have confirmed many of his concerns. The wars in Ukraine and Gaza illustrate AI's growing role in targeting and battlefield automation. These examples illustrate how AI is contributing to the automation of warfare while raising profound questions about responsibility and human control in lethal operations. Despite repeated warnings from the UN, there is still no global control on autonomous weapons, suggesting that the military AI arms race envisioned by Weizenbaum decades ago is already underway.

For the French philosopher Gilbert Simondon, every technical object functions as an amplification of human action in the world. When we speak of technologies associated with warfare, we are ultimately speaking about the expansion of humanity’s capacity for destruction and deterrence. The issue today is that AI increases the psychological and moral distance between those who exercise destructive power and those who bear its consequences. If war, as Prussian military theorist Carl von Clausewitz argued, unfolds within a fog that obscures judgment, AI may intensify that fog by automating how hostile activity is identified and classified. In this sense, it automates not only tasks but part of human cognition itself.

If politics is profoundly shaped by technology, thinkers such as Andrew Feenberg and Langdon Winner have long argued that technology is always embedded with political theory. It is worth remembering that Henry Kissinger first gained prominence in US policy circles after publishing his influential study on nuclear weapons and international politics in 1957. There is a clear parallel between the technological transformations of the twentieth century and the political challenges of the twenty-first. The use of AI on the battlefield highlights one of the central issues in AI regulation: the governance of catastrophic risks.

The case for intervention

This is not about the fantasy of machines deciding to exterminate humanity. Rather, it concerns extreme global harms resulting from the human use of advanced AI systems. The core issue is the cognitive transformation that these technologies may induce in human decision-makers under extreme conditions such as armed conflict. Current AI models can already dramatically enhance the ability of malicious actors to conduct cyberattacks against hospitals, banks, power grids, and other critical infrastructure. Threats associated with autonomous AI agents and the development of biological weapons are no longer merely hypothetical.

There are therefore clear grounds for state intervention. When technologies acquire strategic and military significance, their regulation ceases to be merely an economic issue and becomes a matter of global security. The challenge now is to build democratic regulatory institutions capable of involving society in the governance of advanced AI before the logic of technological competition makes such participation impossible. This will require drawing on philosophy reflection—not only to better understand the nature of AI and its relationship with individuals, but also to reflect on how democratic societies decide which risks they are willing to accept.

In 1979, with Leonid Brezhnev leading the Soviet Union and the United States still shaped by the strategic legacy of Nixon and Kissinger, the Soviet bloc invaded Afghanistan. That same year, the German philosopher Hans Jonas published The Imperative of Responsibility: In Search of an Ethics for the Technological Age. Jonas argued that modern technology had become a threat to the very continuity of human life and that humanity’s only reliable compass was the anticipation of danger. In an era increasingly defined by powerful technologies, foresight about risks becomes a condition for preserving humanity. Fear, far from being merely a negative emotion, serves the essential function of warning us and steering us away from potentially fatal threats.

Before control slips away

By imagining the worst consequences of our choices, we become better able to recognize what is at stake and what must be protected. Jonas argued that in a technologically transformed world it is easier to protect humanity by identifying the evils we must avoid than by identifying the goods we hope to achieve. This insight is particularly relevant to military AI and autonomous weapons. The fear that AI systems may eventually select and attack targets without meaningful human control should not be dismissed as alarmism. On the contrary, awareness of these risks can mobilize society and stimulate democratic debate about limits, responsibilities, and governance mechanisms.

Yet security risks are only the beginning. The greater danger is that autonomy in warfare may reduce human oversight, making conflicts faster, more automated, and more destructive. For this reason, military AI regulation cannot be left solely to governments, armed forces, or technology companies. It must involve civil society and establish rules to limit the extreme risks of autonomous weapons before they exceed our capacity for control.