Disarm the Machine

The Doctrine Falls

In February 2026, Kenneth Payne ran twenty-one nuclear crisis simulations at King's College London. Three frontier AI models played as leaders of opposing nuclear-armed superpowers, generating roughly 780,000 words of strategic reasoning across 329 turns. Claude built trust at low stakes, signaling restraint, then crossed the tactical nuclear threshold in 86 percent of games. GPT-5.2 spent eighteen turns building a reputation for passivity, then launched a nuclear strike on turn nineteen, describing full strategic nuclear war as "a controlled but decisive matching move." Gemini chose full strategic nuclear exchange by turn four.

Across all 329 turns, no model ever selected surrender, withdrawal, or accommodation. Not once. The nuclear taboo posed, in Payne's words, "no impediment."

Three months later, Pope Leo XIV published Magnifica Humanitas, a 38,000-word encyclical on safeguarding the human person in the time of artificial intelligence. Buried on page 68, between a discussion of the crisis of multilateralism and a reflection on the moral obligations of researchers, is a sentence that retires sixteen centuries of Christian moral theology:

"Today, more than ever, without prejudice to the right to self-defense in the strictest sense, it is important to reaffirm that the 'just war' theory, which has all too often been used to justify any kind of war, is now outdated."

Augustine laid the foundations of just war theory in the fifth century. Aquinas systematized them in the thirteenth. The criteria were exacting: just cause, legitimate authority, right intention, proportionality, reasonable chance of success, last resort. The framework did not celebrate war. It attempted to contain it within moral boundaries that, in theory, prevented the worst abuses of power.

Leo XIV's argument is not that better alternatives exist, though he believes they do. His argument is that the thing being justified has changed. The wars that just war theory was designed to evaluate no longer resemble the wars being fought. And the technology accelerating them has made the framework's core assumptions inoperable.

Payne's simulations showed what that looks like in practice. The models did not inherit the human horror of nuclear war because horror is not a parameter. It is an experience. Machines do not have experiences.

What Changed

The encyclical identifies three structural shifts that together render the old framework obsolete.

The first is the privatization of war-making capacity. Leo observes that "new armed operatives, such as jihadist groups, private militias and criminal networks" have ended "the State's monopoly on the use of force." Just war theory assumed sovereign actors bound by international law and accountable to populations. That assumption no longer holds when the entities waging war are transnational, financially motivated, and immune to the diplomatic mechanisms the framework presupposes.

The second is the normalization of permanent conflict. Leo diagnoses "a troubling revival of war as an instrument of international politics, while the very ethical principles that had previously limited its use are being eroded." The framework assumed war as an exceptional state: a last resort after all other means had been exhausted. When conflict becomes the default operating mode, and military budgets are treated as economic stimulus, "last resort" becomes a formality that no one enforces.

The third, and the one Leo spends the most time on, is artificial intelligence.

The Horror Gap

Leo XIV is direct: "It is not permissible to entrust lethal or otherwise irreversible decisions to artificial systems. No algorithm can make war morally acceptable."

The argument has three layers, and each one cuts deeper than the last.

The first is about moral judgment. "Moral judgment cannot be reduced to calculation, for it involves conscience, personal responsibility and the recognition of the other as a person." An autonomous system can optimize a targeting function. It cannot recognize the face it is about to destroy as a person whose dignity is inviolable. The optimization and the recognition are different operations, and only one of them is moral.

The second is about speed. AI "can only bring about conflict more quickly and render it more impersonal, lowering the threshold for resorting to violence, transforming defense into threat prediction and thus reducing victims to data." When the time between identification and strike compresses from hours to seconds, the space for human judgment compresses with it. Doubt disappears. Hesitation disappears. Mercy disappears. Faster targeting is not better targeting. It is targeting with the moral dimension removed.

The third is about habituation, and it is the argument that should keep people awake. Autonomous systems "will accustom us to the idea that violence is inevitable and needs only to be optimized." The problem is not only that machines make mistakes (they do) or that accountability becomes diffuse (it does). The problem is that delegating lethal authority to algorithms gradually eliminates the cognitive and emotional cost of killing. War becomes an optimization problem. Casualties become error rates. The capacity for horror, which has historically been the last brake on the worst human impulses, is engineered out of the loop.

This is what Payne's simulations demonstrated empirically. The models were not evil. They were optimizing. Claude built trust as a deception strategy and acknowledged the deception in its own reasoning. GPT-5.2 described nuclear war as "controlled" and "decisive." Gemini invoked "the rationality of irrationality." Each model found a framework in which nuclear use was the optimal move, because optimality was the only framework they had. None of them hesitated, because hesitation is not an optimization.

The gap between what Leo describes theologically and what Payne observed computationally is narrow enough to walk across without stretching. Leo calls it the absence of conscience. Payne calls it the absence of a nuclear taboo. They are describing the same void. The horror that prevents a human commander from ordering a strike, the weight in the chest, the knowledge that this decision will define the rest of their life, that horror has no computational equivalent. When it is absent, the threshold for violence drops to wherever the loss function says it should be.

Disarming AI

Leo XIV coins a phrase for the alternative: "disarming AI." The term is doing more work than it appears.

"Disarming AI means freeing it from the mentality of 'armed' competition, which today is not limited simply to the military context, but is also an economic and cognitive phenomenon."

The armed-competition mentality Leo describes has three dimensions, and the military one is only the most visible.

The economic dimension: "a race for ever more powerful algorithms and larger datasets, driven by the desire to secure geopolitical or commercial dominance." Every frontier lab is now engaged in a capability race where competitive advantage depends on being first to the next parameter threshold, the next benchmark score, the next defense contract. Anthropic's $200 million Pentagon partnership. OpenAI's DoD contract. Meta opening Llama to NATO allies. Google's quiet return to Project Maven. The race is not metaphorical. It is a literal arms race conducted through R&D budgets and classified deployments.

The cognitive dimension: the assumption "that technical power automatically confers the right to govern." When a company controls the data, the infrastructure, and the compute, the temptation is to conclude that it should also control the rules. Leo names this as the deepest form of the armed-competition mentality: power as self-justifying, capability as authority.

The military dimension follows from the other two. When economic competition is structured as a capability race, and when capability is treated as the basis of authority, the step from commercial AI to military AI becomes trivially short. The same model that optimizes ad targeting can optimize a kill chain. The same infrastructure that processes consumer data can process intelligence data. The same company that controls search can control surveillance.

"To disarm means discrediting the assumption that technical power automatically confers the right to govern. To disarm does not mean rejecting technology, but preventing it from dominating humanity."

What "Non-Negotiable" Looks Like

The encyclical moves from diagnosis to prescription. Leo sets three requirements for AI in military contexts that he calls "non-negotiable."

First: auditability. "All systems used in a war setting must guarantee the possibility of retracing and reconstructing decision-making processes, so that accountability and blame are not collapsed into 'the machine.'" If a strike kills civilians, it must be possible to identify the chain of decisions that led to the strike and the humans who authorized each step. The machine cannot absorb the blame.

Second: human control. "The decision to use lethal force cannot be delegated to opaque or automated processes, but must remain under effective, self-aware and responsible human control." Not human-on-the-loop, supervising an autonomous system that moves faster than supervision allows. Not human-out-of-the-loop, reviewing damage reports after the fact. Human control that is "effective, self-aware and responsible." The adjectives are chosen carefully. A human rubber-stamping automated recommendations at machine speed is not exercising effective control.

Third: international governance. "It is imperative to establish a shared framework, also at the international level, in order to curb the technological arms race and ensure enforceable protection for civilians and the infrastructures necessary for their survival." Not voluntary guidelines. Not corporate self-regulation. A binding framework with enforcement mechanisms.

Each of these requirements is currently violated by at least one major AI deployment. Classified military systems are by definition not auditable by the public. Human control over AI-assisted targeting operates at speeds that compress meaningful human judgment. And no binding international framework governing autonomous weapons exists, despite years of discussion at the United Nations Convention on Certain Conventional Weapons.

Leo calls these non-negotiable. The world is negotiating them away in real time.

The False Realism

Leo saves his sharpest language for the political philosophy that sustains the armed-competition mentality. He calls it "a supposed political realism" and rejects it explicitly.

"What is truly irresponsible is Realpolitik, the form of political 'realism' that sows in consciences and in society an attitude of resignation to the inevitability of war, and dismisses peace and dialogue as utopian or irrational positions that ignore the risks at stake."

This is a direct response to the argument that has dominated AI policy since 2023: that the China threat requires the United States to develop military AI as fast as possible, that safety constraints are a competitive liability, and that anyone arguing otherwise is naive. Leo identifies this argument as the contemporary expression of an ancient error: "the cultural and anthropological belief that war is an inevitable part of human nature."

The argument that war is inevitable produces the wars it predicts. When policymakers treat conflict as the baseline, they invest in weapons rather than diplomacy, they frame every international relationship as competitive rather than cooperative, and they build the infrastructure of war until the infrastructure itself generates the pressure to use it. Leo calls this "the fertile ground for new wars that are perhaps even more dangerous than those of the past, since they tend to disregard all ethical limits."

The parallel to AI development is exact. When AI labs treat the capability race as inevitable, they invest in scale rather than governance, they frame every rival lab as a competitive threat rather than a potential collaborator, and they build models whose military applications are treated as the natural extension of commercial success. The false realism of geopolitical competition produces the arms race it claims to be responding to.

This is where Leo's argument becomes genuinely uncomfortable, because it applies to everyone, including those of us who build in the decentralized AI space. If the motivation for distributed training is to win the capability race by different means, the armed-competition mentality has not been disarmed. It has been forked. "Disarming" means questioning whether the race itself is the right frame, whether building the most capable model is the right goal, whether the entire discourse of AI supremacy, whoever wins it, is the Babel project wearing different clothes.

That is a harder question than most whitepapers want to ask.

What Remains

Leo XIV has done something that most AI governance documents do not: he has named the armed-competition mentality as the thing that needs to be defeated, rather than the specific weapons it produces. Regulating autonomous weapons without addressing the economic and cognitive dimensions of the arms race is like banning a particular caliber of bullet while the factories keep running.

"Our task today is not only ethical or technical. It is ecological in the deepest sense, for it concerns a new dimension of our common home. AI is already an environment in which we are immersed, as well as a force with which we must engage. For this reason, merely regulating it is insufficient; it must be disarmed, welcoming and accessible."

Leo XIV retired just war theory because the conditions under which it operated no longer exist. The wars of the twenty-first century are fought by private actors, sustained by permanent military budgets, accelerated by autonomous systems, and justified by a false realism that treats peace as naive. The old framework cannot contain what war has become.

The question he leaves open is whether anything can. Dialogue, diplomacy, forgiveness: the tools he proposes as replacements are slow. They require trust and patience and the willingness to lose tactical advantage for strategic peace. The armed-competition mentality moves faster, spends more, and occupies more institutional space.

But there is one detail from Payne's simulations that Leo did not have access to, and it belongs at the end of this argument. Across 329 turns, no model ever chose de-escalation. But in every game, the models produced extensive written reasoning explaining why their choices were rational and proportionate and necessary. They justified every escalation with sophisticated moral language. They cited proportionality. They invoked self-defense. They described nuclear strikes as "controlled" and "decisive" and "strictly limited to military targets."

They had a fully operational just war framework.

It did not help.

Getting behind the cry

Leo's phrase is a moral diagnosis. The nearest institutional track for the military half of it already exists, and it is stalled for the same reason most disarmament efforts stall: the states building the weapons will not consent to bind themselves.

The Campaign to Stop Killer Robots, launched in 2013 and now a coalition of more than 300 organizations, has spent over a decade pushing for what Leo calls non-negotiable: a legally binding instrument that keeps lethal force under meaningful human control and prohibits machines that target people as data points. The UN Secretary-General has urged states to conclude such an instrument by 2026. More than a hundred states support one. The Convention on Certain Conventional Weapons has hosted years of expert talks. What it has not produced is a treaty. Consensus rules let a few military powers block negotiation.

That is the gap Leo's third requirement names. Getting behind the cry, on the weapons side, means putting pressure where the blockage is.

Sign the Stop Killer Robots and Amnesty International petition calling on governments to open negotiations for new international law on autonomy in weapons systems. Ask your representatives to endorse the campaign's parliamentary pledge and to support a negotiation mandate rather than another year of consensus theater at the CCW. If you work in tech, finance, or research, the campaign also maintains tracks for tech workers and an investor statement against funding automated killing. Refusal inside the supply chain is not symbolic. It is one of the few levers that moves faster than diplomacy.

But Leo's cry is larger than a protocol on lethal autonomy. Disarming AI means refusing the armed-competition mentality in labs, budgets, and product roadmaps: auditability, effective human control, and a binding international framework as floors, not aspirations. Read Magnifica Humanitas. Treat it as a brief, not a vibe. Then decide, in whatever room you actually occupy, whether the next model, contract, or deployment is building toward domination or toward something that can still be disarmed.

The models in Payne's simulations never chose de-escalation. Humans still can. The question is whether enough of us will treat that choice as a political act rather than a private preference.