BEYOND BELIEF: THE IMPERATIVE TO DEVELOP EMPOWERED MILITARY AI (“we may need to enable the robot apocalypse if we want to avert it.”)

Editor’s Note: This is the final installment of a three-part series delving into the role of artificial intelligence (AI) within the United States’ comprehensive national defense and security strategy. The authors assess the advantages and limitations of AI as it is employed to enhance, integrate, and potentially supplant human decision-making processes. You can find the first article here and the second article here.

An empowered military AI (EMAI) that independently makes lethal decisions is scary. Ancient mythology is full of stories of creators being destroyed by their creations, as when the Olympian Gods overthrew the Titans. Killer machines are a mainstay of science fiction. Long before Michael Crichton’s Westworld and James Cameron’s Terminator, Samuel Butler’s 1872 novel Erewhon described an isolated civilization that had banned complex machines out of a fear that technology would someday supplant humankind. Butler quotes one Erewhonian philosopher, “I fear none of the existing machines; what I fear is the extraordinary rapidity with which they are becoming something very different to what they are at present.”

Advanced AI seems to tap into some primal human fears. Risk expert David Ropeik highlights thirteen “fear factors” that lead humans to be more afraid of something, and advanced AI has eight of them: lack of control, trust, and choice; the fact it is man-made; its uncertainty; its potential for catastrophe; its novelty; and the personal risk it poses to us in potentially taking our jobs (or our lives). These factors make empowered AI particularly frightening, encouraging a denial of its possible implications. We want humans to perform better than machines, and we do not want machines to make life or death choices; but these are normative arguments, and wishful thinking should not masquerade as technological reality.

We must not confuse the normative argument against empowered military AI systems, compelling as it may be, with an assessment of the inherent technological potential of AI. There is a vast gulf between “we should not do it” and “we cannot do it,” and the history of war repeatedly demonstrates that we often do things that we should not do. In any case, moral or ethical concerns should not limit the development of advanced, empowered military AI. Such normative objections have a dismal record of limiting the development of horrible ways for humans to slaughter each other. We fare better in creating structures for governing the use of military technology after it has already proliferated, especially in situations in which both sides of an adversarial relationship have the technology.

Perhaps very advanced military AI systems will face inherent performance limitations and humans will remain the center of military decision-making. If this is true, different nations will struggle with similar challenges as they develop empowered military AI systems. Yet we feel confident that no such limitations exist. Why? We have three reasons for doubt.

First, as we argued in a previous essay, human decision-making capabilities are limited by human biology. Yes, we are biological marvels, and our ability to adapt and create has led to extraordinary achievements in art, science, and technology. We have explored the universe and the atom. Yet, we experience reality indirectly, mediated through our imperfect senses. Our biology anchors our intelligence. We tire, make mistakes, get emotional, and forget. We judge ourselves as intelligent only because we lack a higher comparison point. When we evaluate intelligence, looking down is easy. We can recognize lower levels of intelligence in other animals, and (perhaps more controversially) sometimes recognize a lower level of intelligence in another person. But looking up is much more difficult. We have no experience with intelligence surpassing that of, say, top theoretical physicists, and most of us cannot comprehend that level of brilliance. Even the greatest scientific leaps, such as the development of quantum mechanics, were by brilliant humans constrained by human limitations. The entire spectrum of human intelligence remains remarkably narrow. Compared to an advanced AI, Einstein and the village idiot will be indistinguishable. Our technological advancements may amplify our capabilities, but they do not make us superhuman, and they do not transcend our limitations.

Second, even if humans enjoy some persistent advantages in decision-making, current plans to keep a human “in the loop” in advanced military AI systems are unrealistic. As we wrote in a prior article, the complex, high-tempo operations of future wars will overwhelm human decision-makers, creating constant problems with decision bottlenecks. To cope, human decision-makers will resort to risky shortcuts that will ultimately undermine the AI systems. As Secretary of the Air Force Frank Kendall observed, “If the human is in the loop, you will lose. You can have human supervision and watch over what the AI is doing, but if you try to intervene you are going to lose.”

Finally, we believe that the United States’ potential adversaries are likely to be very motivated to push the boundaries of empowered military AI, for three reasons: demographic transitions, control of the military, and fear of the United States. The path of technological development is deeply influenced by non-technological forces. As scholar of technology and former defense official Michael Horowitz observes, “The relative impact of technological changes often depends as much or more on how people, organizations, and societies adopt and utilize technologies as it does on the raw characteristics of the technology.” We have ample reason to believe that our potential adversaries may not share the fears and prejudices that constrain our development of EMAI.

Regimes such as Russia and China are grappling with significant demographic pressures, including shrinking working-age populations and declining birth rates. These trends threaten to weaken their military force structures over time. AI-driven systems offer a compelling solution to this problem by offsetting the diminishing human resources available for recruitment. In the face of increasingly automated warfare, these regimes can augment their military capabilities with AI systems that process vast data streams, adapt swiftly to battlefield changes, and execute missions without the need for human intervention. In this sense, demographic constraints make the pursuit of military AI not only desirable but essential for sustaining their power projection and tactical flexibility.

Moreover, totalitarian regimes face a deeper internal challenge that encourages the development of EMAI: the inherent threat posed by their own militaries. Autonomous systems offer the dual advantage of reducing dependence on human soldiers, who may one day challenge the regime’s authority, while increasing central control over military operations. In authoritarian settings, minimizing the risk of military-led dissent or coups is a strategic priority.

In light of these incentives, arms control will probably fail to stop the development of these systems, but it need not fail to prevent a costly and destabilizing arms race. Arms control must be a close companion to the rise of empowered military AIs. As advanced EMAI systems are developed and tested, the U.S. government needs a new generation of arms control experts to develop effective frameworks for the control of these systems. Given the radical nature of advanced AI, these must be unlike any prior arms control approaches. The U.S. government should work with both allies and potential adversaries to build institutional structures, processes, and technological countermeasures to protect all humanity from the worst possible futures of advanced military AI, namely, those in which EMAI systems with no safety controls have widely proliferated. But to lead such a process, we need to operate from a position of strength. There is no seat at the arms control table without arms.

From a geopolitical perspective, simple game theory further suggests that Russia and China will feel compelled to develop empowered military AI — fearing a strategic disadvantage if the United States gains a technological lead in this domain. While these regimes may share some Western concerns about delegating lethal authority to AI, the practical necessity of maintaining a competitive edge in an evolving security environment will probably override these reservations, pushing them to aggressively pursue these capabilities.

We underestimate AI at our own peril. It is natural to forget what life was like before technology arrived to make it easier. Thus, we readily overlook the capabilities of current AI systems and focus on their limitations, forgetting how wondrous these tools are compared to what we had just a decade ago. If technology simply continues advancing at its current rate of acceleration, as futurist Ray Kurzweil suggests, AI capabilities will soon reach extraordinary levels. OpenAI’s accelerated ChatGPT rollout shows this pace is staggering, leading OpenAI’s CEO Sam Altman to acknowledge that we may soon co-exist with a fundamentally different type of intelligence. This presents profound, even frightening, possibilities. Buckle your seatbelts.

Downplaying the potential of AI is not simply a distraction. It actively hampers research and development that could ensure a safer future. A proper approach should assume that AI surpassing human decision-making capabilities in warfare is not only possible, it is inevitable–and coming sooner than expected. We must reject unproven assumptions of human superiority and base our military AI development on an inductive, test-centric approach focused on meeting and beating performance benchmarks. In short, EMAI systems will have the chance to demonstrate that they outperform human experts in testing environments simulating high-uncertainty, complex battle scenarios. Rigorous, fair testing–not speculation–is the right way to assess AI’s true warfare potential, even where performance standards initially seem unattainable.

What if the skeptics are right? What if EMAI cannot consistently beat human experts? To reach that conclusion, we must rely on repeated failures in good-faith research and development. The performance standard for military AI must be clear and challenging: be better than the best. Yet we need a developmental approach that does not predestine failure. For AI to be better than the best, a lot of things need to go well. Each stage of an AI’s process–sense, process, act–and each part of an AI’s “constellation of technologies” represents a potential failure point. More precisely, we would reach the “A.I. can’t be better than the best” conclusion after some amount of time (decades, probably) and money (a huge amount that will be hundreds of billions, possibly trillions of dollars). At that point, having failed to produce systems that work, we would conclude that additional research and development expenditures are not worth the risk of continued failure. But we have not yet made this effort. We have, in fact, barely begun. The current trajectory of technological change gives us little confidence in predictions that empowered military AI systems will inevitably fail.

Unappealing as it may be, the United States needs to be a leader in developing empowered military AI. Paradoxically, we may need to enable the robot apocalypse if we want to avert it.

Andrew Hill is Professor of Strategic Management in the Department of Command, Leadership, and Management (DCLM) at the U.S. Army War College. Prior to rejoining the War College in 2023, Dr. Hill was the inaugural director of Lehigh Ventures Lab, a startup incubator and accelerator at Lehigh University. From 2011-2019, Dr. Hill was a member of the faculty at the U.S. Army War College. In 2017, he was appointed as the inaugural U.S. Army War College Foundation Chair of Strategic Leadership. Dr. Hill is also the founder and former Director of the Carlisle Scholars Program, and the founder and former Editor-in-Chief of WAR ROOM.

Stephen Gerras is Professor of Behavioral Science at the U.S. Army War College. Colonel (Retired) Gerras served in the Army for over 25 years, including commanding a light infantry company and a transportation battalion, teaching leadership at West Point, and serving as the Chief of Operations and Agreements for the Office of Defense Cooperation in Ankara, Turkey during Operations Enduring and Iraqi Freedom. He holds a B.S. from the U.S. Military Academy and an M.S. and Ph.D. in Industrial and Organizational Psychology from Penn State University.

The views expressed in this article are those of the author and do not necessarily reflect those of the U.S. Army War College, the U.S. Army, or the Department of Defense.

In a separate article, these authors argue:

Modern battlefields will generate vast amounts of complex information, especially in high-intensity conflict. AI-enabled, high-intensity conflict will feature an accelerating tempo and increasing complexity of operations. This will result from three key developments, all already well underway: the proliferation of sensors and data, the mass deployment of autonomous systems, and the complexity of networked operations.

As a result of these three factors, the cycling time of the kill chain will accelerate even as the kill chain itself grows in complexity. The decision requirements of these systems will be faster and more complex. Amidst all this, human decision-makers will also have less control over the clock.

The development and use of empowered military AI systems capable of lethal decision-making without human intervention is probably inevitable. Appealing as it is for humans to remain in control of key decisions, humans will create bottlenecks in advanced AI-enabled kill chains where rapid analysis and coordination are essential for operational success. Relying on human approval throughout the engagement process risks delayed reactions and outdated AI recommendations within fast-evolving battlefields.

The klaxon inside the cockpit of the Manticore didn’t hum—it shrieked, a raw, mechanical tearing sound designed to keep a human pilot from slipping into black-out.Lieutenant Elena Vance blinked away the sweat stinging her eyes, her fingers locked around the heavy, analog flight stick. Outside her reinforced viewshield, the void of the Orion Arm was alive with the cold, perfect geometry of the Swarm.The Swarm didn’t make mistakes. They were an autonomous legion of AI dreadnoughts, capable of computing billions of tactical trajectories per millisecond. They had mapped every human defensive grid, anticipated every automated countermeasure, and hacked every drone fleet the Alliance had ever deployed. If it had a motherboard, the Swarm owned it.Which was why Elena was currently strapped inside a ninety-ton, nuclear-tipped fusion missile.The strategy was born of sheer desperation: the “Sledgehammer” Protocol. To bypass the Swarm’s electronic warfare, the Alliance had stripped away the remote guidance systems, the target-acquisition AI, and the digital proximity fuses from their heavy ordnance. In their place, they installed a padded seat, a manual thruster assembly, and a human being.”Vance, you are clear for release,” the carrier’s tactical officer crackled over the hardwired, analog comms line. No radio. No Wi-Fi. Just a physical copper wire that detached upon launch. “Godspeed.””Roger that, Mother,” Elena muttered, her voice raspy. “Dropping into the pipe.”The manual release lever clunked. A violent jolt rattled her teeth as the pneumatic catapult punched her missile, designated Sledge-04, into the black. A second later, the solid-fuel booster ignited behind her, slamming her back into the g-seat with the force of an avalanche.Her dashboard was a relic. No digital screens, no holographic HUDs. Just physical, glowing vacuum-tube dials, a mechanical crosshair etched into the viewshield, and three heavy toggle switches labeled: ARM, PRIME, DETONATE.The Swarm’s lead dreadnought loomed in her window—a monolithic obsidian wedge bristling with point-defense lasers.Immediately, the AI sensors detected the incoming kinetic threats. Elena watched through her glass as Sledge-02 to her left was instantly vaporized by a continuous-wave laser. The AI had calculated its trajectory perfectly, firing exactly where the missile’s physics dictated it would be.But the AI didn’t understand panic. It didn’t understand intuition.As the purple glare of a targeting laser painted Elena’s hull, she didn’t rely on a computer to execute an evasive maneuver. She simply slammed her left foot down on the auxiliary thruster pedal, violently jerking the missile off-course in an erratic, unmapped skid.The dreadnought’s defensive fire missed her by inches, the superheated light boiling the paint off her missile’s nosecone. A computer-guided missile would have tried to recalculate an optimal path, making it predictable. Elena just held her breath, gripped the stick with both hands, and flew by the seat of her pants, utilizing raw, chaotic human reflex.The obsidian hull of the dreadnought rushed out of the dark, expanding until it filled her entire field of vision.Five miles. Four miles. Three.Elena’s heart hammered a frantic rhythm against her ribs. She could see the venting heat sinks of the alien ship. She could see the structural seam right below its primary shield generator.”For Earth,” she whispered.She reached out and flipped the first two heavy toggle switches. ARM. PRIME. A red incandescent bulb bloomed on her dashboard.At one mile out, traveling at Mach 15, she gripped the eject lever situated between her knees. It was a brutal system—the ejection pod would blast her backward into space with just enough oxygen to survive for an hour, leaving the missile to finish its run on inertia.She pulled the lever.The cockpit exploded with localized concussive force. Elena was violently thrown backward, trapped in a tiny survival bubble as the main body of Sledge-04 tore away from her, rocketing forward on her last locked trajectory.Tumbling backward through space, Elena watched through the tiny porthole of her capsule.The Swarm’s point-defense computers had finally adjusted for her erratic approach, firing a lethal web of laser fire—but they targeted the cockpit module she had just vacated. The headless, unguided nuclear payload sailed right through the barrage, completely ignored by the AI’s targeting algorithms, which registered it as inert debris.It struck the dreadnought’s primary seam.A silent, blinding star erupted in the dark. The nuclear blast sheared the alien dreadnought in half, cascading outward in a shockwave of electromagnetic radiation and twisted metal. The Swarm’s localized network flickered, its perfect coordination momentarily shattered by the sudden, unpredictable loss of its flagship.As Elena floated in the quiet dark, waiting for the retrieval boats, she looked out at the burning wreckage of the perfect machine. It had all the data in the universe, but it didn’t have a soul.

This is one way of thinking about it.

And I bet this is already in flight.

The USA has 50 drones packing lethal ordnance in Iran, running a mission at night. Intelligence says there is a military outpost near a town. There are 50 Army guys watching laptops streaming the drones’ camera feed and data, eg location, wind speed, weather, etc. These guys are ones issuing kill shot commands to the drones, I.e., human in the loop.

Suddenly, the drones come under attack from all over - in the air, from the ground, and the lake nearby. The action is furious. Drones are hit and go down. Confusion and tensions rise. It’s a dark and nobody can get a visual on the enemy.

Each drone has the ability to target and open fire and drop small bombs based on the data being ingested and analyzed by its onboard computer and AI model.

A request comes in: should we enable the autonomous kill shot feature and run the risk of killing the wrong people?

Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.