The Autonomous Weapons Debate (Still) Has a Definition Problem

Michael C. Horowitz previously served as deputy assistant secretary of defense for Force Development and Emerging Capabilities. He is the director of Perry World House and Richard Perry professor at the University of Pennsylvania. Lauren Kahn previously served as an advisor for Force Development and Emerging Capabilities in the Office of the Under Secretary of Defense for Policy.

Almost every week, as interest in artificial intelligence (AI) grows, there’s another report about “killer robots,” “fully autonomous weapons,” “autonomous weapon systems (AWS),” “lethal autonomous weapon systems (LAWS),” or, simply, “autonomous weapons.” But these terms get thrown around as if there were a shared consensus on what they are. There isn’t one. Even a group of international negotiators in Geneva tasked a decade ago with defining lethal autonomous weapon systems and their potential risks and opportunities only recently came to an agreement on how to conceptually characterize (not yet define) them.

Semantic drift has produced a persistent, pernicious 180-degree problem. The terms are used interchangeably while referring to different capabilities. People may think they are referring to the same systems, but they are really envisioning anything from a cruise missile able to operate in a GPS-denied environment all the way to a Terminator-esque system that decides on its own whom to kill and when. This year’s public rupture between the Pentagon and Dario Amodei‘s AI company, Anthropic, turned in part on the difference between Amodei’s definition of “fully autonomous weapons” and the term preferred by the Pentagon, “autonomous weapon systems.” 

A central tension is that everyone wants autonomy to be a clear binary, yes or no, defined by an obvious technical characteristic of a weapon system. In reality, autonomy is a spectrum defined by the technology being used and the human choices that are made to employ a weapon system. The question isn’t, “Is it autonomous?” but rather, “How autonomous is it?” 

Autonomy reflects the degree of discretion the weapon system has in selecting and engaging targets. AI is one approach to how that autonomy functions, and the one most likely in modern systems. But there is a range of different kinds of autonomous weapon systems that have been developed over decades, and the autonomy in them is often based on older AI approaches or computer vision models. Edge deployment (running an AI system locally on a battlefield platform such as a weapon system) calls for narrow, well-bounded functionality, not the broad capability set of a general-purpose AI model like today’s large language models (LLMs). It is not yet clear how LLMs would add operational value to autonomous weapon systems at present, while significantly expanding the testing and evaluation required to prove a weapon system works effectively, a precondition to using it. The chart below explains how these fit together.

Misunderstanding around AI and weapon systems is already affecting policy decisions in Washington. Congress is wracked by AI concern in the wake of the latest safety warnings from frontier AI companies. For example, the Senate’s draft National Defense Authorization Act (NDAA) contains dozens of pages of specific, micro-level regulation on the Pentagon’s use of autonomous weapon systems and AI in general.

What few people understand is how much autonomy is already in modern military systems—and has been for decades. John Hays Hammond Jr. invented a light-chasing “electric dog” in 1916 with the promise that it would self-guide boats and torpedoes to their targets. The Norden bombsight was used in World War II, combining optics, a computer, and an autopilot for the first time to fly airplanes toward military targets. During the Vietnam War, the United States began using precision-guided bombs that steered themselves onto bridges after humans identified them as targets. In the 1980s, the Pentagon fielded the Army’s Patriot missile system and the Navy’s shipborne Close In Weapon Systems (CIWS), both of which can detect and fire at incoming threats faster than any human.

It is also important to distinguish the decision to deploy any weapon system, including an autonomous weapon system, from other stages of military operations. The first stage is the decision to start a military operation, and the second stage is the subsequent operational targeting process that occurs during a military operation. At present, some militaries use AI decision support systems (AI DSS) at that second stage, to aid commanders and other leaders in identifying and recommending potential targets. Those are not weapon systems. Humans then decide whether to proceed and pass the targets to operators who employ the weapon systems that may or may not be autonomous to varying degrees. The third stage, resulting from that targeting process, is the use of specific weapon systems against targets after the first two stages occur. For example, U.S. Central Command has used the Maven Smart System during Operation Epic Fury to aggregate disparate information streams into one dashboard. This program, according to reports, incorporates LLMs as one of several inputs to help speed up the targeting process. 

Ultimately, however, it is on the human commanders and operators, in consultation with other relevant authorities, to make appropriate legal determinations concerning the use of force before any weapon systems can be used. Senior commanders will have already designated anything that would be specifically targeted by a weapon system as a legitimate target. So even the use of an unsupervised autonomous weapon system (defined below) can only occur after humans have started a military operation and made a decision to use force through an operational targeting process. Weapon systems are specific effectors that are deployed to detect, engage, and destroy a specific target only after the decision has been made to conduct a military operation in general, and a subsequent decision on attacking a target or group of targets has also been made. The figure below delineates these different functions.

A Taxonomy of Weapon Systems

Rather than grading weapon systems by the sophistication of their autonomy or AI, it is more useful to look at the specific capabilities and functions a system performs, under what conditions, and the division of labor between human and machine (meaning any automated process) at each step. Specifically, we can classify weapon systems based on three criteria, all of which come into play after a human has determined that using force is legal and decided to use it: 

The first two criteria give us a framework of three basic types of weapon systems: unguided, where there is no machine role in guiding it after launch; semi-autonomous, where a human selects the target and a machine engages the target; and autonomous, where a machine both selects and engages the target after activation by a human. We can further classify semi-autonomous and autonomous weapons as supervised or unsupervised based on the third question, leading to five total types of weapon systems. These types mark points on the spectrum of autonomy, and how a system is used can move it forward or backward along that spectrum.

Below is an outline of each type of weapon system, constructing a framework that anyone can use to take a specific weapon system and determine where it lives on the autonomy spectrum based on its characteristics and plans for its use. These are the criteria that should determine what regulations, expectations, and standard operating procedures should govern autonomous weapon systems. Most importantly, these criteria can also help us understand how new weapon systems might differ, or not, from other existing weapon systems.

Unguided Weapon Systems

Unguided weapon systems are not directed or guided in any way after launch. Think about a bullet fired from a gun, or an arrow after it is released by an archer.

Semi-Autonomous Weapon Systems

Semi-autonomous weapon systems are launched by a human at a target or group of targets. They then engage the target through technical means that could include AI but traditionally mean capabilities such as homing in on radar signals. Semi-autonomous weapon systems themselves fall into two subcategories: those with human supervision after launch, and those with no human supervision after launch. 

Semi-autonomous weapon systems with human supervision after launch include cruise missiles like the Tomahawk missile, or the Hellfire missile popularized by the United States in the global war on terrorism. These systems offer human operators the ability to monitor the weapon in real time and divert it at the last moment if conditions change and it is deemed necessary. If semi-autonomous weapon systems lose communications with a human operator, they will generally fall out of the sky or otherwise harmlessly scuttle themselves, depending on their programming. An example of a semi-autonomous weapon system with human supervision used in current conflicts is Russia’s one-way attack drone, the Geran-4 Seeker variant.

Semi-autonomous weapon systems with no human supervision after launch have existed since the 1960s. They include most radar guided missiles and many versions of over the horizon air-to-air missiles such as the AIM-120 Advanced Medium-Range Air-to-Air Missile (AMRAAM). After being fired by an operator, a seeker in the weapon activates, causing it to look for a programmed target, like the heat signature of an aircraft or a radar on the ground. There is no human control or supervision after launch. So if the highest heat signature is a commercial airliner or a radar is on top of a school, outcomes that increase civilian harm could occur. Militaries decrease the risk of those outcomes through operational concepts that regulate the use of those systems to ensure that they are only used when they will hit a legitimate target. An example of a semi-autonomous weapon system with no human supervision after launch is Iran’s one-way attack drone, the Shahed-136.

Autonomous Weapon Systems

Autonomous weapon systems are weapon systems that, after human activation, can select and engage targets without human intervention. There are two subcategories of autonomous weapon systems as well: those with human supervision and those without human supervision. 

Autonomous weapon systems with human supervision have a human actively monitoring the weapon. The individual can turn it off or change its behavior if necessary. For example, the United States and dozens of other countries have deployed the CIWS since the early 1980s, a short range gun system that is the last line of defense for naval ships against attack. CIWS has an automatic mode whereby, if the number of targets is too large for a human operator to handle, the operator can activate an algorithm that will automatically select and engage the targets. The individual can also decide to turn off the CIWS if it is deemed necessary. CIWS has been used in the last few years by the U.S Navy to shoot down one-way attack drones launched by the Houthis.

Autonomous weapon systems without human supervision have no data link for humans to pilot or override the weapon. This is generally what is meant by the phrase “fully autonomous weapon system” in practice. 

Increasingly, electronic warfare or a lack of access to satellite data can disrupt data links between human operators and weapon systems on the battlefield. Autonomous weapon systems without human supervision offer militaries the ability to still hit targets in communication-denied environments through preprogrammed routes, computer vision algorithms, or other approaches. They differ from semi-autonomous weapon systems without human supervision because they have the ability to select targets without being aimed directly by a human. 

For example, a semi-autonomous weapon system trying to strike a tank would be fired by a human operator at a tank or group of tanks. In contrast, an autonomous weapon system without human supervision would be fired by a human operator who lacks specific targeting data but knows that there are enemy tanks in a particular geographic area. The autonomous weapon system without human supervision in this instance would likely be programmed with a simple vision algorithm designed to detect whether an object on the ground is an adversary tank or not. Upon launch, it would look for an adversary tank, and if it did not find one, it would fall out of the sky and not explode (depending on its programming). 

Note that the autonomy technology used in an autonomous weapon system without human supervision could be anything from a simple, deterministic algorithm using older technology to a probabilistic large language model, with the latter as the least likely option. Landmines represent a form of unsupervised autonomous weapon systems, though in the United States they are regulated separately rather than by the Pentagon’s policy on autonomy in weapon systems. Russia’s Molniya-2, a medium-range attack and reconnaissance drone, appears to be an example of an autonomous weapon system without human supervision that is being used in current conflicts. Ukraine’s TFL-1 module complicates things more. A human targets the weapon initially, but it then has “last mile” autonomy where it can proceed to hit the target even if a datalink is disrupted, giving it characteristics of both a supervised semi-autonomous supervised and unsupervised autonomous weapon system.

Not All Military Autonomy Is Created Equal

In the broader conversation about the role of AI in society, and in the military, there is no more constant question than about the integration of AI into weapon systems. However, the current vocabulary is a poor foundation for regulatory action in the United States or abroad. Definitions that blur the Tomahawk, the Patriot missile defense system, and Arnold Schwarzenegger’s Terminator risk creating outcomes that either will suddenly change how the United States uses weapons that have been in service for forty years or apply to almost nothing militaries actually field or plan to field. 

A “killer robots” ban, if written literally, could easily affect the future use of CIWS, which has protected U.S. ships and those of dozens of other nations since the early 1980s. But a ban drafted more narrowly to spare CIWS might not capture AI-enabled systems of concern. A requirement for a weapon system to have a datalink overseen by an operator or a human override, in contrast, might inadvertently prohibit semi-autonomous weapon systems that have been used for decades. This definitional dilemma has challenged international and domestic negotiations on autonomous weapon systems.

In the meantime, militaries everywhere are drawing critical lessons from current conflicts between Russia and Ukraine and the United States and Iran that will influence the future of warfare. It is clear that communications will be contested in any war, which is why Ukraine increasingly uses inexpensive drones with modest onboard vision software to complete attacks after jamming severs the link to the operator. More autonomous weapon systems are likely inevitable because, in a world of widespread electronic warfare, militaries need weapon systems that can function effectively despite jamming. Most of the autonomous weapon systems that worry people would also run on software far simpler than a commercial LLM, making them easier to test and validate. Put a different way, the autonomous weapon system debate and the generative AI safety and alignment debate are about different problems.

The framework laid out here provides a path forward by focusing on how autonomous a system truly is, and how it compares to existing systems. Lawmakers and the public need to understand a weapon system’s function, under what conditions it operates, and what human judgment shapes its use before launch. These are all items that governments can make policies around and verify while militaries train to ensure their commanders and operators are fulfilling their duties and that there is always human responsibility for the use of force, no matter the technologies involved.