Why the Pentagon’s ‘killer robots’ are spurring major concerns

As the Defense Department is pushing aggressively to modernize its forces using fully autonomous drones and weapons systems, critics fear the start of a new arms race that could dramatically raise the risk of mass destruction, nuclear war and civilian casualties.

The Pentagon and military tech industry are going into overdrive in a massive effort to scale out existing technology in what has been the Replicator initiative. It envisions a future force in which fully autonomous systems are deployed in flying drones, aircraft, water vessels and defense systems — connected through a computerized mainframe to synchronize and command units.

Arms control advocates fear the worst and worry existing guardrails offer insufficient checks, given the existential risks. Critics call self-operating weapons “killer robots” or “slaughterbots” because they are powered by artificial intelligence (AI) and can technically operate independently to take out targets without human help.

These types of systems have rarely been seen in action, and how they will affect combat is largely unknown, though their impact on the landscape of warfare has been compared to tanks in World War I.

But there are no international treaties governing the use of these weapons, and human rights groups are uneasy about Washington’s ethical guidelines on AI-powered systems and whether they will offer any protection against an array of humanitarian concerns.

“It’s really a Pandora’s box that we’re starting to see open, and it will be very hard to go back,” said Anna Hehir, who leads autonomous weapons system research for the advocacy organization Future of Life Institute (FLI).

“I would argue for the Pentagon to view the use of AI in military use as on par with the start of the nuclear era,” she added. “So this is a novel technology that we don’t understand. And if we view this in an arms race way, which is what the Pentagon is doing, then we can head to global catastrophe.”

Deputy Secretary of Defense Kathleen Hicks first announced Replicator during a defense conference in Washington, D.C., in late August, calling it a “game-changing” initiative that will counter China’s growing ambitions and larger fleet of military resources. It comes after years of research and testing in both the private and public sector, now reaching a point where the U.S. can move to field the technology.

While autonomous systems have been used for decades in some form, such as automatic defensive machine guns, Replicator is designed to produce swarms of AI-powered drones and flying or swimming craft to attack targets.

Hicks said Replicator will use existing funds and personnel to develop thousands of autonomous systems in the next 18-24 months. She has also publicly said that the initiative will remain within ethical guidelines for fully autonomous systems.

Those guidelines come from the Pentagon’s own directives on the use of AI in warfare, updated in January 2023, which focus on ensuring senior level commanders and officials properly review and approve new weapons. The policy stipulates there be a “appropriate level of human judgment” before an AI weapon system can use force.

A Congressional Research Service report noted that the phrase was a “flexible term” that does not apply in every situation, and that another phrase, “human judgment over the use of force,” does not mean direct human control but refers to broad decisions about deployment.

Some critics also point out that there is a waiver throughout the policy that appears to allow for the bypassing of the requirement for senior-level review of AI weapons before deployment.

Eric Pahon, a spokesperson for the Defense Department, declined to speak on the questions raised about the policy, including the language used and the waiver. The Hill sent detailed questions to the Pentagon on the concerns but did not receive a response.

Pahon said in an interview the U.S. was the “world leader in ethical AI standards.”

“We’re always going to have a person responsible for making decisions,” he said. “We’re committed to having a human responsible for decisionmaking.”

Michael Klare, secretary for the Arms Control Association’s board of directors and a senior visiting fellow researching emerging technologies, said he was “dubious that it will always be possible to retain human control over all of these devices.”

He argued these types of self-operating systems could carry out unintended missions such as attacking nuclear facilities.

“The multiplication of these kinds of autonomous devices will increase the potential for unintended or accidental escalation of conflict across the nuclear threshold [that could] trigger a nuclear war,” Klare said.

“We fear that there’s a lot of potential for that down the road,” Klare added. “And that’s not being given careful consideration by the people who are fielding these weapons.”

Fully autonomous systems have been feared for decades, with activists warning about the dire consequences of diminishing human oversight as they grow more ubiquitous.

The leading concerns are:

• Making the decision to go to war will be easier the more heavily the world relies on AI weapons
• Algorithms should not take human life because they cannot comprehend its value and can be biased, for example targeting groups based on race
• AI weapons, expected to be cheap and mass-produced, will proliferate and spread to insurgent groups and bad actors
• The risk of escalation between nuclear powers increases with the use of autonomous systems

The issues are not new, but human rights groups are alarmed about the pace of Replicator and worry its ambitious timeline will not allow for proper testing in the real world.

Hehir, from the FLI, said the timeline announced by Hicks was “very, very fast” for an emerging technology that may prove to be overwhelming. Experts in the field also say the technology is lacking when it comes to humans retaking command over out-of-control autonomous weapons.

“I don’t think that’s a sufficient amount of time,” she said. “If you’re waging war with such a high number of systems in a particular moment or a particular attack, then it’s impossible to exercise meaningful human control over such a number.”

READ MORE HERE