The Navy is eager to keep the secret sauce under the lid, but the scope of the problem, the modeling challenges and mathematical solutions, can be gleaned in this recent paper titled Model-P redictive A sset G uarding by T eam of A utonomous S urface V ehicles I n E nvironment W ith C ivilian B oat . The research isn't directly related to the Navy experiment, but there's a lot of overlap. “The outlined problem can be decomposed into multiple components, e.g., accelerated simulation, trajectory planning for collision- free guidance, learning of interception behaviors, and multi-agent task allocation and planning," the researchers write.

The Navy's breakthrough marks the clearest indication yet that more missions are falling to increasingly automated—and weaponized—systems, with human presence retreating ever deeper into the background. It's a trend that continues to alarm both AI experts and human rights watchers.

’Don’t’ Make Them… Autonomous’

Last May, British artificial intelligence researcher Noel Sharkey of the University of Sheffield , told Defense One that, in his view, armed UAVs were proliferating far too quickly and that the last red line to be crossed was autonomy. “Don’t go to the next step. Don’t make them fully autonomous. That will proliferate just as quickly and then you are really going to be sunk,” he said.

Sharkey is not alone in that concern. Political scientist Matthew Bolton of Pace University New York City’s Dyson College offered a similar opinion. “Growing autonomy in weapons poses a grave threat to humanitarian and human rights law, as well as international peace and security… In modern combat it is often heartbreakingly difficult to tell the difference between a fighter and a non-combatant. Such a task relies on a soldier’s wisdom, discretion and judgment; it cannot and should not be outsourced to a machine. Death by algorithm represents a violation of a person’s inherent right to life, dignity and due process.”

Bolton points to the international bans on landmines as an indicator of where the debate over autonomous weapons systems is headed. “When the vast majority of countries outlawed anti-personnel landmines -- a goal now endorsed by President Obama -- they established that weapons which maim or kill absent of direct human control are morally reprehensible.”

The security of these systems is also of critical importance because hackers, criminals, or enemies who take control of autonomous attack systems could wreak enormous havoc. computer scientist and entrepreneur Steven Omohundro

Other AI experts take a more nuanced view. Building more autonomy into weaponized robotics can be dangerous, according to computer scientist and entrepreneur Steven Omohundro. But the dangers can be mitigated through proper design.

“There is a competition to develop systems which are faster, smarter and more unpredictable than an adversary's. As this puts pressure toward more autonomous decision-making, it will be critical to ensure that these systems behave in alignment with our ethical principles. The security of these systems is also of critical importance because hackers, criminals, or enemies who take control of autonomous attack systems could wreak enormous havoc,” said Omohundro.

Klunder said they’ve built three fail safes into the system. In the event that one of the boats loses contact, it goes dead in the water.

The aspect of the program that Klunder seems most proud of is how much money it could save. Not only is the CARACaS unit is made of cheap, off-the-shelf parts, it can be fitted to a variety of the Navy’s rigid inflatable or RIB boats so no pricy new frames necessary. The brains receive input from regular radar (360 degrees) and conventional electro-optical infrared or EO/IR sensors, which are hardly exotic.

The biggest cost to the program was developing the algorithms. The possibility exists for much larger cost savings by reducing multi-person missions down to single operator tasks. Providing safe passage through places like the Strait of Hormuz just got a lot cheaper.

In a recent report preview from the Center for New American Security, Paul Scharre and James Marshall described the transition to low-cost, more autonomous robotic systems as the force multiplier of the future. “Low-cost uninhabited systems offer a way to bring mass back to the fight. With no human onboard, they can take greater risk. Survivability can be balanced against cost, with swarm resiliency taking the place of platform survivability. Swarms of low-cost uninhabited systems can be used to saturate and overwhelm enemy defenses. The robotics revolution will enable new ways of bringing mass back on the battlefield.”

But recent Defense Department budget decisions actually reflect a waning enthusiasm for unmanned systems, as Alex Velez-Green notes in a provocative piece for the Harvard Political Review, in which casts funding for AI development as hampered by sunk cost projects such as the F-35. “ Unfortunately, the Department of Defense’s current investment outlook does not show an appreciation for the role that swarm robotics will play in the future of warfare. Today, we are investing more than $35 billion in the Littoral Combat Ship program, and expect to spend more than $25 billion in the next several years to make the new, manned Long-Range Strike Bomber deployable by the mid-2020s. Such manned systems will be necessary complements to unmanned systems for the foreseeable future. However, their development cannot come at the expense of the robotic technology that will actually disrupt combat, which is exactly what is happening today,” he writes.

The debate about the ethics of increasingly smart—and ever-more heavily armed--military robots will continue as the technology advances and the systems proliferate around the world . The Beltway battle between those who think the Defense Department is underinvesting in AI at the expense of boondoggles like the F-35 Joint Strike Fighter will also grow more heated as dollars grow more scarce. For Klunder, the issue is more personal.

The timing of the ONR briefing happens to coincide with the 14-year anniversary of the bombing of the USS Cole off the coast of Yemen that killed 17 sailors. It’s an anniversary that Klunder observes with a unique sense of responsibility. “If we had this capability there on that day. We could have saved that ship,” he said. “I never want to see the USS Cole happen again.”