“Helsinki VTS, thank you for permission to depart,” the captain says over the radio. He checks with the Vessel Traffic Service to see if there’s anything to be looking out for. Just one other big ship, but also lots of small boats, enjoying the calm water, which could be hazards. Not a problem for this captain—he has a giant screen on the bridge, which overlays the environment around his vessel with an augmented reality view. He can navigate the Baltic Discoverer confidently out of Finland's Helsinki Port using the computer-enhanced vision of the world, with artificial intelligence spotting and labeling every other water user, the shore, and navigation markers.

This not-too-far-in-the-future vision comes from Rolls-Royce. (One iteration of it, anyway: The Rolls-Royce car company, the jet engine maker, and this marine-focused enterprise all have different corporate owners.) The view provided to the crew of the (fictional) Baltic Discoverer is an example of the company's Intelligent Awareness system, which mashes together data from sensors all over a vessel, to give its humans a better view of the world. But that’s just the early part of the plan. Using cameras, lidar, and radar, Rolls wants to make completely autonomous ships. And it's already running trials around the world.

“Tugs, ferries, and short-sea transport, these are all classes of vessels that we believe would be suitable for completely autonomous operations, monitored by a land based crew, who get to go home every night,” says Kevin Daffey, Rolls-Royce's director of marine engineering and technology. Suitable, because they all currently rely on humans who demand to be paid—and can make costly mistakes. Over the past decade, there have been more than 1,000 total losses of large ships, and at least 70 percent of those resulted from human error. The argument looks a lot like the one for self-driving cars: Machines stay sober and focused, beat out human reaction times, and can look in every direction at once. Get their programming right, and they should crash less than humans do.

Moreover, the economic case for automating shipping is clear: About 100,000 large vessels are currently sailing the world's oceans, and the amount of cargo they carry is projected to grow around 4 percent a year, according to the United Nations Conference on Trade and Development. Beyond preventing accidents, human-free ships could be 15 percent more efficient to run, because they don't need energy-gobbling life support systems, doing things like heating, cooking, and lugging drinking water along for the ride.

[#video: https://www.youtube.com/embed/VC7fZvM-NXI

The ship's sensors are similar to those you'd find on your neighborhood self-driving car, with a few important changes. Their cameras have to provide enough information to identify small objects on the horizon, which means they need a higher pixel count for better resolution, which means more data to sort through. “It’s a terabyte of data a day,” Daffey says. That requires a huge amount of processing power, so Rolls is working with Intel to install server rooms on its vessels. The sensors generate a staggering amount of data. All of it's stored onboard and uploaded to the cloud once per month, when the ships dock.

Collecting the data is only step one. Understanding it comes thanks to a neural net, which Rolls-Royce is training with 5 million images scraped off the internet. It’s using crowdsourcing to identify objects (like car companies do with road features) to train its AI what boats look like from every angle, how markers appear, and where coastlines begin and end. It’s also running a trial system on a Japanese ferry out of Kobe, which typically runs at night, so is able to collect a lot of thermal-camera night-vision images, and on ferries which run off the southwest coast of Finland. “The great thing about the sea there is the huge amount of traffic, so we see lots of different, difficult-to-identify craft, like pleasure vessels,” Daffey says.