In the aftermath of an earthquake, a snakelike robot that can crawl through rubble and tight air pockets is able to access places that no person could — or should — be able to go.

The Sarcos Guardian S, a small robotic visual inspection platform, is designed for exactly those scenarios: searching for cracks in industrial pipelines, finding people trapped in unstable buildings, sensing whether hazardous gases at an accident site could pose a safety risk to first responders.

Today, the robot is controlled by someone working at a safe distance, who sees the scene through its cameras and guides it with the equivalent of a video game joystick. Now, Microsoft and Sarcos are collaborating to add intelligent capabilities to the Guardian S that would allow it to navigate more autonomously — freeing the operator to focus on more important decisions.

The idea of automated industrial applications and using robots isn’t new. Robot arms now move products along an assembly line, machines turn hunks of metal into parts, a car shifts gears without your input.

But that’s a far cry from systems that are actually autonomous — ones that are capable of sensing their surroundings and knowing what to do when confronted with unfamiliar situations. Instead of performing specific tasks repeatedly without variation, these autonomous systems can dynamically respond to changing environments to solve a difficult problem. They also have vast potential to augment how people do their jobs or to perform work that is unsafe or cost-prohibitive for people to do.

Microsoft is building an end-to-end toolchain to help make it easier for every developer and every organization to create autonomous systems for their own scenarios — whether that’s a robot that can help in life-threatening situations, a drone that can inspect remote equipment or systems that help reduce downtime in a factory by autonomously calibrating equipment.

“Machines have been progressing on a path from being completely manual to having a fixed automated function to becoming intelligent where they can actually deal with real-world situations themselves,” said Gurdeep Pall, Microsoft vice president for Business AI. “We want to help accelerate that journey, without requiring our customers to have an army of AI experts.”

Today at the Microsoft Build developers conference, the company is announcing the platform’s first component: a limited preview program for developers to work with its experts to build intelligent agents using Microsoft AI and Azure tools that can autonomously run physical systems. That team includes longtime Microsoft researchers and engineers and experts from Bonsai, which Microsoft acquired last year.

Microsoft’s platform to help developers create autonomous systems employs:

Unique machine teaching tools that enable domain experts to use their knowledge to create AI systems without data science skills

Simulation technologies, such as Microsoft’s AirSim or industry simulators, that allow machines to learn in safe yet highly realistic environment

It will also draw from Microsoft’s diverse portfolio of Internet of Things services, an easy-to-use deep reinforcement learning platform and other AI solutions, and tools like ROS for Windows that allow developers to build intelligent robotic systems — all running on a trusted and secure platform, whether it’s on a device or in the cloud.

Early customers who participate in the limited preview program will learn how to use the same autonomous systems tools as companies like Toyota Material Handling, which is working with Microsoft to develop intelligent and autonomous forklifts.

Sarcos, for instance, was looking for an autonomous systems solution that would combine the best of what machines have to offer with human intellect and intuition, said Kristi Martindale, executive vice president and chief marketing officer for Sarcos.

Today, the person controlling a commercial Guardian S robot has to direct some of his or her attention to pushing buttons and levers on a joystick to guide it through tight spaces and over varied terrain. It can take several steps to appropriately manipulate each segment of the snake over a common landscape feature like stairs.

Using elements of Microsoft’s toolchain, engineers were able to develop an autonomous control system that enables the snakelike robot to avoid obstacles, navigate stairs and climb a metallic wall on its own.

In a real-world scenario, the operator would still play a role in guiding the robot. But if the Guardian S can sense its surroundings and perform all the intermediate motions to traverse stairs on its own, the operator can focus on assessing the scene and making more critical judgement calls, Martindale said.

“We are looking to offload the tasks that can be automated — how does the robot climb a stair? How does it move around obstacle? — so the operator can focus on the more important parts of the job,” she said. “The human is still there to say, ‘No you actually want to go to that obstacle over there because maybe that obstacle is a person who is hurt.’”

A journey from automated to autonomous systems

When people think of autonomous systems, many go straight to the vision of the fully autonomous car that drives itself while you sit in the back seat and read a book, said Mark Hammond, former Bonsai CEO and Microsoft general manager for Business AI.

But car manufacturers have been integrating autonomous features into cars for years, like cruise control or anti-lock braking systems that sense what a driver is trying to do when they encounter a hazard on a wet, slippery road. If that person slams on the brakes in a way that might lock the wheels, that control system takes over and prevents the car from losing traction.

Microsoft’s vision is to help other types of companies — from smart building and energy companies to industrial manufacturers — achieve these incremental steps towards autonomy in their own industries. As the Sarcos robot example shows, many will find the greatest value with humans still in the loop, Hammond said.

“In any sort of operation where you have a mechanical system that interacts with the physical world, you can probably make it smarter and more autonomous,” Hammond said. “But keeping people in the loop is still very desirable, and the goal is really to increase the capabilities of what those humans can do.”

Reinforcement learning is a branch of AI in which algorithms learn by executing a series of decisions and are rewarded or penalized based on which actions get them closer to an end goal. It’s well suited to help machines learn how to do autonomous control tasks, like deciding how to steer an underground drill or angle a tractor blade depending on whether the earth is lumpy or sandy or rocky.

But while deep reinforcement learning algorithms have successfully beat people in video games, mastering real world tasks has been more challenging. In the physical world, the dynamic environments that an autonomous system might encounter — with people and objects moving in unpredictable ways or minute-by-minute changes in temperature or weather — can be far more complicated. Pinpointing exactly where the system went wrong in a long sequence of steps is a difficult computational task.