An ambitious robotics project that combines artificial intelligence, machine learning and advanced sensors to understand and assist humans in real time could be truly "revolutionary", according to the team working on it. The SecondHands humanoid, being developed for online supermarket Ocado, could soon be helping factory engineers fix mechanical faults and even learn on the job.

The robot will be completely autonomous and should be able to help with everything from fetching tools to holding objects and even assisting with cleaning and engineering tasks. The project is a collaboration between the technology arm of the online supermarket and four universities across the European Union. The robotics team at Ocado Technology believe it could become "the most advanced assistive robot in the world".

SecondHands will use 3D vision to see both depth and colour, with artificial intelligence allowing it to learn by example and respond to its surroundings. Once trained in a series of basic tasks the robot should be able to increase its own intelligence and act independently. SecondHands will also be able to understand natural speech, allowing it to respond to voice commands.


We want our technicians to be able to rely on these robots Graham Deacon, robotics research team leader, Ocado Technology

In order to operate in a factory designed for humans SecondHands will be based heavily on human morphology. Early versions might operate on wheels, but in the future the robot could move on tank tracks or even legs. It could also have extra abilities such as telescopic arms to make it more useful as an assistant. The robot will be flexible enough to work easily alongside humans, with torque-controlled arms, anthropomorphic hands and a bendable torso.

Read next This CIA spy game reveals the secrets of successful teams This CIA spy game reveals the secrets of successful teams

The robot will eventually be put to work alongside engineers at Ocado's vast logistics factories in the UK, which handle more than 167,000 orders per week. When something goes wrong with a mechanical component SecondHands will help engineers carry out repairs quickly and safely. It could also operate in areas too dangerous for humans, examining high-speed conveyors at close quarters and handling toxic materials.

The EU is funding the project to the tune of €7m (£5.1m) as part of its Horizon2020 initiative to encourage researchers to work more closely with industry partners. As well as coordinating and contributing to the research Ocado will also be the end user, with the robots designed specifically for its factories. If the project is successful the team at Ocado are hopeful it will find uses elsewhere.


The first SecondHands prototype will be operational at an Ocado testing facility in 18 months time and it is hoped the final version will be assisting engineers in factories in 2020. Unlike current collaborative and assistive robots, such as those competing in the recent Darpa challenge, Ocado says SecondHands will work just as quickly as a human.

The robot will use artificial intelligence, machine learning and advanced sensors to assist engineers autonomously Ocado

"The big challenge is to get the robot to proactively do stuff, understanding where it is in the task and then doing something useful," Graham Deacon, leader of Ocado Technology's robotics research team tells WIRED.co.uk. Deacon describes Darpa as "a bit like watching paint dry" as the robots slowly perform simple tasks. "We want our technicians to be able to rely on these robots. These robots have got to work in real time and respond in the right timeframes and be something that the technicians feel comfortable relying on."

Read next Covid-19 has shown how easy it is to automate white-collar work Covid-19 has shown how easy it is to automate white-collar work

SecondHands' potential for high-level reasoning, Deacon explains, is a work of artificial intelligence. Software will help the robot construct a vast knowledgebase around the tasks it carries out and then understand how they can be applied to other problems. In this sense, the robot will learn on the job.

The structure of SecondHands will be based on the next-generation ARMAR robot, developed at the Karlsruhe Institute of Technology in Germany. Ocado will work on the software side, with a speciality of vision-based grasping and manipulation. Additional research into artificial intelligence and contextual understanding is being undertaken at University College London, La Sapienza University of Rome and Ecole Polytechnique Federale de Lausanne.

As it keeps learning it will become more useful Alex Harvey, head of project management, Ocado Technology


The aim of the project is to create a robotic assistant that doesn't require any human input but that understands what it needs to do based on its own understanding and intelligence. "We would expect the robots to be able to track what the engineer is doing, understand the task that the engineer is trying to perform and then synthetically understand its own capabilities as a robot to proactively offer assistance," says Alex Harvey, head of project management at Ocado Technology.

In one example the robot would be able to understand that an engineer is climbing a ladder would need help holding a safety guard once it had been removed. Having seen and understood how this task is performed, SecondHands would then be able to apply its knowledge to other tasks without being told how. "For it to be truly useful it has have a base capability and it has to be able to learn on the job and it has to be able to get better. As it keeps learning it will become more useful," Harvey says. "If SecondHands existed today it would not be stretching the truth to say it would be the most advanced assistive robot in the world," Deacon says. "In five years time somebody else might be doing something similar, but if it were to exist today it would be the most advanced robot of its time."

Paul Clarke, Ocado's director of technology, has even grander ambitions. "Robots of the future could even use these capabilities to actually build warehouses," he says.