It’s not always this easy to spot emotion Westend61/Getty

Can you recognise when someone is unwell just by studying their face? Understanding expressions can help doctors improve their diagnoses, but it’s a difficult skill to practise. So a group of engineers have made a tool for training clinicians: a robot that can express pain.

Many doctors already use robotic patient simulators in their training to practise procedures and test their diagnostic abilities. “These robots can bleed, breathe and react to medication,” says Laurel Riek at the University of California, San Diego. “They are incredible, but there is a major design flaw – their face.”

Patient simulators usually have static faces, often with an open mouth so doctors can practise checking airways. This means that, unlike a real patient, they show no emotion.


To change this, Riek and her team have given a robotic face the ability to make expressions of pain, disgust and anger, to help emulate realistic patient feedback. “Interpreting a patient’s facial expressions can help determine if they are having a stroke, are in pain or are having a reaction to medication, so doctors need to be able to do this from day one,” Riek says.

In preparation for a trial at a medical school later this year, the researchers tested how well people could perceive emotions from the robot’s facial expressions. They also tested their ability to assess the expressions of a virtual avatar that might offer an alternative training option.

Mapping pain

To create the robot and avatar, the researchers collected videos of people expressing pain, disgust and anger, and used face-tracking software to convert their expressions into a series of moving points. They then mapped these onto the robot face and the avatar. The robot used was Hanson Robotics’ Philip K. Dick, a humanoid modelled after the sci-fi writer that has realistic rubber skin and can move its facial features.

Videos of the robot and avatar were shown to 102 volunteers, who had to judge which emotion matched which expression. Half the volunteers were clinicians, such as doctors, nurses and pharmacists, and half had no medical background.

The clinicians turned out to be less accurate than the non-clinicians at recognising both pain and anger. In the starkest difference, the clinicians correctly identified pain expressed by the virtual avatar only 54 per cent of the time, compared with 83 per cent for the non-clinicians. Averaged across the robot and avatar, the groups had similar accuracy levels for disgust.

This follows previous research that suggests doctors are worse at interpreting pain in humans than laypeople and tend to underestimate the severity of pain. This could partly be a result of medical training decreasing levels of empathy.

Expressive robots

The researchers think the expressive robots and avatars could help train doctors to better interpret pain. A robot could be particularly useful as students would be able to practise assessing physical symptoms while also reading facial expressions. They presented their research this week at the Conference on Human-Robot Interaction in Vienna, Austria.

“This work could be used very soon to better train our medical professionals and improve patient outcomes,” says Priscilla Briggs, a software engineer at Google with a background in human-robot interaction. But further work will be required to show that a robot’s expressions can improve clinicians’ performance, she says.

In the trial that Riek and her colleagues plan for later this year at the University of California, San Diego, student doctors will use the robot in simulated scenarios such as a person recovering from a stroke.

“We will explore how realism and expressiveness affects student learning outcomes, their sense of immersion and how well they assess patient cues in order to accurately and safely intervene,” says Riek.