Guo Xinhua wants to teach computers to echolocate. He and his colleagues have built a device, about the size of a thin laptop, that emits sound at frequencies 10 times higher than the shrillest note a piccolo can sustain. The pitches it produces are inaudible to the human ear. When Guo’s team aims the device at a person and fires an ultrasonic pitch, the gadget listens for the echo using its hundreds of embedded microphones. Then, employing artificial intelligence techniques, his team tries to decipher what the person is doing from the reflected sound alone.

The technology is still in its infancy, but they’ve achieved some promising initial results. Based at the Wuhan University of Technology, in China, Guo’s team has tested its microphone array on four different college students and found that they can identify whether the person is sitting, standing, walking, or falling, with complete accuracy, they report in a paper published today in Applied Physics Letters. While they still need to test that the technique works on more people, and that it can identify a broader range of behaviors, this demonstration hints at a new technology for surveilling human behavior.

Guo’s device belongs to a category of technology known as human activity recognition, in which a computer analyzes signals to figure out what people are doing. Smartwatch pedometers, for example, convert acceleration and rotation data into the number of steps the wearer has taken. Now, researchers like Guo are designing systems that can identify more complicated human behavior, with the help of more sophisticated AI techniques. Some work with sound data, like Guo; others are developing better image recognition algorithms. Some researchers have even shown that they can identify simple human poses by analyzing ambient Wi-Fi signals, says computer scientist Albrecht Schmidt of the Ludwig Maximilian University of Munich. “When humans move through the signals, they change them,” he says. Fluctuations in Wi-Fi signals can reveal, for example, a person clapping, making a phone call, or squatting.

XINHUA GUO

Guo’s team has designed an algorithm that relates a sound signal to a specific human posture. After the device captures the echo, the algorithm first removes some ambient noise, and then analyzes the data for patterns. The mixture of frequencies in the reflected sound, for example, can offer hints about what’s happening in a room. Whatever pose a person is holding will end up reflecting back more of one pitch than another. The algorithm exploits these differences to determine the person’s posture.