The hands say “interpreter” but not everyone understands Timothy Revell

Handwriting will never be the same again. A new glove developed at the University of California, San Diego, can convert the 26 letters of American Sign Language (ASL) into text on a smartphone or computer screen. Because it’s cheaper and more portable than other automatic sign language translators on the market, it could be a game changer. People in the deaf community will be able to communicate effortlessly with those who don’t understand their language. It may also one day fine-tune our control of robots.

ASL is a language all of its own, but few people outside the deaf community speak it. For many signing is their only language, as learning written English, for example, can be difficult without having the corresponding sounds to go with it.

“For thousands of people in the UK, sign language is their first language,” says Jesal Vishnuram, the technology research manager at the charity Action on Hearing Loss. “Many have little or no written English. Technology like this will completely change their lives.”


When they need to communicate with people who are not versed in ASL, their options are limited. In the UK, someone who is deaf is entitled to a sign language translator at work or when visiting a hospital, but at a train station, for example, it can be incredibly difficult to communicate with people who don’t sign. In this situation a glove that can translate for them would make life much easier.

Knuckle sensors

The device consists of a standard sports glove kitted out with nine flexible strain sensors that are placed over different knuckles. When a user bends their fingers or thumb, the sensors stretch, and their electrical resistance goes up. The software uses these signals to work out the configuration of the hand.

Motion sensors on the back of the glove work out whether the hand is still or in motion, a necessary step to differentiating similar letters. For example, both the signs for “i” and “j” involve extending just the little finger. But for “i” the hand remains still, whereas to signify “j” you rotate your hand 180 degrees. The motion sensors detect these differences.

All these signals are then sent via Bluetooth to an app on your phone, which will display what you want to say.

The flexible sensors mean that you hardly notice that you are wearing the glove, says Timothy O’Connor who is working on the technology at the University of California, San Diego.

That would be an improvement over earlier, similar technology – other translation gloves can have brittle parts and be an effort to use.

Cameras convert signs

This has led other researchers and companies to look at converting sign language into text using cameras. This needs no special equipment other than a smartphone. However, all these devices are often power-hungry, tend not to work very well in low-light conditions, and need to be placed somewhere at a distance from the person signing, which is not always practical.

At the moment, O’Connor’s glove can only interpret the 26 letters of ASL, allowing people to spell out words letter by letter. This is already useful, but to become truly convenient, it will need to be able to translate whole words and phrases too.

“Many people rely heavily on friends and family to translate for them or they simply end up being cut off from the hearing world. So even having a device that could just translate basic phrases would have a huge effect,” says Vishnuram.

Another difficulty can be the big differences between sign languages. For example, the ASL alphabet uses only one hand, but the British Sign Language version uses two. So to import this to the UK would involve some extra work.

Beyond translating sign language, O’Connor and his colleagues are also working on using the same techniques to control robots. “One application in the pipe line is a 3D printed robot hand that we can control using the glove,” says O’Connor. Being able to control a robotic hand this way could find a use in robotic surgery or for bomb-diffusing robots.

Journal reference: PLoS One, DOI: 10.1371/journal.pone.0179766