Sign language translating devices are cool. But are they useful? DeepASL's camera. (Michigan State University)
Sign language translating devices are cool. But are they useful?
Lexile

Assign to Google Classroom

Over the past several decades, researchers have regularly developed devices. They are meant to translate American Sign Language (ASL) to English. They hope to ease communication. They aim to help people who are deaf and hard of hearing and the hearing world talk. Many of these technologies use gloves. These gloves capture the motion of signing. They can be bulky and awkward.

A group of researchers at Michigan State University (MSU) has developed a glove-less device. It is the size of a tube of Chapstick. They hope it will improve ASL-English translation.

The technology is called DeepASL. It uses a camera device to capture hand motions. Then it feeds the data through a deep learning algorithm. It matches it to signs of ASL. It is unlike many previous devices. DeepASL can translate whole sentences rather than single words. It doesn't require users to pause between signs.

"This is a truly non-intrusive technology," says Mi Zhang. He is a professor of electrical and computer engineering. He led the research.

Zhang and his team hope DeepASL can help people who are deaf and hard of hearing by serving as a real-time translator. It could be especially useful in emergency situations, Zhang says. The device can be used with a phone or tablet. It can also be used with a computer. It can help teach ASL, Zhang says. 

More than 90 percent of deaf children are born to parents who are hearing. There is a large community of adults who need to learn ASL quickly. DeepASL could serve as a digital tutor. It gives feedback on whether learners are signing correctly.

Zhang has applied for a patent. He hopes to have a device on the market within a year. Because it's based on affordable technology it could be more widely accessible than previous efforts.

Christian Vogler is a professor of communication studies at Gallaudet University. It is a university for people who are deaf or hard of hearing. He is skeptical of devices designed to translate ASL. His skepticism is shared by many in the Deaf community.

Devices generally do not truly 'translate' ASL. They merely recognize hand signs and turn them into an English word per sign, Vogler says. This means key grammatical information is lost. That includes information about whether a phrase is a question or a negation. And if a phrase is a relative clause and so forth. 

DeepASL does translate full sentences. But some features of ASL grammar go beyond hand signs. Facial expressions are often used as modifiers. An eyebrow raising can turn a phrase into a question and body positioning can show when the ASL user is quoting someone else.
So far, "none of the systems have been even remotely useful to people who sign," Vogler says. He adds that researchers often seem to have "very little contact with the [Deaf and hard of hearing] community and very little idea of their real needs."

Zhang's team did not test the device on people who were deaf and hard of hearing. He tested it on students in a sign language translation program. Zhang emphasizes that DeepASL is designed to enable only basic communication. It is just a starting place. He says his team hopes to extend DeepASL's capabilities in the future. He wants it to capture facial expressions as well.

"That will be the next significant milestone for us to reach," he says.

Vogler says it's a positive that the MSU technology is using deep learning methods. These have had success with spoken language. The device doesn't need a glove. But it likely has the same pitfalls of any previous system. That's because it doesn't capture face and body movements.

Vogler thinks researchers should move away from the idea that sign language recognition devices can really meet in-person communication needs.

"We have many options for facilitating in-person communication. And until we have something that actually respects the linguistic properties of signed languages and the actual communication behaviors of signers, these efforts will go nowhere near supplanting or replacing them," he says. 

"Instead, people need to work with actual community members, and with people who understand the complexities of signed languages."

Vogler says it would be useful for sign language recognition technology like MSU's to work with voice interfaces like Alexa. The growth of these interfaces is an accessibility challenge for people who are deaf and hard of hearing. That is similar to the challenges the internet has presented for people who are blind over the years. That's because it is a largely visual medium.

"We presently do not have an effective and efficient way to interact with these voice interfaces if we are unable to, or do not want to, use our voice," he says. "Sign language recognition is a perfect match for this situation, and one that actually could end up being useful and getting used."

Filed Under:  
Assigned 94 times
CRITICAL THINKING QUESTION
Why do you think researchers are hesitant to reach out to the Deaf community to get help creating their devices?
Write your answers in the comments section below


COMMENTS (28)
  • Edileh-E2
    9/27/2019 - 10:41 a.m.

    I really enjoyed this article because my grandparents are both kind of hard of hearing. They wear these cool hearing aids the they wear in one ear and it transmits it into the other ear. They cant hear everything or see everything but they are pretty good for hearing.That id the reason that I liked this article.

  • kristynf-
    9/30/2019 - 01:52 p.m.

    Researchers are hesitant to reach out to the deaf community because they might not be able to communicate to them.

  • justinag-
    9/30/2019 - 01:53 p.m.

    I think sign language is very smart. It allows the deaf community to still communicate with other people.

  • enolas-
    9/30/2019 - 01:54 p.m.

    I feel that the DeepASL product can be useful to learning ASL, but maybe not really translate it.

  • alexiss-7
    9/30/2019 - 01:58 p.m.

    researchers are hesitant to reach out to thee deaf community to get help creating their devices because they ant truly understand the struggles of the deaf community.

  • samanthaf-14
    9/30/2019 - 01:59 p.m.

    This article keeps you interested and I thought it was kind of cool that Michigan State was part of it because my cousin goes there. This device will help in the future for sure especially with the deaf and hearing impaired.

  • justinag-
    9/30/2019 - 02:07 p.m.

    I think it is because they can not visually communicate face to face like sign language.

  • turnerb-
    9/30/2019 - 02:14 p.m.

    I think that these devises will help out our whole community because the deaf can communicate with every one.

  • ethanm-10
    9/30/2019 - 02:18 p.m.

    It is a very good idea and it is good someone thought of it and is being made.It will really help the people that are hard of hearing or def. it will also help kids who are def or hard of hearing also.

  • laurenw-4
    9/30/2019 - 02:21 p.m.

    These translating devices are very useful because some people don't have the time to learn sign language so they can just use these devices.

Leave a comment
ADVERTISEMENT
ADVERTISEMENT
Leave a comment