Someone Developed an App That Lets You Speak to an Amazon Echo Using Sign Language

By Tom Pritchard on at

Lots of people like these new smart speakers tech companies have been basically giving away, but they're not the most inclusive devices in the world. For one they rely on voice and sound to operate, which isn't great for people who are deaf, mute, or both, so some things have been done to try and fix that. That includes one intrepid developer figuring out how to communicate with an Amazon Echo using sign language gestures.

Abhishek Singh created the app in question, and using a camera based system it was possible to communicate with the Alexa voice assistant using American Sign Language and seeing her responses as text on screen. Singh himself is not deaf, but developed this app for those who are and ensure that they're not left behind as the world increasingly adopts voice recognition systems. Here's a video of it in action:

As you can see it's not Alexa or the Echo that's doing the hard work, rather it's the app on the laptop next to it. It interprets Singh's gestures and says them out loud for the Echo to hear. The app then transcribes Alexa's response using Google's Text-to-speech software and displays it on screen for the user to read. As you can see it's not 100 per cent perfect, but it's pretty damn close.

In this case the laptop is the intermediary between the user and the Echo, but Singh says there's no reason why screen-based virtual assistants couldn't offer this functionality by default. The Echo Show has a built in camera that lets users make video calls, as do the Fire Tablets which can be converted into makeshift Echo Shows using 'Show Mode'.

While it might take some time for Amazon and other companies to roll out proper sign language support to their devices, Amazon has announced Alexa Captioning is rolling out outside of the US from today. It's available on the Echo Show and Echo Spot, with the toggle inside the accessibility section of the settings menu. US users will also be getting Tap to Alexa, which lets them get information out of their Echo without having to say anything. That'll be rolling out internationally in future, though Amazon didn't say when.

It's not quite the same as having an Echo that can understand sign language, but it's a good start. [BBC News]