UW Researchers Develop Smartwatch App That Alerts Deaf and Hard-Of-Hearing People To Nearby Sounds

A Smartwatch is a popular device that helps improve its wearer’s health. Now University of...

In the second prototype of HomeSound, the tablets sent information to a smartwatch, which is how the researchers got the idea to make the standalone app. Photo credit: Jain et al./CHI 2020,University of Washington

A Smartwatch is a popular device that helps improve its wearer’s health. Now University of Washington researchers have developed a smartwatch app for deaf and hard-of-hearing people who want to be aware of nearby sounds. Dubbed SoundWatch, the wearable device picks up a sound the user is interested in — examples include a siren, a microwave beeping or a bird chirping — SoundWatch will identify it and send the user a friendly buzz along with information about the sound.

The researchers presented their findings at the ACM conference on computing and accessibility.

Read more: This Smart Glove Interprets Sign Language In Real Time

“This technology provides people with a way to experience sounds that require an action — such as getting food from the microwave when it beeps. But these devices can also enhance people’s experiences and help them feel more connected to the world,” said lead author Dhruv Jain, a UW doctoral student in the Paul G. Allen School of Computer Science & Engineering. “I use the watch prototype to notice birds chirping and waterfall sounds when I am hiking. It makes me feel present in nature. My hope is that other d/Deaf and hard-of-hearing people who are interested in sounds will also find SoundWatch helpful.”

The team started this project by designing a system for d/Deaf and hard-of-hearing people who wanted to be able to know what was going on around their homes, reports UW News.

“I used to sleep through the fire alarm,” said Jain, who was born hard of hearing.

The first system, called HomeSound, uses Microsoft Surface tablets scattered throughout the home which act like a network of interconnected displays. Each display provides a basic floor plan of the house and alerts a user to a sound and its source. The displays also show the sound’s waveforms, to help users identify the sound, and store a history of all the sounds a user might have missed when they were not home.

Because smartwatches have limited storage and processing abilities, the team needed a system that didn’t eat the watch’s battery and was also fast and accurate. First the researchers compared a compressed version of the HomeSound classifier against three other available sound classifiers. The HomeSound variant was the most accurate, but also the slowest.

To speed up the system, the team has the watch send the sound to a device with more processing power — the user’s phone — for classification. Having a phone classify sounds and send the results back to the watch not only saves time but also maintains the user’s privacy because sounds are only transferred between the user’s own devices.

The team tested the SoundWatch app in March 2020 — before Washington’s stay-at-home order — with eight d/Deaf and hard-of-hearing participants in the Seattle area.

Read more: Buzz: A Wearable Wristband That Allows You to Sense Sound Through Your Skin

People found the app was useful for letting them know if there was something that they should pay attention to. For example: that they had left the faucet running or that a car was honking. On the other hand, it sometimes misclassified sounds (labeling a car driving as running water) or was slow to notify users.

The team is also developing HoloSound, which uses augmented reality to provide real-time captions and other sound information through HoloLens glasses.

Sam Draper
October 29, 2020

Innovation of the Month

Do you want to discover more, visit the website
Visit Website

Other news

Positive Results for Theranicas Neuromodulation Device

Nerivio is supposed to be used for the preventive treatment of episodic and chronic migraine.

Digital CBT Most Cost-Effective Intervention for Poor Sleep, Reveals New Study

A new study authored by researchers at Johns Hopkins University, the University of California at...

Lenovo Launches ThinkReality A3 – Advanced and Versatile Smart Glasses at the CES 2021

Today, at the virtual Consumer Electronics Show (CES 2021), Lenovo introduced the ThinkReality A3...

Spanish Researchers Develop Chest Strap System That Monitors Severity of Symptoms in Chronic Fatigue Syndrome

Researchers at the Universitat Autònoma de Barcelona (UAB), Universitat Politècnica de Catalunya ...
Discover more