SoundWatch
Open Access
- 20 May 2022
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in Communications of the ACM
- Vol. 65 (6), 100-108
- https://doi.org/10.1145/3531447
Abstract
Smartwatches have the potential to provide glanceable, always-available sound feedback to people who are deaf or hard of hearing (DHH). We present SoundWatch, a smartwatch-based deep learning application to sense, classify, and provide feedback about sounds occurring in the environment. To design SoundWatch, we first examined four low-resource sound classification models across four device architectures: watch-only, watch+phone, watch+phone+cloud, and watch+cloud. We found that the best model, VGG-lite, performed similar to the state of the art for nonportable devices although requiring substantially less memory (∼1/3rd) and that the watch+phone architecture provided the best balance among CPU, memory, network usage, and latency. Based on these results, we built and conducted a lab evaluation of our smartwatch app with eight DHH participants. We found support for our sound classification app but also uncovered concerns with misclassifications, latency, and privacy.Funding Information
- National Science Foundation (IIS-1763199)
This publication has 22 references indexed in Scilit:
- A Personalizable Mobile Sound Detector App Design for Deaf and Hard-of-Hearing UsersPublished by Association for Computing Machinery (ACM) ,2016
- Reliable detection of audio events in highly noisy environmentsPattern Recognition Letters, 2015
- Head-Mounted Display Visualizations to Support Sound Awareness for the Deaf and Hard of HearingPublished by Association for Computing Machinery (ACM) ,2015
- A Pilot Study about the Smartwatch as Assistive Device for Deaf PeoplePublished by Association for Computing Machinery (ACM) ,2015
- A comparative study of in-sensor processing vs. raw data transmission using ZigBee, BLE and Wi-Fi for data intensive monitoring applicationsPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2014
- VisAural:Published by Association for Computing Machinery (ACM) ,2014
- Evaluating non-speech sound visualizations for the deafBehaviour & Information Technology, 2006
- Using thematic analysis in psychologyQualitative Research in Psychology, 2006
- Tactual display of consonant voicing as a supplement to lipreadingThe Journal of the Acoustical Society of America, 2005
- Content analysis for audio classification and segmentationIEEE Transactions on Speech and Audio Processing, 2002