Researchers have developed a wearable ‘smart choker’ that uses a combination of flexible electronics and artificial intelligence techniques to allow people with speech impairments to communicate by detecting tiny movements in the throat.
The smart choker, developed by researchers at the University of Cambridge, incorporates electronic sensors in a soft, stretchable fabric, and is comfortable to wear. The device could be useful for people who have temporary or permanent speech impairments, whether due to laryngeal surgery, or conditions such as Parkinson’s, stroke or cerebral palsy.
By incorporating machine learning techniques, the smart choker can also successfully recognise differences in pronunciation, accent and vocabulary between users, reducing the amount of training required.
The choker is a type of technology known as a silent speech interface, which analyses non-vocal signals to decode speech in silent conditions – the user only needs to mouth the words in order for them to be captured. The captured speech signals can then be transferred to a computer or speaker to facilitate conversation.
Tests of the smart choker showed it could recognise words with over 95% accuracy, while using 90% less computational energy than existing state-of-the art technologies. The results are reported in the journal npj Flexible Electronics.
“Current solutions for people with speech impairments often fail to capture words and require a lot of training,” said Dr Luigi Occhipinti from the Cambridge Graphene Centre, who led the research. “They are also rigid, bulky and sometimes require invasive surgery to the throat.”
The smart choker developed by Occhipinti and his colleagues outperforms current technologies on accuracy, requires less computing power, is comfortable for users to wear, and can be removed whenever it’s not needed. The choker is made from a sustainable bamboo-based textile, with strain sensors based on graphene ink incorporated in the fabric. When the sensors detect any strain, tiny, controllable cracks form in the graphene. The sensitivity of the sensors is more than four times higher than existing state of the art.
“These sensors can detect tiny vibrations, such as those formed in the throat when whispering or even silently mouthing words, which makes them ideal for speech detection,” said Occhipinti. “By combining the ultra-high sensitivity of the sensors with highly efficient machine learning, we’ve come up with a device we think could help a lot of people who struggle with their speech.”
Vocal signals are incredibly complex, so associating a specific signal with a specific word requires a high level of computational processing. “On top of that, every person is different in terms of the way they speak, and machine learning gives us the tools we need to learn and adapt the interpretation of signals from person to person,” said Occhipinti.
The researchers trained their machine learning model on a database of the most frequently used words in English, and selected words which are frequently confused with each other, such as ‘book’ and ‘look’. The model was trained with a variety of users, including different genders, native and non-native English speakers, as well as people with different accents and different speaking speeds.
Thanks to the device’s ability to capture rich dynamic signal characteristics, the researchers found it possible to use lightweight neural network architectures with simplified depth and signal dimensions to extract and enhance the speech information features. This resulted in a machine learning model with high computational and energy efficiency, ideal for integration in battery-operated wearable devices with real-time AI processing capabilities.
“We chose to train the model with lots of different English speakers, so we could show it was capable of learning,” said Occhipinti. “Machine learning has the capability to learn quickly and efficiently from one user to the next, so the retraining process is quick.”
Tests of the smart choker showed it was 95.25% accurate in decoding speech. “I was surprised at just how sensitive the device is,” said Occhipinti. “We couldn’t capture all the signals and complexity of human speech before, but now that we can, it unlocks a whole new set of potential applications.”
Although the choker will have to undergo extensive testing and clinical trials before it is approved for use in patients with speech impairments, the researchers say that their smart choker could also be used in other health monitoring applications, or for improving communication in noisy or secure environments.
The research was supported in part by the EU Graphene Flagship and the Engineering and Physical Sciences Research Council (EPSRC), part of UK Research and Innovation (UKRI).
Reference:
Chenyu Tang et al. ‘Ultrasensitive textile strain sensors redefine wearable silent speech interfaces with high machine learning efficiency.’ npj Flexible Electronics (2024). DOI: 10.1038/s41528-024-00315-1