Researchers from two prominent California universities, UC San Francisco and UC Berkeley, have achieved a significant milestone in the realm of neurotechnology.
They've designed a cutting-edge brain-computer interface (BCI) that allowed a woman, paralyzed due to a brainstem stroke, to articulate thoughts and communicate via a digital avatar. Notably, this innovation is the first of its kind, converting brain signals into both speech and facial expressions.
Image Credit: ioat via Shutterstock / HDR tune by Universal-Sci
This technological breakthrough is capable of transforming these brain signals into text with remarkable efficiency, clocking nearly 80 words per minute. This feat showcases a significant leap from existing BCI technology. The research, recently published on Aug. 23, 2023, in the journal Nature, carries the hope of Dr. Edward Chang, a leading figure in the study, that this BCI will soon gain FDA approval. Chang has dedicated over a decade to BCI research and is enthusiastic about its potential to revolutionize communication for patients with similar conditions.
In the past, Chang's research team had displayed the capability to turn brain signals into text. Their latest endeavor, however, is even more ambitious. The aim was to capture the intricate nuances of speech and the associated facial movements during a conversation. To bring this to fruition, the team adopted a sophisticated approach. They carefully implanted a matrix of 253 electrodes onto critical speech areas of the woman's brain. These electrodes picked up the brain signals that, under normal circumstances, would have controlled her speech muscles.
The project's success relied heavily on meticulously training its embedded artificial intelligence. Rather than training the AI to recognize entire words, which could be cumbersome and slow, the team adopted a novel strategy. They trained the AI to decode words using phonemes – the smallest units of spoken language. For instance, the word "Hello" was broken down into its phonemes: "HH," "AH," "L," and "OW." By focusing on just 39 key phonemes, the system could effectively decipher any English word, making it both faster and more accurate.
To ensure the digital voice closely resembled the woman's original voice, the team used an innovative algorithm alongside personal recordings from her past. For the facial animations of the digital avatar, a collaboration with Speech Graphics, a company known for its expertise in AI-powered facial animation, was pivotal. This synergy allowed for the precise simulation of facial expressions and movements, ranging from basic lip movements to complex emotions like happiness and sadness.
One of the significant future goals of this project is to transcend the current wired setup. The research team is keen on crafting a wireless version of the BCI. Such an enhancement would not only provide users with more freedom but also drastically improve their quality of life, granting them the independence to interact seamlessly with their environment and digital devices.
If you are interested in reading more about the researcher, be sure to check out the paper published in the peer-reviewed science journal nature, listed below.
Sources and further reading:
Too busy to follow science news during the week? - Consider subscribing to our (free) newsletter - (Universal-Sci Weekly) - and get the 5 most interesting science articles of the week in your inbox
FEATURED ARTICLES:
The way you prepare your vegetables affects their nutritional value. What is the healthiest method to prepare them?