A group of neuroscientists has created an interface capable of translating what a patient hears, well understood by an external listener. This is an important step that could pave the way for new forms of computer-mediated communication for people unable to speak because they suffer from neurodegenerative diseases or from brain injuries.
THE TURNING OF THE AI
The study, described in Scientific Reports, was made possible thanks to an artificial intelligence system that has learned to recognise the recurring traits of brain waves and to translate them into words. Every time we talk or imagine talking, in fact, the brain produces characteristic patterns of brainwaves, and the same happens when we listen to someone talking, or imagine listening.
So far, attempts to decode brainwaves were based on the computerised analysis of spectrograms – the graphical representations of sound frequencies – but the results left something to be desired.
FROM THE APPEAL TO THE WORD
To train the vocoder scientists implanted electrodes in the brain of epileptic patients, which for therapeutic purposes also allowed to record their brain activity while listening to short stories told by four different readers.
At this point, when the vocoder algorithm is sufficiently “trained”, the same patients listened to a speaker who pronounced the numbers from 0 to 9. The vocoder analysed the brain waves produced in their auditory cortex, and used them to translate those thoughts that they had “heard” into words. The sound produced was analysed and cleaned up by a system of neural networks, a type of AI that mimics the structure and function of neurons in the human brain. External listeners recognised the numbers listed by the robotic voice in 75% of the cases, proof that the signal was well recognisable.
A BRIDGE TO COMMUNICATE
The next steps will be to test the system with more articulate sentences, but above all to test it with the brain signals emitted when a person speaks, tries to speak or imagines to speak. If it works, one could imagine that in the future, at least in a decade, an implant (invasive) based on cerebral electrodes similar to that of some epileptic patients will translate the patient’s thoughts directly into words.