To improve on past limitations, Cogan teamed up with fellow Duke Institute for Brain Sciences faculty memberย Jonathan Viventi, Ph.D., whose biomedical engineering lab specializes in making high-density, ultra-thin, and flexible brain sensors.
For this project, Viventi and his team packed an impressive 256 microscopic brain sensors onto a postage stamp-sized piece of flexible, medical-grade plastic. Neurons just a grain of sand apart can have wildly different activity patterns when coordinating speech, so itโs necessary to distinguish signals from neighboring brain cells to help make accurate predictions about intended speech.
After fabricating the new implant, Cogan and Viventi teamed up with several Duke University Hospital neurosurgeons, includingย Derek Southwell, M.D., Ph.D.,ย Nandan Lad, M.D., Ph.D., andย Allan Friedman, M.D., who helped recruit four patients to test the implants. The experiment required the researchers to place the device temporarily in patients who were undergoing brain surgery for some other condition, such as ย treating Parkinsonโs disease or having a tumor removed. Time was limited for Cogan and his team to test drive their device in the OR.
โI like to compare it to a NASCAR pit crew,โ Cogan said. โWe don’t want to add any extra time to the operating procedure, so we had to be in and out within 15 minutes. As soon as the surgeon and the medical team said โGo!โ we rushed into action and the patient performed the task.โ
The task was a simple listen-and-repeat activity. Participants heard a series of nonsense words, like โava,โ โkug,โ or โvip,โ and then spoke each one aloud. The device recorded activity from each patientโs speech motor cortex as it coordinated nearly 100 muscles that move the lips, tongue, jaw, and larynx.
Share this article
Afterwards, Suseendrakumar Duraivel, the first author of the new report and a biomedical engineering graduate student at Duke, took the neural and speech data from the surgery suite and fed it into a machine learning algorithm to see how accurately it could predict what sound was being made, based only on the brain activity recordings.
For some sounds and participants, like /g/ in the word โgak,โ ย the decoder got it right 84% of the time when it was the first sound in a string of three that made up a given nonsense word.
Accuracy dropped, though, as the decoder parsed out sounds in the middle or at the end of a nonsense word. It also struggled if two sounds were similar, like /p/ and /b/.
Overall, the decoder was accurate 40% of the time. That may seem like a humble test score, but it was quite impressive given that similar brain-to-speech technical feats require hours or days-worth of data to draw from. The speech decoding algorithm Duraivel used, however, was working with only 90 seconds of spoken data from the 15-minute test.
Duraivel and his mentors are excited about making a cordless version of the device with aย recent $2.4M grantย from the National Institutes of Health.
โWe’re now developing the same kind of recording devices, but without any wires,โ Cogan said. โYou’d be able to move around, and you wouldn’t have to be tied to an electrical outlet, which is really exciting.โ
While their work is encouraging, thereโs still a long way to go for Viventi and Coganโs speech prosthetic to hit the shelves anytime soon.
โWe’re at the point where it’s still much slower than natural speech,โ Viventi said in aย recent Duke Magazine pieceย about the technology, โbut you can see the trajectory where you might be able to get there.โ
