Writing with our brain: the evolution of the interface, by Okezue Bell

This is a technical article. Feel free to approach without any knowledge, or a lot of knowledge about animal physiology, neurology / neuroscience, mathematics, AI, neurochemistry, neurobiology, engineering, and brain-computer interfaces. However, if you want to better understand the concepts discussed in this article, I highly recommend that you read the topic first.

Recent advances in biomedical fields, alongside neuroscience and the development of interfaces, have led to a sophisticated area of ​​product development known as – they go by many names – brain-machine interfaces (BMI), a class of electronic devices capable of receiving, interpreting and possibly manipulating the electrical signals of action potentials released by neuronal activity to stimulate physiological changes, or to control certain functions on digital devices.

To date, most IMC implementations have focused primarily on two areas: neurological research and esculapian applications; Uses of BCIs have also been investigated in robotic space, although this is a relative niche area of ​​study with this technology.

Some of the most prominent examples of the use of BMI technology are those of the company. Neuralink, as well as in independent projects, developing prostheses, imaging the brain connectome, deciphering the neurological pathways that make up the gut-brain axis, and automating robotic appendages for specific surgical uses.

The following is a review of BCIs that use reconstruction of neural images for brain-to-text communication in handwriting.

Neural control interfaces provide two-way communication pathways to enable brain-machine interactions. This channel serves as an electrical bridge, based on signals emitted by the brain during mental activity, harnessing an electrochemical sodium-potassium gradient for biodetection.

Current methods of measuring non-invasive neural activity have a high margin of error, posing problems with priming data and / or using individual brain data to inform machine operation. Although semi-invasive methods are currently under development, the most reliable BMIs today are intracortical.

In Stanford’s high-performance brain-text communication via handwriting, a new method of cognitive restoration via neural interfaces is proposed, in which a patient’s thoughts are converted to writing, as opposed to the more conventional approach. consisting in trying to restore motor function for an individual through an implant, so that he can potentially write. While this is a viable long-term solution, it is highly unlikely that such treatments will be available within the next decade.

While the P300-event keyboard / mouse input concept achieves a similar goal in terms of communication, it does not exactly replicate the user’s intention or take into account more dexterous, non-linear actions, just as it does. writing in cursive or drawing instead of writing.

This new BCI instead collects data from the motor cortex and uses RNN-based neural decoding to reconstruct the writing concept for a patient who suffered SCI paralysis of the hand. The participant achieved typing speeds of 90 characters per minute with raw accuracy of 94.1% online and greater than 99% accuracy offline with general purpose automatic correction, which is close to typing speed average (115 wpm) of participant demographics!

First, they started by determining the optimal BCI locus. Previous literature has already provided researchers with the intuition that intention for general motor functions is informed by the motor cortex, hence the name. However, they still had to study whether more sophisticated motor actions could be generalized to the motor cortex, so they also recorded the electrical activity of the precentral gyrus (Brodmann Area 4 [PG, BA4] neuronal site):

Neuralink-Fig shows the hand button – the epsilon-shaped gyrus on the PG responsible for the power of the manual override.

Data collected from the button sections of the hand of the precentral gyrus via microelectrode arrays (hands limited to µmotive movements) of a T5 quadriplegic participant were performed to reconstruct handwriting, thereby decoding fast and agile movements for paralyzed people.

Using PCA, the researchers were able to render high-variance neural dimensions to create correlations with the activity of the pen imagined on lined paper, and how that would construct a letter. Thirty percent of the neural variance was linear decoding of pen tip speed [the vector representation of how fast and in which direction the pen was moved (and ultimately the displacement of the pen across the paper as well)]. Essentially, the participants imagined themselves writing on paper.

To re-visualize the high dimensionality data of the three main main components, the researchers used the integration of t or t-SNE distribution stochastic neighbors for dimensionality reduction to a 2D visualization. The data were also time-warped to minimize temporal variability.

e. Note that t-SNE charts can be misleading to interpret. Hyperparameters have a huge effect on the result, and the sizes of the data clusters are negligible, because dense clusters are naturally extended (the opposite is true for sparse clusters) by t-SNE. The importance of cluster distance can also be quite elusive; here some of the clusters are not well separated, which means that their distances are of some importance. D. handwriting rendering for each character in the pen tip path (orange = start). b + c. example of principal component analysis for d, e and m; vs is a time-distorted neural activity that minimizes fluctuations / errors between trials.

The t-SNE graph above illustrates the groupings of neural activity for different members of the alphabet. In the case of overlapping clusters, these characters are written in the same way, which means that their motor encodings are homogeneous.

Then, simply by implementing the nearest neighbor classification, the research found an accuracy of 94.1% with a confidence interval (CI) of 95% = [92.6, 95.8], which means that residual motor cortex activity for agile actions is still a feasible BCI option for people who have been paralyzed for a long time.

By using an RNN to add letters to create sentences, Stanford neuro-researchers were also able to develop a sentence decoding system.

20ms bins of neural activity would be converted by an RNN to a pt-d, or probability time series (as opposed to the original neural population time series xt), which served as a corrective prediction for each new one. character with a 1s delay, which resulted in greater certainty. The character probability threshold gave what the researchers called raw online output – real-time decoding output or offline output (a retrospective conclusion) could be calculated from an offline Viberti research. based on probability informed by a custom 50,000 word bigram language model. . The average character and word error rate (with 95% CI) for the handwriting BCI over the 5 days is shown:

After five days of recycling, it was found that there was better performance in unsupervised recycling of the decoder. It has also been found that increased temporal variety can make movements easier to decode!

Overall, the prospect of reactivating written communication for those who are crippled is quite exciting and sparks discussions for a more robust implementation of such extensive technology. Thanks to the researchers (Willett, et. Al.) Who designed, executed and published this study and their results! I would recommend checking and reading the document to get all the details of the BCI they developed.

Before you leave…

My name is Okezue, a developer and researcher obsessed with learning and building things, especially when it comes to biology or computer science. Check out my social networks here or contact me: [email protected]

I write something new every day / week so hope to see you again soon! Be sure to comment and leave some applause on that as well, especially if you liked it! I enjoyed writing it! ??

Twitter | LinkedIn | Website | Newsletter

© 2021 by Okezue bell. All rights reserved.

Source link

Previous WATCH: Shaun White's dog refuses to let him skate; Bites board in protest
Next Readers' Choice: Vote Now for the Times Free Press Wrapping Paper Design Contest for Young Artists