Systems and methods decoding intended symbols from neural activity

Patent metadata

Field	Value
Publication number	US-11640204-B2
Application number	US-202017006645-A
Country	US
Kind code	B2
Filing date	Aug 28, 2020
Priority date	Aug 28, 2019
Publication date	May 2, 2023
Grant date	May 2, 2023

Abstract

Systems and methods for decoding intended symbols from neural activity in accordance with embodiments of the invention are illustrated. One embodiment includes a symbol decoding system for brain-computer interfacing, including a neural signal recorder implanted into a brain of a user, and a symbol decoder, the symbol decoder including a processor, and a memory, where the memory includes a symbol decoding application capable of directing the processor to obtain neural signal data from the neural signal recorder, estimate a symbol from the neural signal data using a symbol model, and perform a command associated with the symbol.

First claim

What is claimed is:

1. A symbol decoding system for brain-computer interfacing, comprising:
a neural signal recorder implanted into a brain of a user; and
a symbol decoder, the symbol decoder comprising:
a processor; and
a memory, where the memory comprises a symbol decoding application capable of directing the processor to:
obtain neural signal data from the neural signal recorder, where the neural signal data describes action potentials associated with visualization of the act of hand writing a symbol;
estimate the symbol from the neural signal data using a trained symbol model; and
perform a command associated with the symbol.

2. The symbol decoding system for brain-computer interfacing of claim 1 , wherein the neural signal recorder is a microelectrode array comprising a plurality of electrodes.

3. The symbol decoding system for brain-computer interfacing of claim 2 , wherein the neural signal data describes spikes of neurons in proximity to respective electrodes in the plurality of electrodes.

4. The symbol decoding system for brain-computer interfacing of claim 1 , further comprising at least one output device.

5. The symbol decoding system for brain-computer interfacing of claim 4 , wherein the output device is selected from the group consisting of: vocalizers, displays, prosthetics, and computer systems.

6. The symbol decoding system for brain-computer interfacing of claim 1 , wherein the trained symbol model is selected from the group consisting of: recurrent neural networks (RNNs), long short-term memory (LSTM) networks, temporal convolutional networks, and hidden Markov models (HMMs).

7. The symbol decoding system for brain-computer interfacing of claim 1 , wherein the trained symbol model is a recurrent neural network (RNN), and to estimate the symbol from the neural signal data, the symbol decoding application further directs the processor to:
temporally bin the neural signal data to create at least one neural population time series;
convert the at least one neural population time series into at least one time probability series; and
identify a most likely symbol from the at least one time probability series after a time delay triggered by identification of a high probability of a new character in the at least one time probability series.

8. The symbol decoding system for brain-computer interfacing of claim 1 , wherein the memory further comprises a symbol database comprising:
a plurality of symbols; and
a plurality of commands;
wherein each symbol in the plurality of symbols is associated with a command.

9. The symbol decoding system for brain-computer interfacing of claim 8 , wherein:
the plurality of symbols comprises letters of an alphabet; and
each letter of the alphabet is associated with a command to print the letter to a text string.

10. The symbol decoding system for brain-computer interfacing of claim 9 , wherein the symbols for each letter in the alphabet are difference maximized.

11. A method for decoding symbols from neural activity, comprising:
obtaining neural signal data from a neural signal recorder implanted into a brain of a user, where the neural signal data describes action potentials associated with visualization of the act of hand writing a symbol;
estimating the symbol from the neural signal data using a trained symbol model; and
perform a command associated with the symbol using a symbol decoder.

12. The method for decoding symbols from neural activity of claim 11 , wherein the neural signal recorder is a microelectrode array comprising a plurality of electrodes.

13. The method for decoding symbols from neural activity of claim 12 , wherein the neural signal data describes spikes of neurons in proximity to respective electrodes in the plurality of electrodes.

14. The method for decoding symbols from neural activity of claim 11 , further comprising performing the command using at least one output device.

15. The method for decoding symbols from neural activity of claim 14 , wherein the output device is selected from the group consisting of: vocalizers, displays, prosthetics, and computer systems.

16. The method for decoding symbols from neural activity of claim 11 , wherein the trained symbol model is selected from the group consisting of: recurrent neural networks (RNNs), long short-term memory (LSTM) networks, temporal convolutional networks, and hidden Markov models (HMMs).

17. The method for decoding symbols from neural activity of claim 11 , wherein the trained symbol model is a recurrent neural network (RNN), and estimating the symbol from the neural signal data comprises:
temporally binning the neural signal data to create at least one neural population time series;
converting the at least one neural population time series into at least one time probability series; and
identifying a most likely symbol from the at least one time probability series after a time delay triggered by identification of a high probability of a new character in the at least one time probability series.

18. The method for decoding symbols from neural activity of claim 11 , wherein the symbol and the command are stored in a symbol database comprising:
a plurality of symbols; and
a plurality of commands;
wherein each symbol in the plurality of symbols is associated with a command.

19. The method for decoding symbols from neural activity of claim 18 , wherein:
the plurality of symbols comprises letters of an alphabet; and
each letter of the alphabet is associated with a command to print the letter to a text string.

20. The method for decoding symbols from neural activity of claim 19 , wherein the symbols for each letter in the alphabet are difference maximized.

Assignees

• Univ Leland Stanford Junior

Inventors

• Shenoy Krishna V
• Henderson Jaimie M
• Willett Francis Robert

Classifications

• G06N3/09

  Supervised learning · CPC title

• G06N3/0442

  characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06F18/41

  Interactive pattern learning with a human teacher · CPC title

• G06F3/015 Primary

  Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection · CPC title