Apparatus and method for decoding sensory and cognitive information from brain activity

Patent metadata

Field	Value
Publication number	US-9451883-B2
Application number	US-201213725893-A
Country	US
Kind code	B2
Filing date	Dec 21, 2012
Priority date	Mar 4, 2009
Publication date	Sep 27, 2016
Grant date	Sep 27, 2016

Abstract

Decoding and reconstructing a subjective perceptual or cognitive experience is described. A first set of brain activity data produced in response to a first brain activity stimulus is acquired from a subject using a brain imaging device. An encoding model is used to convert the brain activity data into a corresponding set of predicted response values. A second set of brain activity data produced in response to a second brain activity stimulus is acquired from a subject and decoded using a decoding distribution derived from the encoding model, and the probability the second set of brain activity data corresponds to said predicted response values is determined. The second set of brain activity stimuli is then reconstructed based on the probability of correspondence between the second set of brain activity data and the predicted response values.

First claim

What is claimed is:

1. A method for decoding and reconstructing a subjective perceptual or cognitive experience, the method comprising:
acquiring a first set of brain activity data from a first subject, wherein said first set of brain activity data is acquired using a first brain imaging device, and wherein said first set of brain activity data is produced in response to a first set of brain activity stimuli;
converting said first set of brain activity data into a corresponding set of encoding model parameter values for each of one or more linearizing feature spaces;
acquiring a second set of brain activity data from said first subject or a second subject, wherein said second set of brain activity data is acquired using a second brain imaging device, and wherein said second set of brain activity data is produced in response to a second set of brain activity stimuli;
creating a decoding database comprising one or more items;
generating a corresponding set of predicted brain activity signals for each of the one or more items in the decoding database using the set of encoding model parameter values corresponding to each of the one or more linearizing features spaces;
determining a probability for each of the one or more items that said second set of brain activity data corresponds to said set of predicted brain activity signals corresponding to the item;
selecting at least one item from the decoding database based on the probability determined for each of the one or more items in the decoding database; and
producing a reconstructed set of brain activity stimuli based on the selected at least one item.

2. A method as recited in claim 1 , wherein said first set of brain activity data or said second set of brain activity data is acquired using a brain imaging technique selected from the group consisting of EEG, MEG, fMRI, fNIRS, SPECT and ECoG.

3. The method of claim 1 , wherein said first set of brain activity stimuli is selected from the group consisting of sensory stimuli, motor stimuli, and cognitive stimuli, or any combination thereof.

4. The method of claim 1 , wherein the selected at least one item comprises a plurality of selected items, and
the reconstructed set of brain activity stimuli is produced by computing an average or weighted average of the plurality of selected items.

5. The method of claim 1 , wherein said second set of brain activity stimuli is selected from the group consisting of sensory stimuli, motor stimuli, and cognitive stimuli, or any combination thereof.

6. A method as recited in claim 1 , wherein determining the probability for each of the one or more items comprises constructing a decoding distribution, and
said decoding distribution is calculated using a formula that comprises:

p
(

α
⁢


r
)

=

∑
s

⁢

∑
h

⁢

p
(

r
⁢



s
,
h

)

⁢

p
(

s
⁢


α
)

⁢

p
(

r
train

⁢



s
train

,
h

)

⁢

p
⁡

(
h
)

⁢

p
⁡

(
α
)

Z

wherein α represents a value of said second set of brain activity stimuli, r is a measurement of said second set of brain activity data, s is said second set of brain activity stimuli, and h is the set of encoding model parameter values corresponding to each of the one or more linearizing feature spaces.

7. A method as recited in claim 6 ,
wherein constructing said decoding distribution further comprises:
evaluating said r, said h, and said s of said decoding distribution by constructing a portion of the formula, the portion being p(r|s,h);
integrating over all possible parameter values with respect to p(h); and
integrating over all possible stimuli with respect to p(s|α).

8. A method as recited in claim 1 , wherein the encoding model parameter values corresponding to each of the one or more linearizing feature spaces are part of an encoding model that describes a relationship between said first set of brain activity data and said first set of brain activity stimuli,
said encoding model comprises a non-linear transform for each of the one or more linearizing feature spaces, and
the non-linear transform for at least one of the one or more linearizing feature spaces is a Gabor wavelet transformation.

9. A method as recited in claim 1 , wherein the encoding model parameter values corresponding to each of the one or more linearzing feature spaces are part of an encoding model that describes a relationship between said first set of brain activity data and said first set of brain activity stimuli,
said encoding model comprises a non-linear transform for each of the one or more linearizin

Assignees

• Univ California

Inventors

• Gallant Jack L
• Naselaris Thomas
• Kay Kendrick
• Prenger Ryan

Classifications

• G06F2218/12

  Classification; Matching · CPC title

• A61B5/0075

  by spectroscopy, i.e. measuring spectra, e.g. Raman spectroscopy, infrared absorption spectroscopy (A61B5/0071 takes precedence) · CPC title

• A61B5/04009

  Human Necessities · mapped topic

• A61B5/04017

  Human Necessities · mapped topic

• A61B5/0042 Primary

  for the brain · CPC title