Display system interface using visually-evoked cortical potentials

Patent metadata

Field	Value
Publication number	US-12197648-B2
Application number	US-202318104913-A
Country	US
Kind code	B2
Filing date	Feb 2, 2023
Priority date	Feb 2, 2023
Publication date	Jan 14, 2025
Grant date	Jan 14, 2025

Abstract

A brain-computer interface system includes a video processor for producing a display signal, a temporal controller for producing a plurality of repetitive visual stimulus (RVS) signals with different respective temporal aspects, a display device that receives the display signal and displays a corresponding image on a plurality of different display regions and receives the RVS signals and displays corresponding RVS in respective ones of the display regions, an electroencephalographic (EEG) sensor for sensing a visually-evoked cortical potential (VECP) signal in a user with eyes fixated on a viewed one of the display regions, and a VECP processor for processing the VECP signal to identify the respective temporal aspect of the respective RVS of the viewed display region to estimate the eye fixation location. The RVS are generated independently of the display update/refresh rate and at sufficiently high frequencies to avoid flicker perceptible to the user.

First claim

What is claimed is:

1. A brain-computer interface system, comprising:
a video processor configured to produce a display control signal;
a temporal controller configured to produce a plurality of repetitive visual stimulus (RVS) signals, each with a different respective temporal aspect, wherein the RVS signals with the different respective temporal aspects are separate and independent from and not embedded in the display control signal;
a display device for displaying an image and having a plurality of display regions, each of the plurality of display regions including a plurality of light emitting elements, the display device being configured to create an image in response to receiving both the display control signal and the RVS signals, the display control signal controls each of the plurality of light emitting elements and the RVS signals control each of the display regions;
an electroencephalographic (EEG) sensor configured to sense a visually-evoked cortical potential (VECP) signal in a user whose eyes are fixed on a viewed one of the display regions; and
a VECP processor configured to process the VECP signal to identify the viewed one of the display regions.

2. The brain-computer interface system of claim 1 , wherein the temporal aspects of the RVS signals include the shape of the frequencies, waveform amplitudes, duty cycle, or phases of the RVS signals, other temporal characteristics of the RVS signals, or a combination thereof.

3. The brain-computer interface system of claim 1 , wherein the temporal aspects of the RVS signals are the sufficiently high frequencies that do not produce visually perceptible flicker of the displayed RVS.

4. The brain-computer interface system of claim 3 , wherein the sufficiently high frequencies of the RVS signals are not an even divisor of an update rate of the display device.

5. The brain-computer interface system of claim 3 , wherein the sufficiently high frequencies of the RVS signals are greater than one-half an update rate of the display device.

6. The brain-computer interface system of claim 1 , wherein the sufficiently high frequencies of the RVS signals are between about 30 and about 120 Hz.

7. The brain-computer interface system of claim 1 , wherein the temporal controller is configured to send a synchronization signal to the VECP processor, wherein the synchronization signal indicates a pulse onset of the RVS signals with the different respective temporal aspects, and in response the VECP processor determines the onset of the RVS for each of the display regions of the display device.

8. The brain-computer interface system of claim 1 , wherein the display device includes a display module and a backlight, wherein the display module is configured to receive the display control signal from the video processor to produce the displayed image and the backlight is configured to receive the RVS signals with the different respective temporal aspects from the temporal controller to produce the RVS with the different respective temporal aspects in the respective display regions.

9. The brain-computer interface system of claim 1 , further comprising an electrooculogram (EOG) sensor configured to sense eye movement and/or fixation in the user and sending a corresponding EOG signal to the VECP processor, wherein the VECP processor is configured to process the EOG signal to identify the viewed one of the display regions indicated by the EOG and performs a reconciliation process to confirm that the viewed display region indicated by the EOG signal matches the viewed display region indicated by the EEG signal.

10. The brain-computer interface system of claim 1 , wherein the temporal controller is provided by a plurality of temporal controllers in a distributed arrangement that each produce one or more of the repetitive visual stimulus (RVS) signals with the different respective temporal aspects.

11. The brain-computer interface system of claim 1 , wherein the display device includes an array of light-emitting elements that displays the image in the display regions based on the display control signal and another array of light-emitting elements that displays the RVS in the display regions based on the separate and independent RVS signals.

12. A brain-computer interface system, comprising:
a video processor configured to produce a display control signal;
a temporal controller configured to producing a plurality of repetitive visual stimulus (RVS) signals each with a different respective temporal aspect, wherein the temporal controller is separate and independent from the video processor, wherein the RVS signals with the different respective temporal aspects are not embedded in the display control signal and are separate and independent from the display control signal, and wherein the temporal aspects of the RVS signals include frequencies, amplitudes, or phases of the RVS signals, other temporal characteristics of the RVS signals, or a combination thereof;
a display device for displaying an image and having a display module and a backlight, wherein the display module includes a plurality of display regions, each of the plurality of display regions including a plurality of light emitting elements, the display module being configured to receive both the display control signal and the RVS signals, the display control signal controls each of the plurality of light emitting elements and the RVS signals are mapped to and control each of the plurality of the display regions so that the RVS displayed by each respective one of the display regions have a different respective one of the temporal aspects than the RVS displayed by each other of the display regions, and wherein the frequencies of the RVS signals are at least 30 Hz;
an electroencephalographic (EEG) sensor configured to sense a visually-evoked cortical potential (VECP) signal in a user whose eyes are fixed on a viewed one of the display regions; and
a VECP processor configured to process the VECP signal to identify the viewed one of the display regions, wherein the temporal controller is configured to send a synchronization signal to the VECP processor, wherein the synchronization signal indicates a pulse onset of the RVS signals with the different respective temporal aspects, and in response the VECP processor determines the onset of the RVS for each of the display regions of the display device.

13. The brain-computer interface system of claim 12 , wherein the temporal aspects of the RVS signals are the sufficiently high frequencies that do not produce visually perceptible flicker of the displayed RVS.

14. The brain-computer interface system of claim 13 , wherein the sufficiently high frequencies of the RVS signals are not an even divisor of an update rate of the display device.

15. The brain-computer interface system of claim 13 , wherein the sufficiently high frequencies of the RVS signals are at least one-half an update rate of the display device.

16. The brain-computer interface system of claim 12 , further comprising an electrooculogram (EOG) sensor configured to sense eye movement and/or fixation in the user and sending a corresponding EOG signal to the VECP processor, wherein the VECP processor processes the EOG signal to identify the viewed one of the display regions indicated by the EOG and performs a reconciliation process to confirm that the viewed display region indicated by the EOG signal matches the viewed display region indicated by the EEG signal.

17. The brain-computer interface system of claim 12 , wherein the temporal controller is provided by a plurality of temporal controllers in a d

Assignees

• United States of Americas as represented by the Administrator of NASA
• Nasa

Inventors

• Miller Michael E
• Borghetti Brett J
• Kennedy Kellie D
• Stephens Chad L
• Pope Alan T

Classifications

• G02B27/017

  Head mounted · CPC title

• G06F3/013 Primary

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• G09G5/005

  Adapting incoming signals to the display format of the display terminal · CPC title

• G09G2340/0435

  Change or adaptation of the frame rate of the video stream · CPC title

• G06F3/015 Primary

  Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection · CPC title