Periodic motion-based visual stimulus

Patent metadata

Field	Value
Publication number	US-11847261-B2
Application number	US-202217815532-A
Country	US
Kind code	B2
Filing date	Jul 27, 2022
Priority date	Feb 3, 2020
Publication date	Dec 19, 2023
Grant date	Dec 19, 2023

Abstract

A computer device is provided that includes a display device, and a sensor system configured to be mounted adjacent to a user's head and to measure an electrical potential near one or more electrodes of the sensor system. The computer device further includes a processor configured to present a periodic motion-based visual stimulus having a changing motion that is frequency-modulated for a target frequency or code-modulated for a target code, detect changes in the electrical potential via the one or more electrodes, identify a corresponding visual evoked potential feature in the detected changes in electrical potential that corresponds to the periodic motion-based visual stimulus, and recognize a user input to the computing device based on identifying the corresponding visual evoked potential feature.

First claim

The invention claimed is:

1. A computer device comprising:
a display device;
a sensor system configured to be mounted adjacent to a user's head and to measure an electrical potential near one or more electrodes of the sensor system; and
a processor configured to:
present a periodic motion-based visual stimulus having a changing motion that is frequency-modulated for a target frequency, wherein:
the periodic motion-based visual stimulus is frequency-modulated to change at the target frequency; and
the target frequency of the periodic motion-based visual stimulus includes a fundamental frequency and one or more harmonic frequencies;

detect changes in the electrical potential via the one or more electrodes;
identify a corresponding visual evoked potential feature in the detected changes in the electrical potential that corresponds to the periodic motion-based visual stimulus, wherein to identify the corresponding visual evoked potential feature, the processor is configured to identify a peak in a frequency domain of the detected changes in electrical potential at a corresponding frequency that corresponds to the target frequency of the periodic motion-based visual stimulus;
determine that the user is attending to the periodic motion-based visual stimulus based on a magnitude of the corresponding visual evoked potential feature; and
recognize a user input to the computer device based on determining that the user is attending to the periodic motion-based visual stimulus.

2. The computer device of claim 1 , wherein the processor is configured to recognize the user input based on identifying peaks at corresponding frequencies that correspond to the fundamental frequency and the one or more upper harmonic frequencies of the periodic motion-based visual stimulus.

3. The computer device of claim 1 , wherein the periodic motion-based visual stimulus includes a periodic motion selected from the group consisting of a rotational motion, an oscillating motion, and a changing grating pattern.

4. The computer device of claim 1 , wherein the periodic motion-based visual stimulus is a rapid serial visual presentation of images.

5. The computer device of claim 1 , wherein the display device is a near-eye display device, and wherein the periodic motion-based visual stimulus is an animated three-dimensional virtual object presented by the near-eye display device.

6. The computer device of claim 1 , wherein the magnitude is a magnitude in the frequency domain of the electrical potential.

7. The computer device of claim 1 , wherein the processor is further configured to present a plurality of periodic motion-based visual stimuli having different target frequencies, each periodic motion-based visual stimulus being associated with respective interface elements of a plurality of interface elements.

8. The computer device of claim 7 , wherein the processor is further configured to:
determine a user attended periodic motion-based visual stimulus from among the plurality of periodic motion-based visual stimuli; and
recognize the user input to be directed to an interface element associated with the user attended periodic motion-based visual stimulus.

9. The computer device of claim 8 , wherein the processor is further configured to select, as the user attended periodic motion-based visual stimulus, a periodic motion-based visual stimulus of the plurality of periodic motion-based visual stimuli that has a highest-magnitude visual evoked potential feature among the respective visual evoked potential features of the periodic motion-based visual stimuli.

10. The computer device of claim 1 , wherein the computer device is a head-mounted display device.

11. A method comprising:
at a computer device having a processor and a sensor system configured to be mounted adjacent to a user's head and to measure an electrical potential near one or more electrodes of the sensor system:
presenting a periodic motion-based visual stimulus having a changing motion that is frequency-modulated for a target frequency, wherein:
the periodic motion-based visual stimulus is frequency-modulated to change at the target frequency; and
the target frequency of the periodic motion-based visual stimulus includes a fundamental frequency and one or more harmonic frequencies;

detecting changes in the electrical potential via the one or more electrodes;
identifying a corresponding visual evoked potential feature in the detected changes in the electrical potential that corresponds to the periodic motion-based visual stimulus, wherein identifying the corresponding visual evoked potential feature includes identifying a peak in a frequency domain of the detected changes in electrical potential at a corresponding frequency that corresponds to the target frequency of the periodic motion-based visual stimulus;
determining that the user is attending to the periodic motion-based visual stimulus based on a magnitude of the corresponding visual evoked potential feature; and
recognizing a user input to the computer device based on determining that the user is attending to the periodic motion-based visual stimulus.

12. The method of claim 11 , further comprising presenting a plurality of periodic motion-based visual stimuli having different target frequencies, each periodic motion-based visual stimulus being associated with respective interface elements of a plurality of interface elements.

13. The method of claim 12 , further comprising:
determining a user attended periodic motion-based visual stimulus from among the plurality of periodic motion-based visual stimuli; and
recognizing the user input to be directed to an interface element associated with the user attended periodic motion-based visual stimulus.

14. The method of claim 12 , wherein determining the user attended periodic motion-based visual stimulus includes selecting, as the user attended periodic motion-based visual stimulus, a periodic motion-based visual stimulus of the plurality of periodic motion-based visual stimuli that has a highest-magnitude visual evoked potential feature among the respective visual evoked potential features of the periodic motion-based visual stimuli.

15. A computer device comprising:
a display device;
a sensor system configured to be mounted adjacent a user's head and to measure an electrical potential near one or more electrodes of the sensor system; and
a processor configured to:
present a plurality of interface elements via the display device;
present a plurality of periodic motion-based visual stimuli, each periodic motion-based visual stimulus being associated with respective interface elements of the plurality of interface elements, wherein:
the periodic motion-based visual stimuli are frequency-modulated to change at respective target frequencies; and
the target frequency of each of the periodic motion-based visual stimuli includes a fundamental frequency and one or more harmonic frequencies;

detect changes in the electrical potential via the one or more electrodes;
determine that the user is attending to a user attended periodic motion-based visual stimulus from among the plurality of periodic motion-based visual stimuli at least in part by:
identifying a plurality of visual evoked potential features that correspond to the plurality of periodic motion-based visual stimuli in the detected changes in the electrical potential, wherein to identify each of the visual evoked potential features, the processor is configured to identify a respective peak in a frequency domain of the detected changes in electrical potential at a corresponding frequency that corresponds t

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Wilson Andrew D
• Si Mohammed Hakim
• Holz Christian
• Lee Adrian Kuo Ching
• Tashev Ivan Jelev
• Gamper Hannes
• Cutrell Edward Bryan
• Johnston David Emerson
• Emmanouilidou Dimitra
• Jalobeanu Mihai R

Classifications

• G06F3/015 Primary

  Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection · CPC title

• A61B5/378

  Visual stimuli · CPC title

• G01R19/0084

  Measuring voltage only · CPC title

• G02B27/0176

  characterised by mechanical features · CPC title

• G06F1/163

  Wearable computers, e.g. on a belt · CPC title