Human-computer interaction method, human-computer interaction apparatus, and storage medium

Patent metadata

Field	Value
Publication number	US-12449900-B2
Application number	US-202318518194-A
Country	US
Kind code	B2
Filing date	Nov 22, 2023
Priority date	May 24, 2021
Publication date	Oct 21, 2025
Grant date	Oct 21, 2025

Abstract

A human-computer interaction method, a human-computer interaction apparatus, and a storage medium are provided. The method includes: displaying a human-computer interface, where the human-computer interface includes an operable object and a visual stimulus region; and executing a first operational command on the operable object, where the first operational command is determined based on a first signal, the first signal is an electroencephalography signal generated when a user gazes at a first region, the first region is one of a plurality of regions around the visual stimulus region, different electroencephalography signals are generated when the user gazes at different regions of the plurality of regions, and the different electroencephalography signals indicate to execute different operational commands related to the operable object. According to this application, the user does not need to gaze directly at a visual stimulus and can deliver different operational commands quickly, which improves user experience.

First claim

What is claimed is:

1. A human-computer interaction method applied to an electronic device, the method comprising:
displaying a human-computer interface, wherein the human-computer interface comprises an operable object and a visual stimulus region;
executing a first operational command on the operable object, wherein the first operational command is determined based on a first signal, the first signal is an electroencephalography signal generated in response to a user gazing at a first region, the first region is one of a plurality of regions around the visual stimulus region, different electroencephalography signals are generated in response to the user gazing at different regions of the plurality of regions, and the different electroencephalography signals indicate to execute different operational commands related to the operable object; and
wherein an included angle between a line of sight obtained in response to the user gazing at the first region and a line of sight obtained in response to the user gazing at a center point of the visual stimulus region is in a preset range.

2. The method according to claim 1 , wherein the different electroencephalography signals are evoked in different regions in a primary visual cortex in response to the user gazing at the different regions, and the first region is determined based on a region in which the first signal appears in the primary visual cortex.

3. The method according to claim 1 , wherein the visual stimulus region overlaps a display region of the operable object;
the plurality of regions are regions that are distributed in a plurality of directions centering around the visual stimulus region; and
the first region is a region distributed in a first direction, and the first direction is one of the plurality of directions.

4. The method according to claim 3 , wherein the first operational command instructs the operable object to move in the first direction; or
the first operational command instructs to resize the display region of the operable object towards the first direction.

5. The method according to claim 3 , wherein the regions in the plurality of directions comprise at least one auxiliary graphic, and the at least one auxiliary graphic is configured to indicate a position for gazing to the user;
the first region comprises a first auxiliary graphic of the at least one auxiliary graphic; and
the first signal is an electroencephalography signal generated in response to the user gazing at the first auxiliary graphic.

6. The method according to claim 1 , wherein the human-computer interface comprises a plurality of visual stimulus regions, the first signal comprises a plurality of electroencephalography signals that are generated in response to the plurality of visual stimulus regions stimulating vision of the user, and the plurality of electroencephalography signals separately indicate positional relationships between the first region and the plurality of visual stimulus regions.

7. The method according to claim 6 , wherein the first region comprises a first sub-region and a second sub-region, the plurality of electroencephalography signals comprise a first sub-signal and a second sub-signal, the first sub-signal is an electroencephalography signal generated in response to the user gazing at the first sub-region, the second sub-signal is an electroencephalography signal generated in response to the user gazing at the second sub-region, and the first sub-signal precedes the second sub-signal; and
wherein that the first operational command is determined based on the first signal comprises that the first operational command is determined based on both the first sub-signal and the second sub-signal.

8. The method according to claim 6 , wherein the first region is the display region of the operable object, and the executing the first operational command on the operable object comprises: executing a select operational command on the operable object.

9. A human-computer interaction apparatus, wherein the apparatus comprises:
a display, configured to cooperate with a processor to display a human-computer interface, wherein the human-computer interface comprises an operable object and a visual stimulus region;
a processor, configured to execute a first operational command on the operable object, wherein the first operational command is determined based on a first signal, the first signal is an electroencephalography signal generated in response to a user gazing at a first region, the first region is one of a plurality of regions around the visual stimulus region, different electroencephalography signals are generated in response to the user gazing at different regions of the plurality of regions, and the different electroencephalography signals indicate to execute different operational commands related to the operable object; and
wherein an included angle between a line of sight obtained in response to the user gazing at the first region and a line of sight obtained in response to the user gazing at a center point of the visual stimulus region is in a preset range.

10. The apparatus according to claim 9 , wherein the different electroencephalography signals are evoked in different regions in a primary visual cortex in response to the user gazing at the different regions, and the first region is determined based on a region in which the first signal appears in the primary visual cortex.

11. The apparatus according to claim 9 , wherein the visual stimulus region overlaps a display region of the operable object;
the plurality of regions are regions that are distributed in a plurality of directions centering around the visual stimulus region; and
the first region is a region distributed in a first direction, and the first direction is one of the plurality of directions.

12. The apparatus according to claim 11 , wherein the regions in the plurality of directions comprise at least one auxiliary graphic, and the at least one auxiliary graphic is configured to indicate a position for gazing to the user;
the first region comprises a first auxiliary graphic of the at least one auxiliary graphic; and
the first signal is an electroencephalography signal generated in response to the user gazing at the first auxiliary graphic.

13. A human-computer interaction apparatus, comprising a processor and a memory, wherein the memory is configured to store a computer program, and the processor is configured to invoke the computer program to perform a method comprising:
displaying a human-computer interface, wherein the human-computer interface comprises an operable object and a visual stimulus region;
executing a first operational command on the operable object, wherein the first operational command is determined based on a first signal, the first signal is an electroencephalography signal generated in response to a user gazing at a first region, the first region is one of a plurality of regions around the visual stimulus region, different electroencephalography signals are generated in response to the user gazing at different regions of the plurality of regions, and the different electroencephalography signals indicate to execute different operational commands related to the operable object; and
wherein an included angle between a line of sight obtained in response to the user gazing at the first region and a line of sight obtained in response to the user gazing at a center point of the visual stimulus region is in a preset range.

14. The method according to claim 13 , wherein the different electroencephalography signals are evoked in different regions in a primary visual cortex in response to the u

Assignees

• Huawei Tech Co Ltd
• Univ Tianjin

Inventors

• Ming Dong
• Jiang Yongbin
• Xu Minpeng
• Li Pengkun
• Zhou Xiaoyu
• Hu Tao

Classifications

• A61B5/378

  Visual stimuli · CPC title

• G06F3/011

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06F3/013

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• G06F3/015 Primary

  Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection · CPC title