Generation of graphical feedback in a computer system

Patent metadata

Field	Value
Publication number	US-10452140-B2
Application number	US-201816216788-A
Country	US
Kind code	B2
Filing date	Dec 11, 2018
Priority date	Feb 1, 2006
Publication date	Oct 22, 2019
Grant date	Oct 22, 2019

Abstract

The present invention relates to control of a computer system, which includes a data processing unit, a display and an eye tracker adapted to register a user's gaze point with respect to the display. The data processing unit is adapted to present graphical information on the display, which includes feedback data reflecting the user's commands entered into the unit. The data processing unit is adapted to present the feedback data such that during an initial phase, the feedback data is generated based on an absolute position of the gaze point. An imaging device of the system is also adapted to register image data representing movements of a body part of the user and to forward a representation of the image data to the data processing unit. Hence, during a phase subsequent to the initial phase, the data is instead generated based on the image data.

First claim

What is claimed:

1. A method for operating a computer system, the method comprising:
causing a screen of a display device to display a first image;
receiving image data from a first image sensor;
determining, based on the image data from the first image sensor, that a gaze of a user is directed away from the screen;
receiving an activation input from the user; and
causing, based on the gaze of the user being directed away from the screen and receipt of the activation input, the screen to display a second image, wherein the second image is different than the first image.

2. The method for operating a computer system of claim 1 , wherein receiving the activation input comprises:
receiving the activation input from a mechanically actuated input device.

3. The method for operating a computer system of claim 2 , wherein the activation input is selected from a group consisting of:
movement of a mouse; and
pressing or releasing of a key on a keyboard.

4. The method for operating a computer system of claim 1 , wherein receiving the activation input comprises:
receiving the activation input from a microphone, wherein the activation input comprises a voice command.

5. The method for operating a computer system of claim 1 , wherein receiving the activation input comprises:
determining, based on the image data from the first image sensor or image data from a second image sensor, that the gaze of the user has moved.

6. The method for operating a computer system of claim 1 , wherein determining that the user is gazing away from the screen comprises:
determining that the user is gazing below the screen.

7. The method for operating a computer system of claim 1 , further comprising:
determining, based on the image data from the first image sensor or image data from a second image sensor, that the gaze of the user is directed at the screen; and
causing, based on the gaze of the user being directed at the screen, the screen to display the first image.

8. The method for operating a computer system of claim 1 , further comprising:
determining, based on the image data from the first image sensor or image data from a second image sensor, that the gaze of the user is directed at the screen; and
causing, based on the gaze of the user being directed at the screen, the screen to display a third image, wherein:
the third image is different than the first image; and
the third image is different than the second image.

9. The method for operating a computer system of claim 1 , further comprising:
receiving image data from a second image sensor;
determining, based on the image data from the second image sensor, a position of a body part of the user; and
causing, based on the position of the body part of the user, the screen to display a third image.

10. The method for operating a computer system of claim 9 , wherein the body part is selected from a group consisting of:
at least one eye of the user; and
at least one limb of the user.

11. A computer system, wherein the computer system comprises:
a display device comprising a screen;
a first image sensor; and
a processor, wherein the processor is configured to:
cause the screen to display a first image;
receive image data from the first image sensor;
determine, based on the image data from the first image sensor, that a gaze of a user is directed away from the screen;
receive an activation input from the user; and
cause, based on the gaze of the user being directed away from the screen and receipt of the activation input, the screen to display a second image, wherein the second image is different than the first image.

12. The computer system of claim 11 , wherein determining that the user is gazing away from the screen comprises:
determining that the user is gazing below the screen.

13. The computer system of claim 11 , wherein the processor is further configured to:
determine, based on the image data from the first image sensor or image data from a second image sensor, that the gaze of the user is directed at the screen; and
cause, based on the gaze of the user being directed at the screen, the screen to display the first image.

14. The computer system of claim 11 , wherein the processor is further configured to:
determine, based on the image data from the first image sensor or image data from a second image sensor, that the gaze of the user is directed at the screen; and
cause, based on the gaze of the user being directed at the screen, the screen to display a third image, wherein:
the third image is different than the first image; and
the third image is different than the second image.

15. The computer system of claim 11 , wherein the processor is further configured to:
receive image data from a second image sensor;
determine, based on the image data from the second image sensor, a position of a body part of the user; and
cause, based on the position of the body part of the user, the screen to display a third image.

16. A non-transitory computer readable medium having instructions stored thereon, wherein the instructions, when executed, cause at least one processor to perform operations comprising:
cause a screen of a display device to display a first image;
receive image data from a first image sensor;
determine, based on the image data from the first image sensor, that a gaze of a user is directed away from the screen;
receive an activation input from the user; and
cause, based on the gaze of the user being directed away from the screen and receipt of the activation input, the screen to display a second image, wherein the second image is different than the first image.

17. The non-transitory computer readable medium of claim 16 , wherein determining that the user is gazing away from the screen comprises:
determining that the user is gazing below the screen.

18. The non-transitory computer readable medium of claim 16 , wherein the operations further comprise:
determining, based on the image data from the first image sensor or image data from a second image sensor, that the gaze of the user is directed at the screen; and
causing, based on the gaze of the user being directed at the screen, the screen to display the first image.

19. The non-transitory computer readable medium of claim 16 , wherein the operations further comprise:
determining, based on the image data from the first image sensor or image data from a second image sensor, that the gaze of the user is directed at the screen; and
cause, based on the gaze of the user being directed at the screen, the screen to display a third image, wherein:
the third image is different than the first image; and
the third image is different than the second image.

20. The non-transitory computer readable medium of claim 16 , wherein the operations further comprise:
receiving image data from a second image sensor;
determining, based on the image data from the second image sensor, a position of a body part of the user; and
causing, based on the position of the body part of the user, the screen to display a third image.

Assignees

• Tobii Ab

Inventors

• Elvesjö John
• Olsson Anders
• Sahlén Johan

Classifications

• G06F3/167

  Audio in a user interface, e.g. using voice commands for navigating, audio feedback · CPC title

• G06F3/013 Primary

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• G06F3/017

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G09G5/08

  Cursor circuits · CPC title

• G06F3/0481

  based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance · CPC title