Gaze and saccade based graphical manipulation

Patent metadata

Field	Value
Publication number	US-11619989-B2
Application number	US-201815864858-A
Country	US
Kind code	B2
Filing date	Jan 8, 2018
Priority date	Mar 4, 2013
Publication date	Apr 4, 2023
Grant date	Apr 4, 2023

Abstract

According to the invention, a system for presenting graphics on a display device is disclosed. The system may include an eye tracking device and a graphics processing device. The eye tracking device may be for determining a gaze point of a user on a display device. The graphics processing device may be for causing graphics to be displayed on the display device. The graphics displayed on the display device may be modified such that the graphics are of higher quality in an area including the gaze point of the user, than outside the area.

First claim

What is claimed is:

1. A system comprising:
an eye tracking device; and
one or more processors configured to:
display an image on a display device;
determine a gaze direction of a user based on data from the eye tracking device;
determine an initial gaze point of the user on the display device based on the data from the eye tracking device;
modify the image on the display device based on the initial gaze point to result in a first image that is then displayed on the display device, wherein modifying the image comprises altering an image quality of a first area of the image based on a distance from the first area to the initial gaze point, and wherein altering the image quality of the first area includes continuously decreasing the image quality from the initial gaze point toa boundary of the first area;
detect a user saccade and a velocity of the user saccade;
determine an anticipated gaze point of the user on the display device wherein the anticipated gaze point is based on a velocity of the user saccade; and
modify a second area of the first image that includes the anticipated gaze point to result in a second image that is then displayed on the display device, wherein modifying the second area comprises altering an image quality of the second area of the first image, and wherein a portion of or whole of the second image displayed on the display device is different than the first image displayed on the display device.

2. The system of claim 1 , wherein altering the image quality of the first area of the image based on the distance from the first area to the initial gaze point comprises:
linearly decreasing the image quality of the first area from the initial gaze point to the boundary of the first area linearly as the distance from the area to the gaze point increases.

3. The system of claim 1 , wherein altering the image quality of the first area of the image based on the distance from the first area to the initial gaze point comprises:
non-linearly decreasing the image quality of the first area from the initial gaze point to the boundary of the first area in a non linear fashion as the distance from the area to the gaze point increases.

4. The system of claim 1 , wherein, after the detection of the user saccade, the one or more processors are configured to further alter the image quality of the first area of the image by:
increasing the image quality of the first area relative to an image quality of a remainder of the image when the first area comprises the anticipated gaze point.

5. The system of claim 1 , wherein, after the detection of the user saccade, the one or more processors are configured to further alter the image quality of the first area of the image based on the distance from the first area to the anticipated gaze point by:
decreasing the image quality of the first area from the anticipated gaze point to a region of the first area that is distal from the anticipated gaze point as the distance from the first area to the anticipated gaze point increases.

6. The system of claim 5 , wherein modifying the first image based on the initial gaze point comprises:
increasing the image quality at the anticipated gaze point.

7. The system of claim 1 , wherein the second area comprises the initial gaze point and the anticipated gaze point.

8. The system of claim 1 , wherein the second area includes separate and distinct areas around the initial gaze point and the anticipated gaze point.

9. The system of claim 1 , wherein the anticipated gaze point is located in the second area outside the first area of the image.

10. The system of claim 1 , wherein the second area is shaped triangular, trapezoidal, circular, oval or square.

11. A method comprising:
displaying an image on a display device;
receiving data from a first eye tracking device relating to a gaze direction of a first user;
determining an initial gaze point of the first user on the display device based on the data from the first eye tracking device;
modifying the image on the display device based on the initial gaze point to result in a first image that is then displayed on the display device, wherein modifying the image comprises altering an image quality of a first area of the image based on a first plurality of distances, wherein a distance from the first plurality of distances is measured between the initial gaze point of the first user to a corresponding location of a plurality of locations within the first area of the image, and wherein altering the image quality of the first area includes continuously decreasing the image quality from the initial gaze point to a boundary of the first area;
detecting a user saccade of the first user and a velocity of the user saccade;
determining an anticipated gaze point of the first user on the display device wherein the anticipated gaze point is based on a velocity of the user saccade; and
modifying a second area of the first image that includes the anticipated gaze point to result in a second image that is then displayed on the display device, wherein modifying the second area comprises altering an image quality of the second area of the first image, and wherein a portion of or whole of the second image displayed on the display device is different than the first image displayed on the display device.

12. The method of claim 11 , further comprising:
receiving data from a second eye tracking device relating to a gaze direction of a second user; and
determining an initial gaze point of a second user on the display device based on the data from the second eye tracking device, wherein modifying the image quality of the image is further based on a second plurality of distances from the initial gaze point of the second user to the plurality of locations.

13. The method of claim 11 , wherein modifying the image quality of the image is further based on:
a distance from the first user to the display device.

14. The method of claim 11 , wherein modifying the image quality of the image comprises:
selecting the first area of the first image comprising the initial gaze point of the first user;
and
changing an image quality of the first area from a higher image quality proximate to the anticipated gaze point of the first user to a lower image quality at a boundary of the first area.

15. The method of claim 14 , wherein the first area comprises a shape selected from a list consisting of:
a circular shape; an oval shape; a square shape; and
a rectangular shape; and a polygonal shape.

16. A non-transitory computer readable medium having instructions stored thereon executable by a computing device to cause the computing device to perform operations comprising:
displaying an image on a display device;
receiving data from an eye tracking device relating to a gaze direction of a first user;
determining an initial gaze point of the first user on the display device image based on the data from the eye tracking device;
modifying the image on the display device based on the initial gaze point to result in a first image that is then displayed on the display device, wherein modifying the image comprises altering an image quality of a first area of the image based on a first plurality of distances, wherein a distance from the first plurality of distances is measured between the initial gaze point of the first user to a corresponding location of a plurality of locations within the first area of the image, and wherein altering the image quality of the first area includes decreasing the image quality continuously from the initial gaze point to a boundary of the first area;
detec

Assignees

• Tobii Ab
• Tobil AB

Inventors

• Thunström Robin
• Skogö Mårten

Classifications

• G06T2210/36

  Level of detail · CPC title

• G06F3/013 Primary

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• G06T19/20

  Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts · CPC title