Generating in a gaze tracking device augmented reality representations for objects in a user line-of-sight

Patent metadata

Field	Value
Publication number	US-11106915-B1
Application number	US-202016831679-A
Country	US
Kind code	B1
Filing date	Mar 26, 2020
Priority date	Mar 26, 2020
Publication date	Aug 31, 2021
Grant date	Aug 31, 2021

Abstract

Provided are a computer program product, system, and method for generating in a gaze tracking device augmented reality representations for objects in a user line-of-sight. A determination is made of user coordinates of a user position and coordinates of an axis of a line-of-sight of the user from the gaze tracking device. Requested object information is received indicating objects along the axis of the line-of-sight at different distances from the user position, including a first object in the line-of-sight that is visible to the user at a first distance from the user position and at least one additional object. Each of the at least one additional object is at a further distance from the user position than the first distance. Descriptions of the objects are rendered to the user in a view of the gaze tracking device along the axis of the line-of-sight.

First claim

What is claimed is:

1. A computer program product for providing information on objects to render on a user operated gaze tracking device, the computer program product comprising a computer readable storage medium having computer readable program code embodied therein that executes to perform operations, the operations comprising:
determining user coordinates of a user position and coordinates of an axis of a line-of-sight of the user from the gaze tracking device;
requesting object information based on the user position and the coordinates of the axis of the line-of-sight from a map database to cause a query at the map database to determine object information for objects along the axis of the line-of-sight at determined distances from the user position;
receiving the object information indicating objects along the axis of the line-of-sight at different distances from the user position, including a first object in the line-of-sight that is visible to the user at a first distance from the user position and at least one additional object, wherein each of the at least one additional object is at a further distance from the user position than the first distance, wherein the object information includes descriptions of the objects, and wherein an obstructed object along the axis of the line-of-sight for which object information is received is not visible to the user from the user position on the axis of the line-of-sight as a result of being obstructed; and
rendering the descriptions of the objects to the user in a view of the gaze tracking device along the axis of the line-of-sight including a description for the obstructed object.

2. The computer program product of claim 1 , wherein the rendering the descriptions of the objects comprises rendering augmented reality representations of the descriptions of the objects in the view of the gaze tracking device displayed along the axis of the line-of-sight.

3. The computer program product of claim 2 , wherein the rendering the augmented reality representations of the objects comprises:
rendering an augmented reality representation for each object of the objects to appear at a depth in a perspective view of the user along the axis of the line-of-sight corresponding to a distance of the object from the user position.

4. The computer program product of claim 2 , wherein the operations further comprise:
receiving user selection of an augmented reality representation for an object following a first augmented reality representation for the first object along the axis of the line-of-sight; and
rendering the selected augmented reality representation in the gaze tracking device to appear ahead of the first augmented reality representation along the axis of the line-of-sight in response to receiving the user selection.

5. The computer program product of claim 4 , wherein the rendering the selected augmented reality representation further comprises rendering the selected augmented reality representation with larger dimensions than the first augmented reality representation for the first object and rendering the selected augmented reality representation with additional content from a description of the object associated with the selected augmented reality representation in the object information.

6. The computer program product of claim 5 , wherein before receiving user selection of the augmented reality representation, performing:
rendering the first augmented reality representation larger than the augmented reality representations for additional objects following the first object along the axis of the line-of-sight; and
rendering each augmented reality representation, for each additional object following the first object along the axis of the line-of-sight, smaller in appearance than the augmented reality representation for a preceding object along the axis of the line-of-sight closer to the user position.

7. The computer program product of claim 2 , wherein the operations further comprise:
receiving user input commands to scroll to a selected one of the augmented reality representations along the axis of the line-of-sight other than the first object; and
rendering the augmented reality representation of the selected augmented reality representation to appear first in the axis of the line-of-sight to provide a visual effect to the user that the selected augmented reality representation is moving forward toward the user and a first augmented reality representation for the first object is moving away from the user.

8. The computer program product of claim 2 , wherein the object information indicates a plurality of objects on an axis orthogonal to the axis of the line-of-sight further away from the user position than the first distance, wherein the operations further comprise:
rendering a sequential display of augmented reality representations of the objects along the axis orthogonal to the axis of the line-of-sight;
receiving user selection of one of the augmented reality representations along the axis orthogonal to the axis of the line-of-sight; and
rendering the selected augmented reality representation along the axis orthogonal to the axis of the line-of-sight to appear in front of other of the augmented reality representations along the axis orthogonal to the axis of the line-of-sight.

9. The computer program product of claim 2 , wherein the object information includes an image of an object for each of the objects indicated in the object information, wherein the operations further comprise:
detecting eye strain for the user along the axis of the line-of-sight; and
in response to detecting the eye strain, performing:
determining an object the user is viewing in the axis of the line-of-sight;
determining the image for the object the user is viewing in the object information; and
superimposing an augmented reality representation using the image for the object the user is viewing over the object the user is viewing to enhance a user view of the object the user is viewing.

10. The computer program product of claim 1 , wherein the operations further comprise:
gathering biometric data from a user via a biometric gathering device coupled to the user;
determining, from the gathered biometric data, an activity state of the user; and
determining objects along the axis of the line-of-sight within a predetermined distance from the user that have a description in the object information associated with the determined activity state of the user, wherein augmented reality representations are rendered for objects associated with the determined activity state of the user.

11. A system for providing information on objects to render on a user operated gaze tracking device, comprising:
a processor; and
a computer readable storage medium having computer readable program code embodied therein that executes to perform operations, the operations comprising:
determining user coordinates of a user position and coordinates of an axis of a line-of-sight of the user from the gaze tracking device;
requesting object information based on the user position and the coordinates of the axis of the line-of-sight from a map database to cause a query at the map database to determine object information for objects along the axis of the line-of-sight at determined distances from the user position;
receiving the object information indicating objects along the axis of the line-of-sight at different distances from the user position, including a first object in the line-of-sight that is visible to the user at a first distance from the user position and at least one additional object, wherein each of the at least one additional object is at a further distance from the user position than

Assignees

• IBM

Inventors

• Nagar Raghuveer Prasad
• Rakshit Sarbajit K
• Jose Reji
• Prasad Kavita

Classifications

• G06V20/20 Primary

  in augmented reality scenes · CPC title

• G06V40/18

  Eye characteristics, e.g. of the iris · CPC title

• G06F16/587

  using geographical or spatial information, e.g. location · CPC title

• G06F16/29

  Geographical information databases · CPC title

• G06F3/013

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title