Gaze direction tracking

Patent metadata

Field	Value
Publication number	US-9740282-B1
Application number	US-201514589422-A
Country	US
Kind code	B1
Filing date	Jan 5, 2015
Priority date	Jan 5, 2015
Publication date	Aug 22, 2017
Grant date	Aug 22, 2017

Abstract

Various examples are directed to systems and methods that measure the gaze direction of a user. For example, gaze direction may be utilized to generate an eye-equivalent representation for the user, where the eye-equivalent representation depicts eyes directed according to the gaze direction.

First claim

What is claimed is:

1. A computer implemented virtual reality system, the system comprising:
a body portion;
a gaze detector positioned to sense a gaze direction of a user, wherein the gaze direction indicates where the user is looking;
an inward-directed display positioned in the body portion to generate an inward display output directed towards and visible to the user;
an outward-directed display positioned on the body portion over the eyes of the user to generate an outward display visible to people near the user; and
a computing device in communication with the inward-directed display, the outward-directed display and the gaze detector, wherein the computing device comprises at least one processor and operatively associated memory, and wherein the computing device is programmed to:
receive the gaze direction of the user from the gaze detector;
cause an imaging device to capture an image of the eyes of the user;
generate an eye-equivalent image for the user, wherein the eye-equivalent image comprises the captured image of the eyes of the user captured by the image capture device and directed according to the gaze direction; and
display the eye-equivalent image at the outward-directed display.

2. The system of claim 1 , wherein generating the eye-equivalent image comprises capturing the image of the eyes of the user at about the time that the gaze direction is sensed by the gaze detector.

3. The system of claim 1 , wherein generating the eye-equivalent image comprises selecting a previously captured image of the eyes of the user corresponding to the gaze direction.

4. The system of claim 1 , further comprising a second outward-directed display, wherein the computing device is further programmed to:
identify a first object in a virtual reality environment of the user, wherein the first object is positioned in the virtual reality environment along the gaze direction relative to a position of an avatar of the user in the virtual reality environment;
determine that the first object is sharable; and
display a representation of the first object on the second outward-directed display.

5. A computer implemented virtual reality system, the system comprising:
a body portion;
a gaze detector positioned to sense a gaze direction of a user;
an outward-directed display positioned on the body portion at least partially over the eyes of the user to generate an outward display visible to people near the user;
an inward-directed display positioned in the body portion to generate an inward display output visible to the user; and
a computing device in communication with the outward-directed display and the gaze detector, wherein the computing device comprises at least one processor and operatively associated memory, and wherein the computing device is programmed to:
receive the gaze direction of the user from the gaze detector;
cause an image capture device to capture an image of the eyes of the user;
generate an eye-equivalent representation for the user, wherein the eye-equivalent representation comprises the captured image of the eyes of the user captured by the image capture device and directed according to the gaze direction; and
display the eye-equivalent representation at the outward-directed display.

6. The system of claim 5 , wherein generating the eye-equivalent representation comprises causing the image capture device to capture the image of the eyes of the user at about the time that the gaze direction is sensed by the gaze detector.

7. The system of claim 5 , wherein generating the eye-equivalent representation comprises selecting a previously captured image showing the user's eyes directed in the gaze direction.

8. The system of claim 5 , further comprising a second outward-directed display, wherein the computing device is further programmed to:
identify a first object in a virtual reality environment of the user, wherein the first object is positioned in the virtual reality environment along the gaze direction relative to a position of the user in the virtual reality environment; and
display a representation of the first object on the second outward-directed display.

9. The system of claim 8 , wherein the computing device is further programmed to:
from the gaze direction, determine a focus area of the first object; and
transmit the focus area of the first object to at least a second computing device.

10. The system of claim 5 , wherein the computing device is further programmed to:
receive from a second computing device an indication of an object and an indication of a focus region of the object;
display the object on the inward-directed display; and
highlight the focus region of the object on the inward-directed display.

11. A computer implemented system, the system comprising:
a gaze detector positioned to sense a gaze direction of a user, wherein the gaze direction indicates where the user is looking;
a computing device in communication with the gaze detector, wherein the computing device comprises at least one processor and operatively associated memory, and wherein the computing device is programmed to:
receive the gaze direction of the user from the gaze detector;
render an avatar of the user, wherein a gaze direction of the avatar corresponds to the gaze direction of the user;
identify a first object in a first virtual reality environment based at least in part on the first object being positioned along the gaze direction relative to the avatar of the user; and
determine a location of the first object in a second virtual reality environment, wherein the location of the first object in the second virtual reality environment is different than a location of the first object in the first virtual reality environment, and wherein rendering the avatar comprises rendering the avatar in the second virtual reality environment, and wherein the gaze direction of the avatar in the second virtual reality environment is directed to the location of the first object in the second virtual reality environment.

12. The system of claim 11 , wherein the computing device is further programmed to:
identify a line beginning at the avatar and directed in the gaze direction; and
determine that the first object intersects the line.

13. The system of claim 11 , wherein the computing device is further programmed to determine that the first object is not present in a second virtual reality environment, wherein rendering the avatar comprises rendering the avatar in the second virtual reality environment, and wherein the gaze direction of the avatar is directed away from other objects in the second virtual reality environment.

14. The system of claim 11 , wherein the gaze direction of the avatar is indicated by eyes of the avatar.

15. The system of claim 11 , wherein the gaze direction of the avatar is indicated by a directed object pointing in the gaze direction.

16. The system of claim 11 , wherein the computing device is programmed to render the avatar of the user in a virtual reality environment.

17. A computer implemented system for generating a representation of a panoramic environment having a panoramic field-of-view, the system comprising:
a computing device comprising at least one processor and operatively associated memory, wherein the computing device is programmed to:
receive a first gaze direction of a user;
determine a first user field-of-view of the user based on the gaze direction, wherein the user field-of-view is at least a first portion of the panoramic field-of-view;
determine a first focus area of the first user field-of-view;

Assignees

• Amazon Tech Inc

Inventors

• Mcinerny Michael James

Classifications

• G02B27/0093

  with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking · CPC title

• H04N7/147

  Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals (selecting H04Q) · CPC title

• H04N7/157

  defining a virtual conference space and using avatars or agents (computer conference optimisation or adaptation H04L12/1827) · CPC title

• G06F3/013 Primary

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• G02B27/017 Primary

  Head mounted · CPC title