Gaze-driven augmented reality

Patent metadata

Field	Value
Publication number	US-2016259977-A1
Application number	US-201415028233-A
Country	US
Kind code	A1
Filing date	Oct 10, 2014
Priority date	Oct 11, 2013
Publication date	Sep 8, 2016
Grant date	—

Abstract

Augmented reality (AR) systems, methods, and instrumentalities are disclosed. A user's gaze point may be estimated and may be used to search for and present information, e.g., information relating to areas on which the user is focusing. The user's gaze point may be used to facilitate or enable modes of interactivity and/or user interfaces that may be controlled by the direction of view of the user. Biometric techniques may be used to estimate an emotional state of the user. This estimated emotional state may be used to be the information that is presented to the user.

First claim

What is claimed is:

1 . A method for providing augmented reality (AR) to a user, comprising:
estimating a gaze point of the user to define a region of interest (ROI); estimating an emotional state of the user; sending information associated with the ROI and the emotional state of the user to a server; receiving information regarding one or more objects in the ROI from the server; and displaying the information regarding the one or more objects in the ROI.

2 . The method of claim 1 , further comprising refining or filtering the information about the objects in the ROI based on the emotional state of the user.

3 . The method of claim 1 , wherein a size of the ROI is adjusted based upon a number of objects detected.

4 . The method of claim 1 , wherein a size of the ROI is based upon a user activity.

5 . The method of claim 1 , wherein the emotional state of the user is determined by estimating a point for the emotional state on a valence/arousal chart.

6 . The method of claim 1 , further comprising using voice recognition of the user to determine the emotional state.

7 . The method of claim 1 , further comprising using facial recognition of the user to determine the emotional state.

8 . The method of claim 1 , further comprising using at least one of a galvanic skin response (GSR) or electroencephalographic (EEG) data of the user to determine the emotional state.

9 . The method of claim 1 , further comprising extracting images from the ROI and creating a set of descriptors for features of the objects.

10 . The method of claim 9 , wherein the descriptors are used to search for information about the objects in the ROI.

11 . The method of claim 9 , wherein the descriptors include a determined location.

12 . The method of claim 1 , further comprising storing the information associated with the ROI and the emotional state.

13 . A device, comprising:
a processor adapted to: estimate a gaze point of the user to define a region of interest (ROI); estimate an emotional state of the user; send information associated with the ROI and the emotional state of the user to a server; receive information regarding one or more objects in the ROI from the server; and display the information regarding the one or more objects in the ROI.

14 . The device of claim 13 , wherein the information about the objects in the ROI are refined or filtered based on the emotional state of the user.

15 . The device of claim 13 , wherein a size of the ROI is adjusted based upon a number of objects detected.

16 . The device of claim 13 , wherein a size of the ROI is based upon a user activity.

17 . The device of claim 13 , wherein the emotional state of the user is determined by estimating a point for the emotional state on a valence/arousal chart.

18 . The device of claim 13 , wherein the emotional state is estimated using at least one of voice recognition of the user, facial recognition of the user, galvanic skin response (GSR) of the user, or electroencephalographic (EEG) data of the user.

19 . The device of claim 13 , wherein images are extracted from the ROI, and a set of descriptors for features of the objects is created.

20 . The device of claim 13 , wherein the information associated with the ROI and the emotional state is stored.

21 . A device, comprising:
a processor adapted to:
estimate a device user's emotional state;
send information to a server about the user and one or more objects in a region of interest (ROI); and
receive refined or filtered information about the objects in the ROI based upon an estimate from the server of the emotional state of the user.

Assignees

• Interdigital Patent Holdings Inc

Inventors

• Asbun Eduardo
• Reznik Yuriy
• Zeira Ariela
• Sternberg Gregory S
• Neff Ralph

Classifications

• G02B27/017

  Head mounted · CPC title

• A61B5/742

  using visual displays (displays for heart-related electrical signals, e.g. ECG, A61B5/339) · CPC title

• A61B5/0022

  Monitoring a patient using a global network, e.g. telephone networks, internet · CPC title

• G06F3/013 Primary

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• H04W4/30

  Services specially adapted for particular environments, situations or purposes · CPC title