Display of text information on a head-mounted display

Patent metadata

Field	Value
Publication number	US-9984505-B2
Application number	US-201414503196-A
Country	US
Kind code	B2
Filing date	Sep 30, 2014
Priority date	Sep 30, 2014
Publication date	May 29, 2018
Grant date	May 29, 2018

Abstract

A method for presenting text information on a head-mounted display is provided, comprising: rendering a view of a virtual environment to the head-mounted display; tracking an orientation of the head-mounted display; tracking a gaze of a user of the head-mounted display; processing the gaze of the user and the orientation of the head-mounted display, to identify a gaze target in the virtual environment towards which the gaze of the user is directed; receiving text information for rendering on the head-mounted display; presenting the text information in the virtual environment in a vicinity of the gaze target.

First claim

What is claimed is:

1. A method for presenting text information on a head-mounted display, comprising:
rendering a view of a virtual environment to the head-mounted display;
tracking an orientation of the head-mounted display;
tracking a gaze of a user of the head-mounted display;
processing the gaze of the user and the orientation of the head-mounted display, to identify a gaze target in the virtual environment towards which the gaze of the user is directed;
receiving text information for rendering on the head-mounted display;
responsive to receiving the text information, rendering in the virtual environment a virtual device in place of a handheld device held by the user, so that when the gaze of the user defines a real world view direction towards the handheld device then the virtual device is rendered in place of the handheld device in the virtual environment as presented through the head-mounted display, and presenting the text information on the virtual device in the virtual environment in a vicinity of the gaze target, wherein the virtual device is maneuverable in the virtual environment in response to detected movement of the handheld device, such that movement of the virtual device in the virtual environment is tracked to the detected movement of the handheld device.

2. The method of claim 1 ,
wherein the gaze target defines a view depth in the virtual environment, the view depth defined relative to a virtual viewpoint in the virtual environment that is defined for the head-mounted display and that defines a perspective from which the view of the virtual environment is rendered;
wherein the text information is presented in the virtual environment at approximately the view depth.

3. The method of claim 2 , wherein the text information is presented in the virtual environment at a lateral location relative to the gaze target.

4. The method of claim 2 , wherein the text information is presented on an object in the virtual environment that is located at approximately the view depth.

5. The method of claim 1 , wherein the text information is presented at a location defined by the gaze target.

6. The method of claim 1 ,
wherein the gaze target identifies an object in the virtual environment; and
wherein the text information is presented on the object in the virtual environment.

7. The method of claim 1 , wherein determining the orientation of the head-mounted display includes capturing images of the head-mounted display and analyzing the captured images of the head-mounted display.

8. The method of claim 1 , wherein determining the orientation of the head-mounted display includes processing data received from at least one inertial sensor included in the head-mounted display.

9. The method of claim 1 , wherein tracking the gaze of the user includes tracking an orientation of an eye of the user.

10. The method of claim 1 , further comprising:
tracking a location of the head-mounted display;
wherein processing to identify the gaze target in the virtual environment includes processing the location of the head-mounted display.

11. A method for displaying text on a head-mounted display, comprising:
rendering a view of a virtual environment on a head-mounted display;
receiving text information for display to a user of the head-mounted display;
tracking a location and orientation of a controller device in a vicinity of the head-mounted display;
in response to the receiving text information, rendering, in the view of the virtual environment, a virtual display device in place of the controller device held by the user, so that when a gaze of the user defines a real world view direction towards the controller device then the virtual device is rendered in place of the controller device in the virtual environment as presented through the head-mounted display, the virtual display device configured to display the text information, and a location and orientation of the virtual display device in the virtual environment being controlled based on the tracked location and orientation of the controller device.

12. The method of claim 11 , further comprising:
receiving input from a touch-sensitive surface of the controller device;
processing the input to define an interaction with the text information as it is displayed on the virtual display device.

13. The method of claim 12 , wherein the interaction with the text information is selected from scrolling the text information, moving the text information, adjusting a level of zoom, and selecting a portion of the text information.

14. The method of claim 11 , wherein the location and orientation of the virtual display device in the virtual environment are controlled so as to be tracked to the tracked location and orientation of the controller device.

15. A method for displaying text on a head-mounted display, comprising:
rendering a view of a virtual environment to a head-mounted display;
receiving text information for display on the head-mounted display;
in response to the receiving text information, rendering an alert to the head-mounted display;
detecting a predefined response to the alert;
in response to detecting the predefined response to the alert, rendering in the virtual environment a virtual device in place of a controller device held by the user, so that when a gaze of the user defines a real world view direction towards the controller device then the virtual device is rendered in place of the controller device in the virtual environment as presented through the head-mounted display, and rendering the text information on the virtual device, wherein the virtual device is maneuverable in the virtual environment in response to detected movement of the controller device, such that movement of the virtual device in the virtual environment is tracked to the detected movement of the controller device.

16. The method of claim 15 ,
wherein the alert is defined by display of a visual indicator on the head-mounted display;
wherein the predefined response is defined by detection of a view direction of a user of the head-mounted display that is towards the visual indicator.

17. The method of claim 16 , wherein detection of the view direction of the user of the head-mounted display includes determining a gaze direction of the user's eyes.

18. The method of claim 15 ,
wherein the predefined response is defined by detection of a movement of the controller device towards the head-mounted display.

19. The method of claim 15 , wherein in response to gesture input detected from a touch-sensitive surface of the controller device, an interaction with the rendered text information on the virtual display device is defined.

20. A head-mounted display configured to present text information, comprising:
a receiver configured to receive video data;
at least one display configured to render the video data, the rendered video data defining a view of a virtual environment;
at least one device for tracking an orientation of the head-mounted display;
at least one device for tracking a gaze of a user of the head-mounted display, the gaze of the user and the orientation of the head-mounted display being processed to identify a gaze target in the virtual environment towards which the gaze of the user is directed;
wherein rendering the video data defines, responsive to receiving the text information, rendering in the virtual environment a virtual device in place of a handheld device held by the user, so that when the gaze of the user defines a real world view direction towards

Assignees

• Sony Interactive Entertainment Inc

Inventors

• Rimon Noam
• Stafford Jeffrey Roger

Classifications

• G06Q10/40

  Business processes related to social networking or social networking services · CPC title

• G06T11/60 Primary

  Creating or editing images; Combining images with text · CPC title

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06F3/017

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06F3/012

  Head tracking input arrangements · CPC title