Systems and methods for transitioning between transparent mode and non-transparent mode in a head mounted display

Patent metadata

Field	Value
Publication number	US-9908048-B2
Application number	US-201414254881-A
Country	US
Kind code	B2
Filing date	Apr 16, 2014
Priority date	Jun 8, 2013
Publication date	Mar 6, 2018
Grant date	Mar 6, 2018

Abstract

Systems and methods for executing a game presented on a screen of a head mounted display include executing a game. The execution of the game renders interactive scenes of the game on the screen of the HMD. Images identifying a shift in gaze direction of the user wearing the HMD, are received. The gaze shift is detected during viewing of the interactive scenes presented on the HMD screen. Real-world images that are in line with the gaze direction of the user, are captured from a forward-facing camera of the HMD. A portion of the screen is transitioned from a non-transparent mode to a semi-transparent mode in response to the shift in the gaze direction such that at least part of the real world images are presented in the portion of the screen rendering the interactive scenes of the game. The transparent mode is discontinued after a period of time.

First claim

What is claimed is:

1. A method for executing a game presented on a screen of a head mounted display (HMD), comprising,
executing the game, the execution rendering interactive scenes of the game on the screen, the screen being operated in a non-transparent mode;
receiving images of the HMD worn by a user to identify a first spatial position of the HMD in a capture location that is directed toward the user;
receiving images of a controller held by the user to identify a second spatial position of the controller in the capture location, the controller providing input to at least partially drive interaction with the game being executed;
receiving images that are used to identify a shift from a first gaze direction to a second gaze direction of eyes of the user when the user is viewing the interactive scenes of the game presented on the screen of the HMD, the images that are used to identify the shift are captured using one or more internal camera units mounted in the HMD and directed toward the eyes of the user;
receiving real-world images in line with the second gaze direction of the eyes of the user, the real-world images captured from a forward facing camera of the HMD;
transitioning a portion of the screen in the second gaze direction to a transparent mode, in response to detection of the shift, wherein the interactive scenes of the game are not rendered to the portion of the screen so that at least a portion of the captured real-world images is presented in the portion of the screen in place of the interactive scenes, wherein the transitioning is performed upon determining the shift to the second gaze direction is retained for at least a pre-defined period of time; and
discontinuing the transparent mode for the portion after a period of time,
wherein the method operations are performed by a processor.

2. The method of claim 1 , wherein the game is executed on a computer that is communicatively connected to the head mounted display.

3. The method of claim 1 , wherein the game is executed on a game cloud and the head mounted display is communicatively connected to the game cloud.

4. The method of claim 1 , wherein the transparent mode is one of a semi-transparent mode, a simulated transparent mode or a fully transparent mode.

5. The method of claim 1 , wherein the transitioning is additionally caused to occur in response to one of a sound or a movement detected in a real-world vicinity of the user wearing the HMD, an audio or a visual command provided by a user of the HMD, an audio or input detected at the HMD, a command provided from the HMD or the controller, or any combinations thereof.

6. A method for executing a game presented on a screen of a head mounted display (HMD), comprising,
executing the game, the execution rendering interactive scenes of the game on the screen of the HMD, the screen of the HMD being operated in a non-transparent mode;
receiving images that identify a shift in gaze direction of a user wearing the HMD from a first gaze direction to a second gaze direction, when viewing the interactive scenes presented on the screen of the HMD;
receiving real-world images that are in line with the second gaze direction of eyes of the user, the real-world images being captured from a forward-facing camera of the HMD;
transitioning a portion of the screen in the second gaze direction from the non-transparent mode to a semi-transparent mode in response to the shift, such that at least part of the captured real-world images is presented in the portion of the screen as an overlay over the interactive scenes, while a remainder of the screen continues to display the interactive scenes in the non-transparent mode, wherein the transitioning is performed upon determining the shift to second gaze direction lasts for at least a pre-defined period of time; and
discontinuing the semi-transparent mode for the portion after a period of time,
wherein the method operations are performed by a processor.

7. The method of claim 6 , further comprising,
receiving images of the HMD to identify a first spatial position of the HMD relative to a capture location that is directed toward the user; and
receiving images of a controller held by the user to identify a second spatial position of the controller relative to the capture location, the controller providing input to at least partially drive interaction with the game being executed.

8. A method for executing a game presented on a screen of a head mounted display (HMD), comprising,
executing the game;
rendering interactive scenes of the game on the screen of the HMD as the game is executed, the screen of the HMD being operated in a non-transparent mode;
detecting a trigger event occurring in a real-world environment within a vicinity of the HMD during the execution of the game, wherein the trigger event is detected to occur when a change in a gaze condition of the user wearing the HMD is identified, the change in the gaze condition identified using one or more internal camera units mounted in the HMD and directed toward eyes of a user, the trigger event causing a transition of a portion of the screen identified from the change in the gaze condition, from the non-transparent mode to a fully transparent mode, the transitioning includes adjusting one or more optic characteristics of the portion of the screen to switch the portion of the screen to the fully transparent mode, wherein the fully transparent mode enables a user to view a portion of the real-world environment in the vicinity of the HMD through the portion of the screen, wherein the transition of the portion of the screen is performed upon determining the change in gaze condition lasts for at least a pre-defined threshold period of time; and
discontinuing the fully transparent mode for the portion after a period of time,
wherein the method operations are performed by a processor.

9. The method of claim 8 , further includes,
examining the trigger event to determine if the transition of the screen has to be effectuated; and
performing the transition based on the determination.

10. The method of claim 9 , wherein the determination includes,
verifying a cause of the trigger event; and
effectuating the transition when the cause identifies a safety condition in the real-world environment.

11. The method of claim 8 , wherein the gaze condition is one of,
(a) detecting a gaze direction for a period of time;
(b) detecting a gaze pattern over the period of time; and
(c) detecting changes in gaze directions.

12. The method of claim 8 , wherein the transitioning is performed in linear or non-linear gradual increments.

13. The method of claim 8 , wherein discontinuing further includes adjusting the one or more optic characteristics of the portion of the screen to switch to the non-transparent mode, the switch to the non-transparent mode causes rendering of the interactive scenes of the game on the portion of the screen.

14. The method of claim 8 , wherein the screen is a liquid crystal display screen.

15. The method of claim 8 , wherein the game is executed on a computer that is communicatively connected to the HMD, or on a game cloud, wherein the HMD is communicatively connected to the game cloud, or is partly on the game cloud and partly on the computer that are communicatively connected to the HMD, the interactive scenes of the game being streamed from the computer or the game cloud.

16. The method of claim 8 , wherein the trigger event is further detected to occur in response to a sound or a movement detected near a real-world vicinity of the user wearing the HMD, an audio command

Assignees

• Sony Interactive Entertainment Inc

Inventors

• Osman Steven
• Mao Xiaodong

Classifications

• G06F3/011 Primary

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• A63F13/00

  Video games, i.e. games using an electronically generated display having two or more dimensions · CPC title

• A63F13/5255

  according to dedicated instructions from a player, e.g. using a secondary joystick to rotate the camera around a player's character · CPC title

• A63F2300/6653

  for altering the visibility of an object, e.g. preventing the occlusion of an object, partially hiding an object · CPC title

• A63F13/213

  comprising photodetecting means, e.g. cameras, photodiodes or infrared cells (A63F13/219 takes precedence) · CPC title