Head mounted display

Patent metadata

Field	Value
Publication number	US-11504609-B2
Application number	US-202117241063-A
Country	US
Kind code	B2
Filing date	Apr 27, 2021
Priority date	Jun 9, 2013
Publication date	Nov 22, 2022
Grant date	Nov 22, 2022

Abstract

A head mounted display (HMD) is provided. The HMD includes a housing and a view port of the housing. The view port has a screen for rendering an augmented reality scene. Included is a communications device for exchanging streaming data over a network. A depth camera integrated in the housing and oriented to capture depth data of an environment in front of the housing is included. A processor is configured to use the depth data captured by the depth camera to identify spatial positioning of real objects in the environment. A real object is rendered into the augmented reality scene, and the real object is tracked such that insertion of augmented reality objects are placed in coordination with movements of the real object shown in the augmented reality scene. The real object captured by the depth camera is the environment where a user wearing the HMD is located.

First claim

What is claimed is:

1. A head mounted display (HMD), comprising,
a housing;
a view port of the housing, the view port having a screen for rendering an augmented reality scene;
a communications device for exchanging streaming data over a network;
a depth camera integrated in the housing and oriented to capture depth data of an environment in front of the housing; and
a processor is configured to use the depth data captured by the depth camera to identify spatial positioning of real objects in the environment, wherein a real object is captured using a front facing camera of the housing of the HMD and rendered into the augmented reality scene, wherein movements of the real object are tracked using the depth camera and the front facing camera so that that insertion of augmented reality objects are placed into the augmented reality scene in coordination with movements of the real object relative to a tracked position of the HMD;
wherein the processor of the HMD is in communication with a controller used by a user of the HMD, the controller being rendered as a virtual object being held by a virtual hand of the user, the virtual hand is shown in the HMD as an extension of a real arm of the user;
wherein the real object captured by the depth camera is in the environment where a user wearing the HMD is located.

2. The HMD of claim 1 , further comprising,
using the depth camera to further capture depth data for a hand of the user that is extended into and shown in the augmented reality scene.

3. The HMD of claim 1 , wherein a hand is rendered in the augmented reality scene as said virtual hand along with an augmented reality object, the virtual hand is allowed to interact with content rendered in the augmented reality scene.

4. The HMD of claim 1 , wherein the processor executes data for input data, image data, audio data, or combinations of two or more thereof.

5. The HMD of claim 1 , wherein audio data is provided to the HMD as associated with content of the augmented reality scene.

6. The HMD of claim 5 , wherein the content includes game data, video data, internet navigation data, interactive data, combinations of game data and internet data, social media data, or combinations thereof.

7. A head mounted display (HMD), comprising,
a housing;
a view port of the housing, the view port having a screen for rendering an augmented reality scene;
a communications device for exchanging streaming data over a network;
a depth camera coupled to the housing and oriented to capture depth data of an environment in front of the housing;
a front facing camera for capturing image data of the environment in front of the housing;
an inertial sensor integrated in the HMD, the inertial sensor produces inertial data in response to detected movement of the HMD when worn by a user; and
a processor is configured to use the depth data captured by the depth camera and the image data captured by the front facing camera to identify real objects in the environment, wherein a real object is rendered in the augmented reality scene, the real object being tracked using the depth data and the image data such that positions of the real object relative to the HMD are updated in the augmented reality scene, wherein the processor is configured to render an augmented reality object in coordination with said positions of the real object when viewed relative to a tracked position of the HMD that uses said inertial sensor and one or both of said depth camera and front facing camera for said tracked position of the HMD;
wherein the processor of the HMD is in communication with a controller used by a user of the HMD, the controller being rendered as a virtual object being held by a virtual hand of the user, the virtual hand is shown in the HMD as an extension of a real arm of the user.

8. The HMD of claim 7 , wherein the processor executes data for input data, image data, audio data, inertial data, or combinations of two or more thereof.

9. The HMD of claim 8 , wherein audio data is provided to the HMD as associated with content of the augmented reality scene.

10. The HMD of claim 9 , wherein the content includes game data, video data, internet navigation data, interactive data, combinations of game data and internet data, social media data, or combinations thereof.

Assignees

• Sony Interactive Entertainment Inc

Inventors

• Mikhailov Anton
• Mao Xiadong

Classifications

• G02B2027/0187

  slaved to motion of at least a part of the body of the user, e.g. head, eye · CPC title

• G06T19/006

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06F3/012

  Head tracking input arrangements · CPC title

• G02B2027/0112

  comprising device for genereting colour display · CPC title

• A63F13/5255

  according to dedicated instructions from a player, e.g. using a secondary joystick to rotate the camera around a player's character · CPC title