Methods and systems including tracking a head mounted display (HMD) and calibrations for HMD headband adjustments

Patent metadata

Field	Value
Publication number	US-9710057-B2
Application number	US-201514658123-A
Country	US
Kind code	B2
Filing date	Mar 13, 2015
Priority date	Mar 14, 2014
Publication date	Jul 18, 2017
Grant date	Jul 18, 2017

Abstract

Methods and systems are provided for head mounted display (HMD) implementations. In one example implementation, a HMD includes a circuit for communicating with a computing system that processes multimedia content for display in the HMD. Further included is a front unit of the HMD that has a screen for displaying multimedia content, and the front unit has a set of LEDs. The HMD includes an accelerometer and gyroscope disposed in the front unit of the HMD. A rear section of the HMD is provided having a set of LEDs. A headband connecting the front unit to the rear section is included, such that adjustment of the headband changes a separation distance between at least one of the set of LEDs of the front unit and at least one of the set of LEDs of the rear section.

First claim

What is claimed is:

1. A method, comprising,
capturing video frames using a camera, the video frames configured to capture markers on a head mounted display (HMD), the markers on the HMD are analyzed in the captured video frames to determine position and orientation of the HMD for processing changes to scenes generated during rendering of multimedia content that is displayed by the HMD; and
estimating a separation distance between a marker on a front unit of the HMD and a marker on a rear section of the HMD, the front unit and the rear section being coupled together by an adjustable headband, the estimating includes analyzing a plurality of video frames and inertial data captured when the video frames were captured, the analyzing producing an estimated separation distance between the marker on the front unit of the HMD and a marker on the rear section of the HMD,
wherein the estimated separation distance is used during further tracking of position and orientation of the HMD, as markers on the front unit and the rear section are captured and analyzed from the captured video frames.

2. The method of claim 1 , wherein analyzing a plurality of the captured video frames and inertial data incudes tracking inertial data captured during the processing of the plurality of the captured video frames, the inertial data produces a set of values that quantify changes in rotation and changes in position of the HMD, the set of values are mapped identified locations of the markers of the HMD in image data of the plurality of the captured video frames.

3. The method of claim 1 , wherein when the marker on the front unit and the marker on the rear section are identified in the captured video frames, the mapping of the set of values to identified locations of the markers produces the estimated separation distance.

4. The method of claim 3 , further comprising,
calibrating a geometric model of the HMD by updating a current separation distance between the marker on the front unit and the marker on the rear section with the produced estimated separation distance.

5. The method of claim 4 , wherein the estimated separation distance separately processed for a left side of the HMD and for a right side of the HMD, such each of the left side and the right side of the HMD includes a marker on a front unit of the HMD and a marker on a rear section of the HMD.

6. The method of claim 1 , wherein the front unit includes a front face and a side faces, and intersections between of the front face and side faces define at least four transition corners, each of the transition corners including a marker.

7. The method of claim 6 , wherein the markers in the transition corners define a substantial rectangular arrangement when viewed from the camera.

8. The method of claim 7 , wherein each of the markers in the transition corners is defined by an light emitting diodes (LED), the LED having an illumination shape that partially wraps the transition corners so that the illumination shape is disposed partially on the front face and partially on the side faces.

9. The method of claim 1 , wherein at least some of the changes in the scenes generated during rendering change views into and around the scenes based on movements produced from the determined position and orientation of the HMD over time during a session of use.

10. The method of claim 1 , wherein the multimedia content includes one or more of video game content, movie content, internet content, interactive content, video conferencing content, audio content, video clip content, virtual tour content, or combinations thereof.

11. The method of claim 10 , wherein when the multimedia content is video game content, the method further includes,
identifying a marker associated with a game controller in the captured video frames, the marker of the game controller being tracked for position and orientation during a session of game play along with tracking of the HMD.

12. The method of claim 11 , wherein the game controller is used to interact with video game content rendered in the HMD, and the position and orientation of the HMD being tracked independently of the position and orientation of the game controller.

13. The method of claim 1 , further comprising,
from time to time, continuing to estimate the separation distance between the marker on the front unit of the HMD and the marker on the rear section of the HMD;
determining if a new estimated separation distance includes a difference from the estimated separation distance currently used for tracking; and
using the new estimated separation distance instead of the estimated separation distance currently used when the difference exceeds a predefined allowable margin.

14. The method of claim 1 , wherein the marker on the rear section of the HMD includes a pair of markers, each of the pair of markers on the rear section provided with an LED for illumination, the pair of markers being defined on the rear section at a known relative fixed separation.

15. The method of claim 1 , wherein the rear section includes an adjuster for reducing or increasing as size of the adjustable headband.

16. The method of claim 15 , further comprising,
receiving a signal indicative of a flag being set when an adjustment is made to the adjuster; and
re-processing the estimating of the separation distance to produce a new estimated separation distance in response to receiving the signal.

17. A system, comprising,
(a) a computing system;
(b) a head mounted display (HMD) in communication with the computing system, the HMD, including,
(i) a front unit having a screen for displaying multimedia content, the front unit having a marker disposed thereon, and inertial sensors for generating inertial data indicative of changes in position and orientation of the HMD;
(ii) a rear section of the HMD connected to the front unit by an adjustable headband, the rear section having a marker disposed thereon;

(c) a camera in communication with the computing system, the camera configured to capture video frames of the HMD during a session of use, the captured video frames from time to time configured to include the marker of the front unit and the marker of the rear section;
wherein the computing system processes the captured video frames to identify the markers of the front unit and the rear section against the generated inertial data, the processing produces an estimated separation between the marker of the front unit and the marker of the rear section.

18. The system of claim 17 , wherein the markers are defined by light emitting diodes that are configured to illuminate shapes defined on surfaces of the front unit and the rear section.

19. The system of claim 17 , wherein the rear section includes an adjuster for expanding and contracting the adjustable headband.

20. The system of claim 17 , wherein the rear section includes an adjuster for changing a size of the adjustable headband, the adjuster further including a headband adjustment detector that produces a flag indicative of a change to the size, the flag being processed by the computing system to trigger a recalibration of the estimated separation.

21. The system of claim 17 , wherein the inertial sensor of the HMD includes an accelerometer and a gyroscope.

22. The system of claim 17 , wherein the markers of the HMD are defined by LEDs that are configured to illuminate to enhance tracking of the HMD by the camera, the LEDs are activated by an LED control circuit.

23. The syste

Assignees

• Sony Interactive Entertainment Inc

Inventors

• Mikhailov Anton
• Larsen Eric
• Stafford Jeffrey Roger

Classifications

• G06F3/012 Primary

  Head tracking input arrangements · CPC title

• G06T2207/30204

  Marker · CPC title

• G06T7/251

  involving models · CPC title

• G02B27/017 Primary

  Head mounted · CPC title

• G06T19/006

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title