Methods and systems tracking head mounted display (HMD) and calibrations for HMD headband adjustments

Patent metadata

Field	Value
Publication number	US-10409364-B2
Application number	US-201715653465-A
Country	US
Kind code	B2
Filing date	Jul 18, 2017
Priority date	Mar 14, 2014
Publication date	Sep 10, 2019
Grant date	Sep 10, 2019

Abstract

Methods and systems are provided for head mounted display (HMD) implementations. One example implementation includes a circuit for communicating with a computing system that processes multimedia content for display in the HMD. A front unit of the HMD has a screen for displaying multimedia content, and the front unit has a set of LEDs. A rear section of the HMD has a set of LEDs. A connector is provided for coupling the front unit with the rear section of the HMD, such that adjustment of the connector changes a separation distance between at least one of the set of LEDs of the front unit and at least one of the set of LEDs of the rear section. The computing system processes image data captured of the HMD when worn by a user. The image data includes at least one of the set of LEDs of the front unit and at least one of the set of LEDs of the rear section to identify the separation distance for a current adjustment of the connector.

First claim

What is claimed is:

1. A method, comprising,
capturing video frames using a camera, the video frames configured to capture markers on a head mounted display (HMD), the markers on the HMD are analyzed in the captured video frames to determine position and orientation of the HMD; and
determining a separation distance between a first one of the markers on a front unit of the HMD and a second one of the markers on a rear section of the HMD, the first one of the markers on the front unit having first geometric configuration that is different than a second geometric configuration of the second one of the markers on the rear section of the HMD, the first geometric configuration used to detect the front unit of the HMD and the second geometric configuration used to detect the rear section of the HMD, the front unit of the HMD and the rear section of the HMD being coupled together by an adjustable section, the determining of the separation distance includes analyzing the plurality of video frames captured by the camera to determine the separation distance between the marker on the front unit and the marker on the rear section of the HMD;
wherein the separation distance is set in response to placement of the HMD on a head of a user and a setting of the adjustable section, the separation distance being used during further tracking of said position and orientation of the HMD, as the markers on the front unit and the rear section are captured and analyzed from the captured video frames;
wherein the determining the position and the orientation of the HMD further includes processing inertial data from the HMD during processing of the plurality of the captured video frames, the inertial data produces a set of values that quantify changes in rotation or changes in the position of the HMD;
from time to time, continuing to determine the separation distance between the marker on the front unit of the HMD and the marker on the rear section of the HMD, and
determining if a new separation distance includes a difference from the separation distance, wherein the separation distance is currently used for the tracking; and
using the new separation distance instead of the separation distance currently used when the difference exceeds a predefined allowable margin.

2. The method of claim 1 , wherein the marker on the rear section of the HMD includes a pair of markers, each of the pair of markers on the rear section are provided with an LED for illumination, the pair of markers being defined on the rear section at a relative fixed separation.

3. The method of claim 1 , wherein the marker on the front unit and the marker on the rear section are identified in the captured video frames when the user moves the HMD.

4. The method of claim 3 , further comprising,
calibrating a geometric model of the HMD by updating a current separation distance between the marker on the front unit and the marker on the rear section with the separation distance.

5. The method of claim 4 , wherein the separation distance is separately processed for a left side of the HMD and for a right side of the HMD, such each of the left side and the right side of the HMD includes a marker on the front unit of the HMD and a marker on the rear section of the HMD.

6. The method of claim 1 , wherein the front unit Includes a front face and side faces, and intersections between of the front face and side faces define at least four transition corners, each of the transition corners including a marker.

7. The method of claim 6 , wherein the markers in the transition corners define a substantially rectangular arrangement when viewed from the camera.

8. The method of claim 7 , wherein each of the markers in the transition corners is defined by an light emitting diodes (LED), the LEDs having an illumination shape that partially wraps the transition corners so that the illumination shape is disposed partially on the front face and partially on the side faces.

9. The method of claim 1 , wherein the markers are lights.

10. A method, comprising,
capturing video frames using a camera, the video frames configured to capture markers on a head mounted display (HMD), the markers on the HMD are analyzed in the captured video frames to determine position and orientation of the HMD; and
determining a separation distance between a first one of the markers on a front unit of the HMD and a second one of the markers on a rear section of the HMD, the first one of the markers on the front unit having first geometric configuration that is different than a second geometric configuration of the second one of the markers on the rear section of the HMD, the first geometric configuration used to detect the front unit of the HMD and the second geometric configuration used to detect the rear section of the HMD, the front unit of the HMD and the rear section of the HMD being coupled together by an adjustable section, the determining of the separation distance includes analyzing the plurality of video frames captured by the camera to determine the separation distance between the marker on the front unit and the marker on the rear section of the HMD;
wherein the separation distance is set in response to placement of the HMD on a head of a user and a setting of the adjustable section, the separation distance being used during further tracking of die position and orientation of the HMD, as die markers on the front unit and the rear section are captured and analyzed from the captured video frames;
wherein determining the position and orientation of the HMD further includes processing inertial data from the HMD during processing of the plurality of the captured video frames, the inertial data produces a set of values that quantify changes in rotation or changes in the position of the HMD;
wherein the rear section includes an adjuster for reducing or increasing size of the adjustable section of the HMD; and further comprising,
receiving a signal indicative of a flag being set when an adjustment is made to the adjuster; and
re-processing the separation distance to produce a new separation distance in response to receiving the signal.

11. A head mounted display (HMD), comprising,
a circuit for communicating with a computing system that processes multimedia content for display in the HMD;
a front unit of the HMD having a screen for displaying multimedia content, the front unit having a first set of LEDs;
a rear section of the HMD having a second set of LEDs; and
a connector for adjustably coupling the front unit of the HMD with the rear section of the HMD, such that adjustment of the connector changes a separation distance between at least one of the first set of LEDs of the front unit and at least one of the second set of LEDs of the rear section;
wherein the computing system processes image data captured of the HMD when worn by a user, the image data includes at least one of the first set of LEDs of the front unit and at least one of the second set of LEDs of the rear section to identify the separation distance between the at least one of the first set of LEDs of the front unit and at least one of the second set of LEDs of the rear section for a current adjustment of the connector, wherein the set of LEDs of the front unit are arranged in a first pattern and the set of LEDs of the rear section are arranged in a second pattern, the first pattern being different than the second pattern to enable identification of the front unit as compared to the rear section of the HMD, wherein the separation distance is between at least one LED of the front unit and at least one LED of the rear section;
an inertial sensor integrated in the HMD for tracking movement of the HMD when worn by the user, the inertial se

Assignees

• Sony Interactive Entertainment Inc

Inventors

• Mikhailov Anton
• Larsen Eric
• Stafford Jeffrey Roger

Classifications

• G06T2207/10016

  Video; Image sequence · CPC title

• G02B27/017 Primary

  Head mounted · CPC title

• G02B27/0093

  with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking · CPC title

• G02B27/0172

  characterised by optical features · CPC title

• G06T2207/30204

  Marker · CPC title