Head mounted display presentation adjustment

Patent metadata

Field	Value
Publication number	US-9928653-B2
Application number	US-201514685482-A
Country	US
Kind code	B2
Filing date	Apr 13, 2015
Priority date	Apr 14, 2014
Publication date	Mar 27, 2018
Grant date	Mar 27, 2018

Abstract

Embodiments are disclosed for adjusting a presentation on a head-mounted display (HMD). In one or more example embodiments, a method of dynamically orienting a presentation of a HMD includes gathering HMD sensor data via at least one HMD sensor that is installed on an HMD worn by a driver of the vehicle and gathering vehicle sensor data via at least one vehicle mounted sensor mounted to the vehicle. The example method further includes performing an analysis of the HMD sensor data and of the vehicle sensor data to identify a difference between the HMD sensor data and the vehicle sensor data, and calculating, based on the difference, an orientation of the HMD device in relation to the vehicle. The method further includes adjusting a presentation of data on a display of the HMD device based on the orientation.

First claim

The invention claimed is:

1. A method of dynamically orienting a presentation of a head-mounted display (HMD), comprising:
gathering HMD sensor data via at least one HMD sensor installed on an HMD worn by a driver of a vehicle, the HMD sensor data comprising an HMD image captured by the at least one HMD sensor installed on the HMD worn by the driver;
gathering vehicle sensor data via at least one vehicle-mounted sensor mounted on the vehicle;
performing an analysis of each of said HMD sensor data and of said vehicle sensor data to identify a difference between said HMD sensor data and said vehicle sensor data by identifying a difference in an apparent position of an object located in an environment of the vehicle and included in each of the HMD image and a vehicle image captured by the at least one vehicle-mounted sensor;
calculating, based on said difference, an orientation of said HMD in relation to said vehicle; and
adjusting a presentation of data on a display of said HMD based on said orientation.

2. The method of claim 1 , wherein the vehicle sensor data comprises the vehicle image, and wherein the at least one HMD sensor is installed on a location of the HMD that is positioned at a front or side region of a head of the driver when the HMD is worn by the driver.

3. The method of claim 2 , wherein the HMD image includes a driver control area and a scenery seen through at least part of a windshield of the vehicle, wherein the vehicle image includes at least a portion of the scenery imaged from a perspective of the vehicle sensor, and wherein the object is located in the scenery.

4. The method of claim 2 , wherein the HMD image includes a driver control area comprising a set of size-known markers arranged in a predetermined arrangement, and where performing the analysis of the HMD sensor data comprises identifying an apparent size of each member of the set of size-known markers in the HMD image.

5. The method of claim 4 , wherein identifying an apparent position and size of each member of the set of size-known markers comprises detecting the apparent position and size of each member of the set of size-known markers using a Scale-invariant feature transform (SIFT) process.

6. The method of claim 2 , further comprising combining each of the HMD image and the vehicle image to create a stereoscopic image, wherein the vehicle image and the HMD image depict two different views of same scenery.

7. The method of claim 1 , wherein the at least one HMD sensor includes a motion detector installed on the HMD worn by the driver and the HMD sensor data comprises a current measured HMD device motion determined via at least one HMD device motion detector, wherein the vehicle sensor data comprises a current measured vehicle motion determined via at least one vehicle motion detector, and wherein performing the analysis of the HMD sensor data and the vehicle sensor data further comprises identifying a difference between the current HMD device motion and the current vehicle motion, the motion detector including one or more of an accelerometer, a gyroscope, and a compass.

8. The method of claim 7 , further comprising capturing the HMD image, via at least one HMD image sensor installed on the HMD, of a driver control area and a scenery seen through part of a windshield of the vehicle, capturing the vehicle image, via at least one vehicle imaging device mounted on the vehicle, of at least a portion of the scenery, wherein performing the analysis of the HMD sensor data and the vehicle sensor data further comprises identifying a difference between an apparent position of an object located in the scenery and imaged in both the HMD image and the vehicle image, and wherein calculating the orientation is further based on the apparent position.

9. The method of claim 7 , further comprising capturing the HMD image, via at least one HMD image sensor installed on the HMD, of a driver control area comprising a set of size-known markers arranged in a predetermined arrangement, wherein performing the analysis of the HMD sensor data further comprises identifying an apparent position and size of each member of the set of size-known markers in the HMD image, and wherein calculating the orientation is further based on the apparent position, the markers positioned in the driver control area that is visible to the at least one vehicle-mounted sensor.

10. A system for dynamically orienting a presentation of a head-mounted display (HMD), comprising:
at least two image sensors including:
at least one HMD image sensor which is installed on an HMD and worn by a driver of a vehicle that captures, from a location of the driver, an HMD image of a driver control area and a part of a windshield and a scenery seen through said part of said windshield and in front of said vehicle, the scenery including a real-world object having a first apparent position from a perspective of the at least one HMD image sensor; and
at least one device image sensor of an imaging device mounted on a base fixated to said vehicle that captures a vehicle image of said scenery, the real-world object in the scenery having a second apparent position from a perspective of the at least one device image sensor; and

an analysis and calculation module comprising instructions executable by a processor of the HMD to:
perform an analysis of said HMD image and of said vehicle image to identify a difference in the first and second apparent positions of the real-world object located in said scenery and depicted both in said HMD image and in said vehicle image;
calculate, based on said difference, an orientation of said HMD in relation to said vehicle using a calculation including one or more of correlation, transformation, and translation algorithms performed between the HMD image and the vehicle image; and
adjust a presentation of data on a display of said HMD based on said orientation.

11. The system of claim 10 , wherein adjusting the presentation of data on the display of the HMD based on the orientation comprises moving a location of a virtual object by an amount based on the difference between the first and second apparent positions of the real-world object responsive to determining that the data on the display comprises a virtual object tied to a real-world object inside of the vehicle.

12. The system of claim 10 , wherein the HMD image includes a set of size-known markers arranged in a predetermined arrangement, and where performing the analysis of the HMD image comprises identifying an apparent position and size of each member of the set of size-known markers in the HMD image, the markers positioned in the driver control area that is visible to the device image sensor.

13. The system of claim 12 , wherein identifying the apparent position and size of each member of the set of size-known markers comprises detecting the apparent position and size of each member of the set of size-known markers using a Scale-invariant feature transform (SIFT) process.

14. The system of claim 10 , further comprising at least one HMD device motion detector configured to determine a current HMD device motion and at least one vehicle motion detector configured to determine a current vehicle motion, and wherein the instructions are further executable to calculate the orientation of the HMD relative to the vehicle based on a difference between the current HMD device motion and the current vehicle motion, and wherein the vehicle motion detector includes one or more of an accelerometer, a gyroscope, and a compass.

15. The system of claim 10 , wherein the instructions are further executable to combine each of the HMD image an

Assignees

• Harman Int Ind

Inventors

• Atsmon Dan

Classifications

• G02B2027/0178

  Eyeglass type (eyeglass details G02C) · CPC title

• G06F3/04815

  Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object · CPC title

• G02B2027/014

  comprising information/image processing systems · CPC title

• G06F3/012 Primary

  Head tracking input arrangements · CPC title

• G02B2027/0187

  slaved to motion of at least a part of the body of the user, e.g. head, eye · CPC title