Calibration of virtual reality systems

Patent metadata

Field	Value
Publication number	US-9524580-B2
Application number	US-201514589755-A
Country	US
Kind code	B2
Filing date	Jan 5, 2015
Priority date	Jan 6, 2014
Publication date	Dec 20, 2016
Grant date	Dec 20, 2016

Abstract

A virtual reality (VR) console receives slow calibration data from an imaging device and fast calibration data from an inertial measurement unit on a virtual reality headset. Using a model of the VR headset, the VR console identifies model locators corresponding to locators on the VR headset and generates estimated positions for locators included in slow calibration data. The VR console adjusts calibration parameters so a relative distance between estimated positions of the locators and positions of their corresponding model locators is less than a threshold value. From the estimated positions, the VR console generates calibrated positions of a reference point on the VR headset associated with images from the slow calibration data. The VR console determines predicted positions of the reference point from the calibrated positions and adjusts calibration parameters so intermediate estimated positions of the reference point are within a threshold distance of the predicted positions.

First claim

What is claimed is:

1. A system comprising:
a virtual reality (VR) headset including a plurality of locators and an inertial measurement unit (IMU) configured to output fast calibration data comprising one or more intermediate estimated positions of a reference point on the VR headset, each intermediate estimated position separated from a subsequent intermediate estimated position by a position time value;
an imaging device configured to output slow calibration data including a series of images showing portions of observed locators of the plurality of locator, on the VR headset, each image separated from a subsequent image in the series by an image time value that is larger than the position time value; and
a VR console comprising:
a processor configured to execute modules, and
a memory coupled to the processor and including instructions that, when executed by the processor, cause the processor to track the VR headset using the slow calibration data and the fast calibration data, the memory storing the modules, the modules comprising:
an estimation module configured to:
identify model locators each corresponding to a locator on the VR headset and included in at least one image from the slow calibration data using a stored headset model associated with the VR headset, and
generate estimated positions of one or more of the locators on the VR headset and included in at least one image from the slow calibration data using the headset model; and

a parameter adjustment module configured to:
adjust one or more calibration parameters to adjust the estimated positions so a relative distance between the adjusted estimated positions of one or more of the locators on the VR headset and included in at least one image from the slow calibration data and positions of their corresponding model locators are less than a threshold value,
generate calibrated positions of the reference point based at least in part on the adjusted estimated positions of one or more of the locators on the VR headset and included in at least one image from the slow calibration data, a calibrated position associated with an image from the slow calibration data,
determine one or more predicted positions of the reference point based at least in part on the calibrated positions of the reference point, a predicted position associated with a time between subsequent images from the slow calibration data, and
adjust one or more of the calibration parameters so the intermediate estimated positions of the reference point are within a threshold distance of the determined predicted positions of the reference point.

2. The system of claim 1 , wherein the IMU includes a three-axis gyroscope to measure angular velocity.

3. The system of claim 1 , wherein the IMU includes a three-axis accelerometer.

4. The system of claim 1 , wherein the IMU includes a three-axis magnetometer.

5. The system of claim 1 , wherein the plurality of locators are arranged in a pattern on the VR headset that is non-coplanar.

6. The system of claim 1 , wherein the plurality of locators are light emitting diodes (LEDs).

7. The system of claim 6 , wherein the LEDs are modulated to maintain one out of two or more predetermined brightness levels during a time interval.

8. The system of claim 7 , wherein the modulation of the LEDs is selected from a group consisting of: amplitude modulation, frequency modulation, and any combination thereof.

9. The system of claim 7 , wherein the plurality of locators emit in the infrared band and an outer surface of the VR headset is transparent in the infrared band but opaque in the visible band.

10. The system of claim 6 , wherein the LEDs emit light within a specific band selected from a group consisting of a visible band and an infrared band.

11. The system of claim 1 , wherein the VR headset includes a front rigid body and a rear rigid body each having one or more locators, and the front rigid body includes the IMU.

12. The system of claim 11 , wherein the front rigid body is non-rigidly coupled to the rear rigid body via an elastic band.

13. The system of claim 11 , further comprising calibration instructions that, when executed by the processor, cause the processor to:
determine an observed position of the rear rigid body for the particular image time value using the slow calibration data;
determine a predicted position of the rear rigid body for the particular image time value using the fast calibration data and a position vector describing a calibrated offset between the front rigid body and the rear rigid body;
determine that a difference between the observed position and the predicted position is greater than a threshold value;
responsive to a determination that a difference between the observed position of the rear rigid body and the predicted position of the rear rigid body is greater than a threshold value, adjust the position vector by an offset value so the difference between the observed position and the predicted position is less than the threshold value; and
determine a subsequent predicted position of the rear rigid body for an image from the series of images associated with a subsequent image time value occurring after the image time value based on the fast calibration data and the adjusted position vector.

14. A system comprising:
a virtual reality (VR) headset including a plurality of locators and an inertial measurement unit (IMU) configured to output fast calibration data comprising one or more intermediate estimated positions of a reference point on the VR headset, each intermediate estimated position separated from a subsequent intermediate estimated position by a position time value, and the IMU is coupled to a position sensor;
an imaging device configured to output slow calibration data including a series of images showing portions of observed locators of the plurality of locator, on the VR headset, each image separated from a subsequent image in the series by an image time value that is larger than the position time value; and
a VR console comprising:
a processor configured to execute modules, and
a memory coupled to the processor and including instructions that, when executed by the processor, cause the processor to track the VR headset and calibrate the VR headset using the slow calibration data and the fast calibration data, the memory storing the modules, the modules comprising:
an estimation module configured to:
identify model locators each corresponding to a locator on the VR headset and included in at least one image from the slow calibration data using a stored headset model associated with the VR headset, and
generate estimated positions of one or more of the locators on the VR headset and included in at least one image from the slow calibration data using the headset model; and

a parameter adjustment module configured to:
adjust one or more calibration parameters to adjust the estimated positions so a relative distance between the adjusted estimated positions of one or more of the locators on the VR headset and included in at least one image from the slow calibration data and positions of their corresponding model locators are less than a threshold value,
generate calibrated positions of the reference point based at least in part on the adjusted estimated positions of one or more of the locators on the VR headset and included in at least one image from the slow calibration data, a calibrated position associated with an image from the slow calibration data,
determine one or more predicted positions of the reference point based at least in part on the calibrated positions of the reference point, a predicted position ass

Assignees

• Oculus Vr Llc

Inventors

• Katz Dov
• Katsev Maksym
• Konzen Neil
• Lavalle Steve
• Antonov Michael

Classifications

• G06F3/012 Primary

  Head tracking input arrangements · CPC title

• G06T7/75

  involving models · CPC title

• G02B2027/014

  comprising information/image processing systems · CPC title

• G02B2027/0178

  Eyeglass type (eyeglass details G02C) · CPC title

• G02B2027/0187

  slaved to motion of at least a part of the body of the user, e.g. head, eye · CPC title