Calibration of virtual reality systems

Patent metadata

Field	Value
Publication number	US-9779540-B2
Application number	US-201615346138-A
Country	US
Kind code	B2
Filing date	Nov 8, 2016
Priority date	Jan 6, 2014
Publication date	Oct 3, 2017
Grant date	Oct 3, 2017

Abstract

A virtual reality (VR) console receives slow calibration data from an imaging device and fast calibration data from an inertial measurement unit on a virtual reality headset. Using a model of the VR headset, the VR console identifies model locators corresponding to locators on the VR headset and generates estimated positions for locators included in slow calibration data. The VR console adjusts calibration parameters so a relative distance between estimated positions of the locators and positions of their corresponding model locators is less than a threshold value. From the estimated positions, the VR console generates calibrated positions of a reference point on the VR headset associated with images from the slow calibration data. The VR console determines predicted positions of the reference point from the calibrated positions and adjusts calibration parameters so intermediate estimated positions of the reference point are within a threshold distance of the predicted positions.

First claim

What is claimed is:

1. A system comprising:
a virtual reality (VR) headset including a front rigid body, a rear rigid body, a plurality of locators located on each of the front rigid body and the rear rigid body, and an inertial measurement unit (IMU) configured to output fast calibration data comprising one or more intermediate estimated positions of a reference point on the VR headset, the estimated positions describing an estimated position of the VR headset wherein each of the intermediate estimated positions is separated from a subsequent intermediate estimated position by a position time value wherein the front rigid body is non-rigidly coupled to the rear rigid body via an elastic band;
an imaging device external to the VR headset configured to capture images of the VR headset and output slow calibration data including a series of images showing portions of observed locators of the plurality of locators located on the VR headset, each image separated from a subsequent image in the series by an image time value that is larger than the position time value; and
a VR console comprising:
a processor, and
a memory coupled to the processor and including instructions that, when executed by the processor, cause the processor to track the VR headset using the slow calibration data and the fast calibration data.

2. The system of claim 1 , wherein the IMU includes a three-axis gyroscope to measure angular velocity.

3. The system of claim 1 , wherein the IMU includes a three-axis accelerometer.

4. The system of claim 1 , wherein the IMU includes a three-axis magnetometer.

5. The system of claim 1 , wherein the plurality of locators are arranged in a pattern on the VR headset that is non-coplanar.

6. The system of claim 1 , wherein the plurality of locators are light emitting diodes (LEDs).

7. The system of claim 6 , wherein the LEDs are modulated to maintain one out of two or more predetermined brightness levels during a time interval.

8. The system of claim 7 , wherein the modulation of the LEDs is selected from a group consisting of: amplitude modulation, frequency modulation, and any combination thereof.

9. The system of claim 7 , wherein the plurality of locators emit in the infrared band and an outer surface of the VR headset is transparent in the infrared band but opaque in the visible band.

10. The system of claim 6 , wherein the LEDs emit light within a specific band selected from a group consisting of a visible band and an infrared band.

11. The system of claim 1 , further comprising calibration instructions that, when executed by the processor, cause the processor to:
identify model locators each corresponding to a locator on the VR headset and included in at least one image from the slow calibration data using a stored headset model associated with the VR headset;
generate estimated positions of one or more of the locators on the VR headset and included in at least one image from the slow calibration data using the headset model;
adjust one or more calibration parameters to adjust the estimated positions so a relative distance between the adjusted estimated positions of one or more of the locators on the VR headset and included in at least one image from the slow calibration data and positions of their corresponding model locators are less than a threshold value;
generate calibrated positions of the reference point based at least in part on the adjusted estimated positions of one or more of the locators on the VR headset and included in at least one image from the slow calibration data, a calibrated position associated with an image from the slow calibration data;
determine one or more predicted positions of the reference point based at least in part on the calibrated positions of the reference point, a predicted position associated with a time between subsequent images from the slow calibration data; and
adjust one or more of the calibration parameters so the intermediate estimated positions of the reference point are within a threshold distance of the determined predicted positions of the reference point.

12. The system of claim 1 , further comprising calibration instructions that, when executed by the processor, cause the processor to:
determine an observed position of the rear rigid body for the particular image time value using the slow calibration data;
determine a predicted position of the rear rigid body for the particular image time value using the fast calibration data and a position vector describing a calibrated offset between the front rigid body and the rear rigid body;
determine that a difference between the observed position and the predicted position is greater than a threshold value;
responsive to a determination that a difference between the observed position of the rear rigid body and the predicted position of the rear rigid body is greater than a threshold value, adjust the position vector by an offset value so the difference between the observed position and the predicted position is less than the threshold value; and
determine a subsequent predicted position of the rear rigid body for an image from the series of images associated with a subsequent image time value occurring after the image time value based on the fast calibration data and the adjusted position vector.

13. A system comprising:
a virtual reality (VR) headset including a front rigid body, a rear rigid body, a plurality of locators located on each of the front rigid body and the rear rigid body, and an inertial measurement unit (IMU) configured to output fast calibration data comprising one or more intermediate estimated positions of a reference point on the VR headset, the estimated positions describing an estimated position of the VR headset wherein each of the intermediate estimated positions is separated from a subsequent intermediate estimated position by a position time value wherein the front rigid body is non-rigidly coupled to the rear rigid body via an elastic band;
an imaging device configured to output slow calibration data including a series of images showing portions of observed locators of the plurality of locators locator, on the VR headset, each image separated from a subsequent image in the series by an image time value that is larger than the position time value; and
a VR console comprising:
a processor, and
a memory coupled to the processor and including instructions that, when executed by the processor, cause the processor to track the VR headset using the slow calibration data and the fast calibration data.

14. The system of claim 13 , wherein the plurality of locators includes a locator that is an LED that is modulated to maintain one out of two or more predetermined brightness levels during a time interval.

15. The system of claim 14 , wherein the modulation of the LED is selected from a group consisting of: amplitude modulation, frequency modulation, and a combination thereof.

16. The system of claim 14 , wherein the LED emits light within a specific band selected from a group consisting of a visible band and an infrared band.

17. The system of claim 16 , wherein the LED emits in the infrared band and an outer surface of the VR headset is transparent in the infrared band but opaque in the visible band.

18. The system of claim 13 , wherein the VR headset includes a front rigid body and a rear rigid body each having one or more locators, and the front rigid body includes the IMU.

Assignees

• Oculus Vr Llc

Inventors

• Katz Dov
• Katsev Maksym
• Konzen Neil
• Lavalle Steven
• Antonov Michael

Classifications

• G06T2207/30204

  Marker · CPC title

• G02B2027/014

  comprising information/image processing systems · CPC title

• G02B2027/0178

  Eyeglass type (eyeglass details G02C) · CPC title

• G06T7/75

  involving models · CPC title

• G02B2027/0138

  comprising image capture systems, e.g. camera · CPC title