Calibration of augmented reality device

Patent metadata

Field	Value
Publication number	US-11244649-B2
Application number	US-202017088498-A
Country	US
Kind code	B2
Filing date	Nov 3, 2020
Priority date	Dec 21, 2017
Publication date	Feb 8, 2022
Grant date	Feb 8, 2022

Abstract

Techniques are described for calibrating a device having a first sensor and a second sensor. Techniques include capturing sensor data using the first sensor and the second sensor. The device maintains a calibration profile including a translation parameter and a rotation parameter to model a spatial relationship between the first sensor and the second sensor. Techniques include determining a calibration level associated with the calibration profile at a first time. Techniques include determining, based on the calibration level, to perform a calibration process. Techniques include performing the calibration process at the first time by generating one or both of a calibrated translation parameter and a calibrated rotation parameter and replacing one or both of the translation parameter and the rotation parameter with one or both of the calibrated translation parameter and the calibrated rotation parameter.

First claim

What is claimed is:

1. A method of updating a calibration profile that models a spatial relationship between a first sensor and a second sensor of a device, the method comprising:
capturing sensor data using one or more sensors of the device;
determining a calibration level associated with the calibration profile based on the sensor data, wherein the calibration profile includes a translation parameter and a rotation parameter; and
determining, based on the calibration level, to perform either a first calibration process or a second calibration process;
wherein performing the first calibration process includes:
generating a calibrated rotation parameter; and
replacing the rotation parameter with the calibrated rotation parameter;

wherein performing the second calibration process includes:
generating both a calibrated translation parameter and the calibrated rotation parameter; and
replacing both the translation parameter and the rotation parameter with both the calibrated translation parameter and the calibrated rotation parameter.

2. The method of claim 1 , wherein performing the first calibration process includes:
replacing only the rotation parameter with the calibrated rotation parameter.

3. The method of claim 1 , wherein the one or more sensors include the first sensor and the second sensor.

4. The method of claim 3 , wherein the sensor data includes:
one or more first images captured using the first sensor; and
one or more second images captured using the second sensor.

5. The method of claim 4 , wherein one or both of the calibrated translation parameter and the calibrated rotation parameter are generated using the sensor data.

6. The method of claim 1 , wherein the one or more sensors include an additional sensor and does not include the first sensor and the second sensor.

7. The method of claim 1 , wherein:
it is determined to perform the first calibration process when the calibration level is less than a calibration threshold; and
it is determined to perform the second calibration process when the calibration level is greater than the calibration threshold.

8. A device comprising:
a first sensor and a second sensor;
a memory device configured to store a calibration profile modeling a spatial relationship between the first sensor and the second sensor, the calibration profile including a translation parameter and a rotation parameter;
a processor coupled to the memory device, wherein the processor is configured to perform operations comprising:
capturing sensor data using one or more sensors of the device;
determining a calibration level associated with the calibration profile based on the sensor data; and
determining, based on the calibration level, to perform either a first calibration process or a second calibration process;

wherein performing the first calibration process includes:
generating a calibrated rotation parameter; and
replacing the rotation parameter with the calibrated rotation parameter;

wherein performing the second calibration process includes:
generating both a calibrated translation parameter and the calibrated rotation parameter; and
replacing both the translation parameter and the rotation parameter with both the calibrated translation parameter and the calibrated rotation parameter.

9. The device of claim 8 , wherein performing the first calibration process includes:
replacing only the rotation parameter with the calibrated rotation parameter.

10. The device of claim 8 , wherein the one or more sensors include the first sensor and the second sensor.

11. The device of claim 10 , wherein the sensor data includes:
one or more first images captured using the first sensor; and
one or more second images captured using the second sensor.

12. The device of claim 11 , wherein one or both of the calibrated translation parameter and the calibrated rotation parameter are generated using the sensor data.

13. The device of claim 8 , wherein the one or more sensors include an additional sensor and does not include the first sensor and the second sensor.

14. A non-transitory computer-readable medium comprising instructions that, when executed by a processor, cause the processor to perform operations for updating a calibration profile that models a spatial relationship between a first sensor and a second sensor of a device, the operations comprising:
capturing sensor data using one or more sensors of the device;
determining a calibration level associated with the calibration profile based on the sensor data, wherein the calibration profile includes a translation parameter and a rotation parameter; and
determining, based on the calibration level, to perform either a first calibration process or a second calibration process;
wherein performing the first calibration process includes:
generating a calibrated rotation parameter; and
replacing the rotation parameter with the calibrated rotation parameter;

wherein performing the second calibration process includes:
generating both a calibrated translation parameter and the calibrated rotation parameter; and
replacing both the translation parameter and the rotation parameter with both the calibrated translation parameter and the calibrated rotation parameter.

15. The non-transitory computer-readable medium of claim 14 , wherein performing the first calibration process includes:
replacing only the rotation parameter with the calibrated rotation parameter.

16. The non-transitory computer-readable medium of claim 14 , wherein the one or more sensors include the first sensor and the second sensor.

17. The non-transitory computer-readable medium of claim 16 , wherein the sensor data includes:
one or more first images captured using the first sensor; and
one or more second images captured using the second sensor.

18. The non-transitory computer-readable medium of claim 17 , wherein one or both of the calibrated translation parameter and the calibrated rotation parameter are generated using the sensor data.

19. The non-transitory computer-readable medium of claim 14 , wherein the one or more sensors include an additional sensor and does not include the first sensor and the second sensor.

20. The non-transitory computer-readable medium of claim 14 , wherein:
it is determined to perform the first calibration process when the calibration level is less than a calibration threshold; and
it is determined to perform the second calibration process when the calibration level is greater than the calibration threshold.

Assignees

• Magic Leap Inc

Inventors

• Chi Yu-Tseh
• Bouguet Jean-Yves
• Sharma Divya
• Huang Lei
• Strelow Dennis William
• Grossmann Etienne Gregoire
• Levine Evan Gregory
• Harmat Adam
• Swaminathan Ashwin

Classifications

• G06T2207/10012

  Stereo images · CPC title

• G09G5/003 Primary

  Details of a display terminal, the details relating to the control arrangement of the display terminal and to the interfaces thereto (specific for a CRT G09G1/165; for a flat panel G09G3/2092) · CPC title

• G06T7/0002

  Inspection of images, e.g. flaw detection · CPC title

• G06F3/147

  using display panels · CPC title

• G09G2320/0693

  Calibration of display systems · CPC title