Dual IMU SLAM

Patent metadata

Field	Value
Publication number	US-11935180-B2
Application number	US-202217715880-A
Country	US
Kind code	B2
Filing date	Apr 7, 2022
Priority date	Oct 18, 2019
Publication date	Mar 19, 2024
Grant date	Mar 19, 2024

Abstract

Examples of the disclosure describe systems and methods for presenting virtual content on a wearable head device. In some embodiments, a state of a wearable head device is determined by minimizing a total error based on a reduced weight associated with a reprojection error. A view reflecting the determined state of the wearable head device is presented via a display of the wearable head device. In some embodiments, a wearable head device calculates a preintegration term based on the image data received via a sensor of the wearable head device and the inertial data received via a first IMU and a second IMU of the wearable head device. The wearable head device estimates a position of the device based on the preintegration term, and the wearable head device presents the virtual content based on the position of the device.

First claim

What is claimed is:

1. A method comprising:
receiving, via a sensor of a wearable head device, first image data and second image data;
receiving, via a first inertial measurement unit (IMU) of the wearable head device, first inertial data and third inertial data;
receiving, via a second IMU of the wearable head device, second inertial data and fourth inertial data;
calculating a first preintegration term based on the first image data and based further on one or more of the first inertial data and the second inertial data;
calculating a second preintegration term based on the second image data and based further on one or more of the third inertial data and the fourth inertial data;
estimating a position of the wearable head device based on a combination of the first preintegration term and the second preintegration term; and
presenting virtual content, via a display of the wearable head device, based on the estimated position of the wearable head device.

2. The method of claim 1 , wherein the first preintegration term is calculated based further on a bias of the first IMU.

3. The method of claim 1 , wherein:
the first preintegration term is calculated based further on a positional relationship between the first IMU and the second IMU.

4. The method of claim 3 , wherein the positional relationship is determined based on an average linear velocity of the first IMU and based further on an average linear velocity of the second IMU.

5. The method of claim 1 , wherein the first preintegration term is calculated based on one or more of a linear velocity of the first IMU and a linear velocity of the second IMU.

6. The method of claim 1 , wherein:
the first inertial data comprises angular velocity data associated with the first IMU and linear acceleration data associated with the first IMU, and
the second inertial data comprises angular velocity data associated with the second IMU and linear acceleration data associated with the second IMU.

7. The method of claim 1 , wherein the first preintegration term relates a first state of the wearable head device to a second state of the wearable head device, and wherein each of the first state and the second state comprises one or more of respective position data, respective velocity data, respective accelerometer bias data, and respective gyroscope bias data.

8. The method of claim 1 , wherein calculating the first preintegration term comprises generating a factorized graphical model.

9. The method of claim 1 , wherein the first preintegration term is associated with the first IMU.

10. The method of claim 1 , further comprising determining an offset associated with one or more of the first inertial data and the second inertial data, wherein calculating the first preintegration term is based further on the offset.

11. The method of claim 1 , wherein the first IMU and the second IMU are rigidly coupled to the wearable head device.

12. The method of claim 1 , further comprising determining, via a second sensor of the wearable head device, one or more of a light level and a texture level associated with a mixed reality environment, wherein:
the position of the wearable head device comprises a position of the wearable head device with respect to the mixed reality environment, and
one or more of said receiving the second inertial data, said calculating the first preintegration term, said estimating the position of the wearable head device based on the first preintegration term, and said presenting virtual content based on the position of the wearable head device are performed in accordance with a determination that the one or more of the light level and the texture level are below a threshold level.

13. The method of claim 1 , further comprising:
estimating a second position of the wearable head device based on the second preintegration term, and
presenting second virtual content, via the display of the wearable head device, based on the estimated second position of the wearable head device.

14. A system comprising:
a sensor of a wearable head device;
a first inertial measurement unit (IMU) of the wearable head device;
a second IMU of wearable head device;
a display of the wearable head device; and
one or more processors configured to perform a method comprising:
receiving, via the sensor, first image data and second image data;
receiving, via the first IMU, first inertial data and third inertial data;
receiving, via the second IMU, second inertial data and fourth inertial data;
calculating a first preintegration term based on the first image data and based further on one or more of the first inertial data and the second inertial data;
calculating a second preintegration term based on the second image data and based further on one or more of the third inertial data and the fourth inertial data;
estimating a position of the wearable head device based on a combination of the first preintegration term and the second preintegration term; and
presenting, on the display, virtual content based on the estimated position of the wearable head device.

15. The system of claim 14 , wherein the first preintegration term is calculated based further on a bias of the first IMU.

16. The system of claim 14 , wherein the first preintegration term is calculated based further on one or more of a linear velocity of the first IMU and a linear velocity of the second IMU.

17. The system of claim 14 , wherein the first IMU and the second IMU are rigidly coupled to the wearable head device.

18. The system of claim 14 , wherein the method further comprises:
estimating a second position of the wearable head device based on the second preintegration term, and
presenting, on the display, second virtual content based on the estimated second position of the wearable head device.

19. A non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform a method comprising:
receiving, via a sensor of a wearable head device, first image data and second image data;
receiving, via a first inertial measurement unit (IMU) of the wearable head device, first inertial data and third inertial data;
receiving, via a second IMU of the wearable head device, second inertial data and fourth inertial data;
calculating a first preintegration term based on the first image data and based further on one or more of the first inertial data and the second inertial data;
calculating a second preintegration term based on the second image data and based further on one or more of the third inertial data and the fourth inertial data;
estimating a position of the wearable head device based on a combination of the first preintegration term and the second preintegration term; and
presenting virtual content based on the estimated position of the wearable head device.

20. The non-transitory computer-readable medium of claim 19 , wherein the method further comprises:
estimating a second position of the wearable head device based on the second preintegration term, and
presenting second virtual content based on the estimated second position of the wearable head device.

Assignees

• Magic Leap Inc

Inventors

• Huang Yu-Hsiang
• Levine Evan Gregory
• Napolskikh Igor
• Kasper Dominik Michael
• Sanchez Nicuesa Manel Quim
• Sima Sergiu
• Langmann Benjamin
• Swaminathan Ashwin
• Zahnert Martin Georg
• Czuprynski Blazej Marek
• Faro Joao Antonio Pereira
• Tobler Christoph
• Ghasemalizadeh Omid

Classifications

• G06T15/20 Primary

  Perspective computation · CPC title

• G01C19/00

  Gyroscopes; Turn-sensitive devices using vibrating masses; Turn-sensitive devices without moving masses; Measuring angular rate using gyroscopic effects · CPC title

• G01P7/00

  Measuring speed by integrating acceleration (measuring travelled distance by double integration of acceleration G01C21/16) · CPC title

• G01P13/00

  Indicating or recording presence, absence, or direction, of movement (electric switches H01H; counting moving objects G06M7/00) · CPC title

• G01P15/08

  with conversion into electric or magnetic values · CPC title