Visual-based inertial navigation

Patent metadata

Field	Value
Publication number	US-10152795-B2
Application number	US-201615236008-A
Country	US
Kind code	B2
Filing date	Aug 12, 2016
Priority date	Aug 12, 2013
Publication date	Dec 11, 2018
Grant date	Dec 11, 2018

Abstract

A method includes: receiving sensor measurements from a pre-processing module, in which the sensor measurements include image data and inertial data for a device; transferring, using a processor, information derived from the sensor measurements, from a first set of variables associated with a first window of time to a second set of variables associated with a second window of time, in which the first and second windows consecutively overlap in time; and outputting, to a post-processing module, a state of the device based on the transferred information.

First claim

What is claimed is:

1. An electronic computing system for performing navigation, the electronic computing system comprising an electronic processor and memory operable to implement a sliding-window inverse filter module, wherein the sliding-window inverse filter module is configured to:
transfer information, from a first set of variables associated with a first window of time to a second set of variables associated with a second window of time, wherein the first window of time and the second window of time consecutively overlap in time; and
generate a path history of a device based on the first set of variables and the second set of variables,
wherein the first window of time is associated with a corresponding first information matrix including a plurality of first variables representing a position and/or orientation of at least one object proximate to the device and a plurality of second variables representing a state of the device at two or more points in time, and
wherein transferring the information comprises:
marginalizing at least a subset of the first variables associated with the first information matrix of the first window of time to obtain a second information matrix, which still maintains associations between the plurality of second variables;
marginalizing a subset of the second variables associated with the second information matrix to obtain a third information matrix;
scaling the third information matrix to obtain a fourth information matrix; and
generating a fifth information matrix associated with the second window of time based on the fourth information matrix.

2. The electronic computing system of claim 1 , wherein scaling the third information matrix to obtain the fourth information matrix includes dividing the third information matrix by a number of measurements in the first window of time.

3. The electronic computing system of claim 1 , wherein the third information matrix is a summarized representation of the state of the device and of the position and/or orientation of the at least one object proximate to the device at points in time when the first window of time and the second window of time overlap.

4. The electronic computing system of claim 1 , wherein the sliding-window inverse filter module is configured to receive sensor measurements from a pre-processing module, wherein the sensor measurements comprise image data and inertial data for the device, and wherein the fifth information matrix is generated based further on a portion of the sensor measurements corresponding to the second window of time.

5. The electronic computing system of claim 4 , further comprising:
the pre-processing module;
an image detection unit coupled to the pre-processing module, wherein the pre-processing module is configured to receive a plurality of images captured by the image detection unit and derive the image data from the plurality of images; and
a motion sensing unit coupled to the pre-processing module, wherein the pre-processing module is configured to receive the inertial data from the motion sensing unit.

6. The electronic computing system of claim 1 , wherein the sliding-window inverse filter module is further configured to output, to a post-processing module, a state of the device based on the fifth information matrix.

7. The electronic computing system of claim 1 , wherein the plurality of first variables represents 3D positions and/or orientations of image features across a plurality of images, and
wherein the plurality of second variables comprises linear acceleration data and angular velocity data.

8. The electronic computing system of claim 7 , wherein the state of the device further comprises biases for a motion sensing unit.

9. The electronic computing system of claim 1 , wherein the first information matrix comprises:
first information about at least one variable representing an estimate for a position or orientation of an object proximate to the device;
second information about at least one variable representing an estimate of the state of the device; and
shared information between the first information and the second information.

10. A computer-implemented method, comprising:
transferring, using a processor, information from a first set of variables associated with a first window of time to a second set of variables associated with a second window of time, wherein the first window of time and the second window of time consecutively overlap in time; and
generating a path history of a device based on the first set of variables and the second set of variables,
wherein the first window of time is associated with a corresponding first information matrix including a plurality of first variables representing a position and/or orientation of at least one object proximate to the device and a plurality of second variables representing a state of the device at two or more points in time, and
wherein transferring the information comprises:
marginalizing at least a subset of the first variables associated with the first information matrix of the first window of time to obtain a second information matrix, which still maintains associations between the plurality of second variables;
marginalizing a subset of the second variables associated with the second information matrix to obtain a third information matrix;
scaling the third information matrix to obtain a fourth information matrix; and
generating a fifth information matrix associated with the second window of time based on the fourth information matrix.

11. The computer-implemented method of claim 10 , wherein the first information matrix comprises:
first information about at least one variable representing an estimate for a position or orientation of an object proximate to the device;
second information about at least one variable representing an estimate of the state of the device; and
shared information between the first information and the second information.

12. The computer-implemented method of claim 11 , wherein the shared information represents shared confidence between the estimates of a position or orientation of an object proximate to the device and the estimate of the state of the device.

13. The computer-implemented method of claim 11 ,
wherein marginalizing at least the subset of the first variables associated with the first information matrix of the first window of time comprises marginalizing the shared information.

14. A non-transitory computer storage medium encoded with computer program instructions that when executed by one or more electronic computing devices cause the one or more electronic computing devices to perform operations comprising:
transferring, using a processor, information from a first set of variables associated with a first window of time to a second set of variables associated with a second window of time, wherein the first window of time and the second window of time consecutively overlap in time; and
generating a path history of a device based on the first set of variables and the second set of variables,
wherein the first window of time is associated with a corresponding first information matrix including a plurality of first variables representing a position and/or orientation of at least one object proximate to the device and a plurality of second variables representing a state of the device at two or more points in time, and
wherein transferring the information comprises:
marginalizing at least a subset of the first variables associated with the first information matrix of the first window of time to obtain a second information matrix, which still maintains associations between the plurality

Assignees

• Apple Inc
• Univ Minnesota

Inventors

• Flint Alex
• Naroditsky Oleg
• Broaddus Christopher P
• Grygorenko Andriy
• Roumeliotis Stergios
• Bergig Oriel

Classifications

• G06T7/73 Primary

  using feature-based methods · CPC title

• G01C21/206

  specially adapted for indoor navigation · CPC title

• G06T2207/20024

  Filtering details · CPC title

• G06T7/0042 Primary

  Physics · mapped topic

• G01C21/165

  combined with non-inertial navigation instruments · CPC title