Visual-based inertial navigation

Patent metadata

Field	Value
Publication number	US-9424647-B2
Application number	US-201414457286-A
Country	US
Kind code	B2
Filing date	Aug 12, 2014
Priority date	Aug 12, 2013
Publication date	Aug 23, 2016
Grant date	Aug 23, 2016

Abstract

A method includes: receiving sensor measurements from a pre-processing module, in which the sensor measurements include image data and inertial data for a device; transferring, using a processor, information derived from the sensor measurements, from a first set of variables associated with a first window of time to a second set of variables associated with a second window of time, in which the first and second windows consecutively overlap in time; and outputting, to a post-processing module, a state of the device based on the transferred information.

First claim

What is claimed is:

1. An electronic computing system for performing navigation, the electronic computing system comprising an electronic processor and memory operable to implement a sliding-window inverse filter module, wherein the sliding-window inverse filter module is configured to:
receive sensor measurements from a pre-processing module, wherein the sensor measurements comprise image data and inertial data for a device;
transfer information, derived from the sensor measurements, from a first set of variables associated with a first window of time to a second set of variables associated with a second window of time, wherein the first and second windows consecutively overlap in time; and
output, to a post-processing module, a state of the device based on the transferred information,
wherein each window in the pair is associated with a corresponding first information matrix, and each first information matrix is for a plurality of first variables representing a position and/or orientation of at least one object proximate to the device and a plurality of second variables representing a state of the device at two or more points in time, and
wherein transferring the information comprises:
marginalizing at least a subset of the first variables associated with the first information matrix of the first window to obtain a second information matrix, which still maintains the association with the plurality of second variables;
marginalizing a subset of the second variables associated with the second information matrix to obtain a third information matrix; and
scaling the third information matrix to obtain a fourth information matrix,
wherein the fourth information matrix is a summarized representation of the information about the state of the device and of the position and/or orientation of the objects proximate to the device at points in time when the first window and the second window in the pair overlap.

2. The electronic computing system of claim 1 , further comprising:
the pre-processing module;
an image detection unit coupled to the pre-processing module, wherein the pre-processing module is configured to receive a plurality of images captured by the image detection unit and derive the image data from the plurality of images; and
a motion sensing unit coupled to the pre-processing module, wherein the pre-processing module is configured to receive the inertial data from the motion sensing unit.

3. The electronic computing system of claim 1 , wherein a subset of the first set of variables associated with the first window overlaps in time with a subset of the second set of variables associated with the second window.

4. The electronic computing system of claim 3 , wherein the overlap between the first set of variables and the second set of variables is at the earliest time that is present in both the first and second windows.

5. The electronic computing system of claim 1 , wherein, for each window, the plurality of first variables represents 3D positions and/or orientations of image features across a plurality of images, and
wherein, for each window, the plurality of second variables comprises linear acceleration data and angular velocity data.

6. The electronic computing system of claim 5 , wherein the state of the device further comprises biases for the motion sensing unit.

7. The electronic computing system of claim 1 , wherein the first information matrix associated with each window comprises:
first information about at least one variable representing an estimate for a position or orientation of an object proximate to the device; and
second information about at least one variable representing an estimate of the state of the device; and
shared information between the first information and the second information.

8. The electronic computing system of claim 7 , wherein the shared information represents shared confidence between the estimates of a position or orientation of an object proximate to the device and the estimate of the state of the device.

9. The electronic computing system of claim 7 ,
wherein marginalizing at least the subset of the first variables associated with the first information matrix of the first window comprises marginalizing the shared information, and
wherein scaling the third information matrix comprises dividing or multiplying the third information matrix by a constant.

10. The electronic computing system of claim 9 , wherein the constant is equal to a size of the first window.

11. A computer-implemented method comprising:
receiving sensor measurements from a pre-processing module, wherein the sensor measurements comprise image data and inertial data for a device;
transferring, using a processor, information derived from the sensor measurements, from a first set of variables associated with a first window of time to a second set of variables associated with a second window of time, wherein the first and second windows consecutively overlap in time; and
outputting, to a post-processing module, a state of the device based on the transferred information,
wherein each window in the pair is associated with a corresponding first information matrix, and each first information matrix is for a plurality of first variables representing a position and/or orientation of at least one object proximate to the device and a plurality of second variables representing a state of the device at two or more points in time, and
wherein transferring the information comprises:
marginalizing at least a subset of the first variables associated with the first information matrix of the first window to obtain a second information matrix, which still maintains the association with the plurality of second variables;
marginalizing a subset of the second variables associated with the second information matrix to obtain a third information matrix; and
scaling the third information matrix to obtain a fourth information matrix,
wherein the fourth information matrix is a summarized representation of the information in the state of the device and of the position and/or orientation of the objects proximate to the device at points in time when the first window and the second window in the pair overlap.

12. The computer-implemented method of claim 11 , wherein a subset of the first set of variables associated with the first window overlaps in time with a subset of the second set of variables associated with the second window.

13. The computer-implemented method of claim 12 , wherein the overlap between the first set of variables and the second set of variables is at the earliest time that is present in both the first and second windows.

14. The computer-implemented method of claim 11 , wherein, for each window, the plurality of first variables represents 3D positions and/or orientations of image features across a plurality of images, and
wherein, for each window, the plurality of second variables comprises linear acceleration data and angular velocity data.

15. The computer-implemented method of claim 14 , wherein the state of the device further comprises biases for the motion sensing unit.

16. The computer-implemented method of claim 11 , wherein the first information matrix associated with each window comprises:
first information about at least one variable representing an estimate for a position or orientation of an object proximate to the device; and
second information about at least one variable representing an estimate of the state of the device; and
shared information between the first information and the second information.

17. The computer-impleme

Assignees

• Apple Inc
• Univ Minnesota

Inventors

• Flint Alex
• Naroditsky Oleg
• Broaddus Christopher P
• Grygorenko Andriy
• Roumeliotis Stergios
• Bergig Oriel

Classifications

• G01C21/206

  specially adapted for indoor navigation · CPC title

• G06T2207/20024

  Filtering details · CPC title

• G06T7/73 Primary

  using feature-based methods · CPC title

• G01C21/188

  for accumulated errors, e.g. by coupling inertial systems with absolute positioning systems · CPC title

• G01C21/1656

  with passive imaging devices, e.g. cameras · CPC title