Extrinsic parameter calibration of a vision-aided inertial navigation system

Patent metadata

Field	Value
Publication number	US-2016005164-A1
Application number	US-201414768733-A
Country	US
Kind code	A1
Filing date	Feb 21, 2014
Priority date	Feb 21, 2013
Publication date	Jan 7, 2016
Grant date	—

Abstract

This disclosure describes various techniques for use within a vision-aided inertial navigation system (VINS). A VINS comprises an image source to produce image data comprising a plurality of images, and an inertial measurement unit (IMU) to produce IMU data indicative of a motion of the vision-aided inertial navigation system while producing the image data, wherein the image data captures features of an external calibration target that is not aligned with gravity. The VINS further includes a processing unit comprising an estimator that processes the IMU data and the image data to compute calibration parameters for the VINS concurrently with computation of a roll and pitch of the calibration target, wherein the calibration parameters define relative positions and orientations of the IMU and the image source of the vision-aided inertial navigation system.

First claim

1 . A method for calibration of a vision-aided inertial navigation system comprising:
receiving image data produced by an image source of the vision-aided inertial navigation system, wherein the image data captures features of a calibration target; receiving, from an inertial measurement unit (IMU) of the vision-aided inertial navigation system, IMU data indicative of motion of the vision-aided inertial navigation system; and computing, based on the image data and the IMU data and using an estimator of the vision-aided inertial navigation system, calibration parameters for the vision-aided inertial navigation system concurrently with computation of a roll and pitch of the calibration target, wherein the calibration parameters define relative positions and orientations of the IMU and the image source of the vision-aided inertial navigation system.

2 . The method of claim 1 , wherein computing, based on the image data and the IMU data, calibration parameters for the VINS concurrently with computation of a roll and pitch of the calibration target comprises:
constructing an enhanced state vector that specifies a plurality of state estimates to be computed, wherein the state vector specifies the plurality of state estimates to include the roll and the pitch of the calibration target as well as the calibration parameters for the VINS; and iteratively processing the enhanced state vector with a Kalman Filter to compute the state the roll and the pitch of the calibration target concurrently with computation of calibration parameters for the VINS.

3 . The method of claim 2 , wherein the enhanced state vector specifies the calibration parameters to be computed as relative positions, orientations and velocities of the IMU and the image source and one or more signal biases for the IMU.

4 . The method of claim 2 , wherein the enhanced state vector specifies the calibration parameters to be computed as an orientation of a global frame of reference relative to an IMU frame of reference, an orientation of the image source relative to the IMU frame of reference, a position and a velocity of the IMU within the global frame of reference, a position of the image source within the IMU frame of reference, and a set of bias vectors for biases associated with signals of the IMU.

5 . The method of claim 1 , wherein iteratively applying a Kalman Filter comprises applying a Multi-state Constraint Kalman Filter (MSC-KF) that applies a constrained estimation algorithm to prevent projection of information from the image data and IMU data along at least one unobservable degrees of freedom of the vision-aided inertial navigation system.

6 . The method of claim 1 , wherein computing the calibration parameters for the VINS concurrently with computation of a roll and pitch of the calibration target comprises applying a constrained estimation algorithm to compute state estimates by projecting the information from the image data and IMU data for at least translations in horizontal and vertical directions while preventing projection of information from the image data and IMU data along the gravity vector.

7 . The method of claim 1 , wherein computing the calibration parameters further comprises processing the image data to reject one or more outliers within a set of features identified within the image data by:
receiving a first portion of the image data as a first set of images produced by the image source; receiving a second portion of the image data as a second set of images produced by the image source; determining an orientation of the first set of images relative to the second set of images based on the IMU data; processing the first set of images and the second set of images to identify a set of common features based on the IMU data.

8 . The method of claim 7 , wherein the first set of images are produced by a first image source of the vision-aided inertial navigation system and the second set of images are produced by a second image source of the vision-aided inertial navigation system.

9 . The method of claim 7 , wherein the first set of images are produced by a first image source of the vision-aided inertial navigation system at a first orientation and the second set of images are produced by the first image source of the vision-aided inertial navigation system at a second rotation without translation of the first image source.

10 . The method of claim 1 , further comprising:
after computing the calibration parameters for the VINS, receiving a second set of images produced by the image source of the vision-aided inertial navigation system; receiving, from the inertial measurement unit (IMU) of the vision-aided inertial navigation system, IMU data indicative of motion of the vision-aided inertial navigation system while capturing the second set of images; processing the second set of images as a sliding window of sequenced images to detect a hovering condition during which a translation of the VINS is below a threshold amount of motion for each of a set of degrees of freedom; and computing a position and an orientation of the VINS based on the sliding window of the sequenced set of images and inertial measurement data for the device.

11 . The method of claim 10 , further comprising:
upon detecting the hovering condition, replacing a newest one of the images in the sliding window of sequenced images with a current image produced by the image source; and upon detecting that the absence of the hovering condition, dropping an oldest one of the images in the sliding window of sequenced images and including the current image within sliding window of sequenced images the current image.

12 . The method of claim 10 , further comprising computing a map of an area traversed by the vision-aided inertial navigation system based at least in part on the sliding window of sequenced images.

13 . The method of claim 10 , further comprising computing an odometry traversed by the device based at least in part on the sliding window of sequenced images.

14 . The method of claim 10 , further comprising computing at least one of a pose of the device, a velocity of the device, a displacement of the device and 3D positions of visual landmarks based at least in part on the sliding window of sequenced images.

15 . The method of claim 1 , wherein the calibration target is not substantially aligned with gravity.

16 . A vision-aided inertial navigation system (VINS) comprising:
an image source to produce image data comprising a plurality of images; an inertial measurement unit (IMU) to produce IMU data indicative of a motion of the vision-aided inertial navigation system while producing the image data, wherein the image data captures features of an external calibration target; a processing unit comprising an estimator that processes the IMU data and the image data to compute calibration parameters for the VINS concurrently with computation of a roll and pitch of the calibration target, wherein the calibration parameters define relative positions and orientations of the IMU and the image source of the vision-aided inertial navigation system.

17 . The vision-aided inertial navigation system (VINS) of claim 16 ,
wherein the estimator constructs an enhanced state vector that specifies a plurality of state estimates to be computed based on the image data and the IMU data, wherein the state vector specifies the plurality of state estimates to include the roll and the pitch of the calibration target as well as the calibration parameters for the VINS, and wherein the estimator iteratively processes the en

Assignees

• Univ Minnesota

Inventors

• Roumeliotis Stergios I
• Kottas Dimitrios G
• Wu Kejian J

Classifications

• G01C21/185

  for gravity · CPC title

• G01C21/188

  for accumulated errors, e.g. by coupling inertial systems with absolute positioning systems · CPC title

• G01C21/1656

  with passive imaging devices, e.g. cameras · CPC title

• G01C21/165 Primary

  combined with non-inertial navigation instruments · CPC title

• G01C21/1652

  with ranging devices, e.g. LIDAR or RADAR · CPC title