Extrinsic parameter calibration of a vision-aided inertial navigation system

Patent metadata

Field	Value
Publication number	US-10254118-B2
Application number	US-201414768733-A
Country	US
Kind code	B2
Filing date	Feb 21, 2014
Priority date	Feb 21, 2013
Publication date	Apr 9, 2019
Grant date	Apr 9, 2019

Abstract

This disclosure describes various techniques for use within a vision-aided inertial navigation system (VINS). A VINS comprises an image source to produce image data comprising a plurality of images, and an inertial measurement unit (IMU) to produce IMU data indicative of a motion of the vision-aided inertial navigation system while producing the image data, wherein the image data captures features of an external calibration target that is not aligned with gravity. The VINS further includes a processing unit comprising an estimator that processes the IMU data and the image data to compute calibration parameters for the VINS concurrently with computation of a roll and pitch of the calibration target, wherein the calibration parameters define relative positions and orientations of the IMU and the image source of the vision-aided inertial navigation system.

First claim

The invention claimed is:

1. A method for calibration of a vision-aided inertial navigation system comprising:
receiving image data produced by an image source of the vision-aided inertial navigation system (VINS), wherein the image data captures features of a calibration target;
receiving, from an inertial measurement unit (IMU) of the VINS, IMU data indicative of motion of the VINS; and
computing, based on the image data and the IMU data and using an estimator of the VINS, calibration parameters for the VINS concurrently with computation of a roll and pitch of the calibration target at least by applying a constrained estimation algorithm to compute state estimates based on the image data and the IMU data while preventing projection of information from the image data and the IMU data along at least one unobservable degree of freedom of the VINS,
wherein the calibration parameters define relative positions and orientations of the IMU and the image source of the VINS.

2. The method of claim 1 , wherein computing, based on the image data and the IMU data, calibration parameters for the VINS concurrently with computation of the roll and pitch of the calibration target comprises:
constructing an enhanced state vector that specifies a plurality of state estimates to be computed, wherein the state vector specifies the plurality of state estimates to include the roll and the pitch of the calibration target as well as the calibration parameters for the VINS; and
iteratively processing the enhanced state vector with a Kalman Filter to compute the roll and the pitch of the calibration target concurrently with computation of calibration parameters for the VINS.

3. The method of claim 2 , wherein the enhanced state vector specifies the calibration parameters to be computed as relative positions, orientations and velocities of the IMU and the image source and one or more signal biases for the IMU.

4. The method of claim 2 , wherein the enhanced state vector specifies the calibration parameters to be computed as an orientation of a global frame of reference relative to an IMU frame of reference, an orientation of the image source relative to the IMU frame of reference, a position and a velocity of the IMU within the global frame of reference, a position of the image source within the IMU frame of reference, and a set of bias vectors for biases associated with signals of the IMU.

5. The method of claim 2 , wherein iteratively applying the Kalman Filter comprises applying a Multi-state Constraint Kalman Filter (MSC-KF) that applies the constrained estimation algorithm to prevent the projection of information from the image data and IMU data along the at least one unobservable degrees of freedom of the VINS.

6. The method of claim 1 , wherein computing the calibration parameters for the VINS concurrently with computation of the roll and pitch of the calibration target comprises applying the constrained estimation algorithm to compute the state estimates by projecting information from the image data and IMU data for at least translations in horizontal and vertical directions while preventing projection of information from the image data and IMU data along the at least one unobservable degree of freedom, the at least one unobservable degree of freedom comprising at least a gravity vector.

7. The method of claim 1 , wherein computing the calibration parameters further comprises processing the image data to reject one or more outliers within a set of features identified within the image data by:
receiving a first portion of the image data as a first set of images produced by the image source;
receiving a second portion of the image data as a second set of images produced by the image source;
determining an orientation of the first set of images relative to the second set of images based on the IMU data;
processing the first set of images and the second set of images to identify a set of common features based on the IMU data.

8. The method of claim 7 , wherein the first set of images are produced by a first image source of the VINS and the second set of images are produced by a second image source of the VINS.

9. The method of claim 7 , wherein the first set of images are produced by a first image source of the VINS at a first orientation and the second set of images are produced by the first image source of the VINS at a second rotation without translation of the first image source.

10. The method of claim 1 , wherein the calibration target is not aligned with gravity.

11. The method of claim 1 , wherein the VINS comprises a robot or a vehicle.

12. The method of claim 1 , wherein the VINS comprises one of a mobile sensing platform, a mobile phone, a workstation, a computing center, or a set of one or more servers.

13. A vision-aided inertial navigation system (VINS) comprising:
an image source to produce image data comprising a plurality of images;
an inertial measurement unit (IMU) comprising at least one of an accelerometer or a gyroscope, the IMU being configured to produce IMU data indicative of a motion of the VINS while producing the image data, wherein the image data captures features of an external calibration target;
one or more processors configured to process the IMU data and the image data to compute calibration parameters for the VINS concurrently with computation of a roll and pitch of the calibration target at least by applying a constrained estimation algorithm to compute state estimates based on the image data and the IMU data while preventing projection of information from the image data and the IMU data along at least one unobservable degree of freedom of the VINS,
wherein the calibration parameters define relative positions and orientations of the IMU and the image source of the VINS.

14. The vision-aided inertial navigation system (VINS) of claim 13 ,
wherein the one or more processors are further configured to construct an enhanced state vector that specifies a plurality of state estimates to be computed based on the image data and the IMU data, wherein the state vector specifies the plurality of state estimates to include the roll and the pitch of the calibration target as well as the calibration parameters for the VINS, and
wherein the one or more processors are further configured to iteratively process the enhanced state vector with a Kalman Filter to compute the roll and the pitch of the calibration target concurrently with computation of calibration parameters for the VINS.

15. The vision-aided inertial navigation system (VINS) of claim 13 , wherein the one or more processors execute within a device that houses the image source and the IMU.

16. The vision-aided inertial navigation system (VINS) of claim 13 , wherein the one or more processors are remote from the image source and the IMU unit.

17. The vision-aided inertial navigation system (VINS) of claim 13 , wherein the VINS comprises one of a mobile sensing platform, a mobile phone, a workstation, a computing center, or a set of one or more servers.

18. The vision-aided inertial navigation system (VINS) of claim 13 , wherein the image source comprises one or more of a cameras that capture 2D or 3D images, a laser scanner that produces a stream of 1D image data, a depth sensor that produces image data indicative of ranges for features within an environment and a stereo vision system having multiple cameras to produce 3D information.

19. The vision-aided inertial navigation system of claim 13 , wherein the vision-aided inertial navigation comprises a robot or a vehicle.

Assignees

• Univ Minnesota

Inventors

• Roumeliotis Stergios I
• Kottas Dimitrios G
• Wu Kejian J

Classifications

• G06T2207/30244

  Camera pose · CPC title

• Y10S901/09

  Closed loop, sensor feedback controls arm movement · CPC title

• G06T2207/10021

  Stereoscopic video; Stereoscopic image sequence · CPC title

• G06F3/0346

  with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors · CPC title

• H04N13/204

  using stereoscopic image cameras (stereoscopic photography G03B35/00) · CPC title