Extrinsic sensor calibration systems and methods

Patent metadata

Field	Value
Publication number	US-11609574-B2
Application number	US-201916676186-A
Country	US
Kind code	B2
Filing date	Nov 6, 2019
Priority date	Nov 13, 2018
Publication date	Mar 21, 2023
Grant date	Mar 21, 2023

Abstract

Techniques are disclosed for systems and methods to provide extrinsic sensor calibration for mobile structures. A sensor calibration system includes first and second sensors coupled to a mobile structure and a logic device. The logic device is configured to receive first and second series of pose measurements corresponding to sensor data provided by the respective first and second sensors, determine a set of intermediate calibration transformation estimates corresponding to the first and second sensors based, at least in part, on a scale-dependent calibration error function and/or the first and second series of pose measurements, and determine an ongoing calibration transformation estimate corresponding to the first and second sensors based, at least in part, on the determined set of intermediate calibration transformation estimates.

First claim

What is claimed is:

1. A method comprising:
receiving first and second series of pose measurements corresponding to sensor data provided by respective first and second sensors mounted to a mobile structure;
determining a set of intermediate calibration transformation estimates corresponding to the first and second sensors based, at least in part, on the first and second series of pose measurements, wherein the determining the set of intermediate calibration transformation estimates comprises:
generating a set of pose windows each comprising a time-contiguous subset of the first and second series of pose measurements; and
determining, for each pose window of the set of pose windows, an intermediate calibration transformation estimate of the set of intermediate calibration transformation estimates based on the corresponding time-contiguous subset of the first and second series of pose measurements;

determining an ongoing calibration transformation estimate corresponding to the first and second sensors based, at least in part, on the determined set of intermediate calibration transformation estimates; and
calibrating data from at least one of the first sensor or the second sensor to a coordinate frame of the mobile structure based on the ongoing calibration transformation estimate.

2. The method of claim 1 , wherein:
the first sensor comprises a first camera with a first field of view and the second sensor comprises a second camera with a second field of view different from the first field of view; and
the sensor data comprises first image data captured by the first camera substantially synchronously relative to second image data captured by the second camera.

3. The method of claim 1 , wherein:
the first sensor comprises a camera mounted at a first position on the mobile structure and the second sensor comprises a position sensor, an orientation sensor, a gyroscope, or an accelerometer.

4. The method of claim 1 , further comprising:
receiving the sensor data from the first and second sensors; and processing the sensor data according to a visual odometry algorithm and/or a simultaneous localization and mapping algorithm to generate the first and second series of pose measurements.

5. The method of claim 1 , wherein,
for each pose window of the set of pose windows, the intermediate calibration transformation estimate of the set of intermediate calibration transformation estimates is determined further based on a scale-dependent calibration error function.

6. The method of claim 5 , wherein:
each pose window of the set of pose windows at least partially overlaps an adjacent pose window; and
the determining the intermediate calibration transformation estimate comprises varying one or more parameters of a test calibration transformation element of the scale-dependent calibration error function to minimize the scale-dependent calibration error function.

7. The method of claim 6 , wherein:
the mobile structure comprises a watercraft; and
the one or more parameters of the test calibration transformation element of the scale-dependent calibration error function comprises a scale parameter of the test calibration transformation element.

8. The method of claim 5 , further comprising:
determining one or more pose rotations corresponding to a pose window of the set of pose windows;
determining at least one of the pose rotations is smaller than a noise threshold; and
discarding the pose window from the set of pose windows prior to the determining the intermediate calibration transformation estimate for the pose window.

9. The method of claim 5 , further comprising:
determining a pose rotation corresponding to a pose window of the set of pose windows is less than a preset rotation threshold; and
determining a limited intermediate calibration transformation estimate for the pose window.

10. The method of claim 1 , wherein the determining the ongoing calibration transformation estimate comprises:
generating an inlier threshold based, at least in part, on the set of intermediate calibration transformation estimates;
comparing, for each intermediate calibration transformation estimate in the set of intermediate calibration transformation estimates, a cost value of a scale-dependent calibration error function;
validating intermediate calibration transformation estimates in the set of intermediate calibration transformation estimates with cost values less than the inlier threshold; and
determining the ongoing calibration transformation estimate based on pose measurements in the first and second series of pose measurements corresponding to validated intermediate calibration transformation estimates in the set of intermediate calibration transformation estimates.

11. A system to perform the method of claim 1 , the system comprising:
a logic device configured to communicate with the first and second sensors mounted to the mobile structure, wherein the logic device is configured to:
receive the first and second series of pose measurements corresponding to the sensor data provided by the first and second sensors;
determine the set of intermediate calibration transformation estimates corresponding to the first and second sensors based, at least in part, on the first and second series of pose measurements;
determine the ongoing calibration transformation estimate corresponding to the first and second sensors based, at least in part, on the determined set of intermediate calibration transformation estimates; and
calibrate the data from the at least one of the first sensor or the second sensor to a coordinate frame of the mobile structure based on the ongoing calibration transformation estimate.

12. A system comprising:
a first sensor mounted at a first position on a mobile structure;
a second sensor mounted at a second position on the mobile structure; and
a logic device configured to communicate with the first and second sensors, wherein the logic device is configured to:
receive first and second series of pose measurements corresponding to sensor data provided by the respective first and second sensors;
determine a set of intermediate calibration transformation estimates corresponding to the first and second sensors based, at least in part, on the first and second series of pose measurements, wherein determining the set of intermediate calibration transformation estimates comprises:
generating a set of pose windows each comprising a time-contiguous subset of the first and second series of pose measurements; and
determining, for each pose window of the set of pose windows, an intermediate calibration transformation estimate of the set of intermediate calibration transformation estimates based on the corresponding time-contiguous subset of the first and second series of pose measurements; and

determine an ongoing calibration transformation estimate corresponding to the first and second sensors based, at least in part, on the determined set of intermediate calibration transformation estimates; and
calibrate data from at least one of the first sensor or the second sensor to a coordinate frame of the mobile structure based on the ongoing calibration transformation estimate.

13. The system of claim 12 , wherein:
the first sensor comprises a first camera with a first field of view and the second sensor comprises a second camera with a second field of view different from the first field of view; and
the sensor data comprises first image data captured by the first camera substantially synchronously relative to second image data captured by the second camera.

14. The system of claim 12 , wherein:
the first sen

Assignees

• FLIR Belgium BVBA

Inventors

• Walters Celyn
• Bowden Richard
• Mendez Maldonado Oscar
• Hadfield Simon

Classifications

• G01S7/40

  Means for monitoring or calibrating · CPC title

• G01S13/88

  Radar or analogous systems specially adapted for specific applications (electromagnetic prospecting or detecting of objects, e.g. near-field detection, G01V3/00) · CPC title

• B63B2035/009

  Wind propelled vessels comprising arrangements, installations or devices specially adapted therefor, other than wind propulsion arrangements, installations, or devices, such as sails, running rigging, or the like, and other than sailboards or the like or related equipment · CPC title

• G01S17/88

  Lidar systems specially adapted for specific applications · CPC title

• B63H25/04

  automatic, e.g. reacting to compass · CPC title