Systems and methods for validating sensor calibration

Patent metadata

Field	Value
Publication number	US-12067786-B2
Application number	US-202217888563-A
Country	US
Kind code	B2
Filing date	Aug 16, 2022
Priority date	Dec 10, 2018
Publication date	Aug 20, 2024
Grant date	Aug 20, 2024

Abstract

Among other things, we describe systems and method for validating sensor calibration. For validating calibration of a system of sensors having several types of sensors, an object may be configured to have a substantially reflective portion such that the sensors can isolate the substantially reflective portion, and their sensor data can be compared to determine, if the detected locations of the substantially reflective portion by each sensor are aligned. For calibrating a system of sensors, an object having known calibration features can be used and detected by each sensor, and the detected data can be compared to known calibration data associated with the object to determine if each sensor is correctly calibrated.

First claim

What is claimed is:

1. A system, comprising:
at least one light detection and ranging sensor configured to detect a distance to at least one location associated with a substantially reflective portion of at least one object;
at least one imaging sensor configured to detect light intensity values associated with the substantially reflective portion of the at least one location of the at least one object;
a computer-readable medium storing computer-executable instructions; and
at least one processor communicatively coupled to the at least one light detection and ranging sensor and the at least one imaging sensor and configured to execute the computer-executable instructions stored on the computer-readable medium, wherein when the at least one processor executes the instructions, the at least one processor carries out operations to:
receive sensor data associated with the detected distance to the at least one location;
generate a first predicted aggregate location associated with the substantially reflective portion based on the sensor data associated with the detected distance to the at least one location,
receive sensor data associated with the detected intensity values of the at least one location,
generate a second predicted aggregate location associated with the reflective portion based on the sensor data associated with the detected intensity values of the at least one location; and
determine an alignment error value based on the first predicted aggregate location and the second predicted aggregate location.

2. The system of claim 1 , wherein the computer-executable instructions comprise a first algorithm for generating the first predicted aggregate location and a second algorithm for generating the second predicted aggregate location, the second algorithm being different from the first algorithm, wherein at least one of the first algorithm and the second algorithm is modified when the alignment error value is greater than a first alignment error threshold.

3. The system of claim 2 , wherein at least one of the first algorithm and the second algorithm is modified such that the generated first predicted aggregate location and the generated second predicted aggregate location are substantially aligned.

4. The system of claim 1 , wherein when the at least one processor executes the instructions, the at least one processor carries out operations to initiate a calibration process of the at least one image capture sensor or the at least one light detection and ranging sensor when the alignment error is more than a second alignment error threshold.

5. The system of claim 1 , wherein the at least one object is further configured to have a substantially non-reflective portion.

6. The system of claim 1 , wherein the substantially reflective portion comprises at least one of a substantially black surface or a substantially white surface.

7. A method comprising:
receiving, with at least one processor, sensor data associated with a detected distance to at least one location from a first sensor, wherein the detected distance is to at least one location associated with a substantially reflective portion of at least one object;
generating, with the at least one processor, a first predicted aggregate location associated with the substantially reflective portion based on the sensor data associated with the detected distance to the at least one location;
receiving, with the at least one processor, sensor data associated with detected intensity values of the at least one location, wherein the detected intensity values are light intensity values at the at least one location associated with the substantially reflective portion of the at least one object;
generating, with the at least one processor, a second predicted aggregate location associated with the reflective potion based on the sensor data associated with the detected intensity values of the at least one location; and
determining, with the at least one processor, an alignment error value based on the first predicted aggregate location and the second predicted aggregate location.

8. The method of claim 7 , wherein the first predicted aggregate location is generated using a first algorithm and the second predicted aggregate location is generated using a second algorithm, the method further comprising modifying at least one of the first algorithm and the second algorithm when the alignment error value is greater than a first alignment error threshold.

9. The method of claim 8 , further comprising modifying at least one of the first algorithm and the second algorithm such that the generated first predicted aggregate location and the generated second predicted aggregate location are substantially aligned.

10. The method of claim 7 , comprising calibrating at least one of the at least one image capture sensor and the at least one light detection and ranging sensor when the alignment error is more than a second alignment threshold.

11. The method of claim 7 , wherein the sensor data associated with the detected distance is LiDAR data.

12. The method of claim 7 , wherein the sensor data associated with the detected distance is radar data.

13. The method of claim 7 , wherein the sensor data associated with the detected intensity values is camera data.

14. The method of claim 7 , comprising initiating a calibration process when the alignment error value is greater than a predetermined threshold, wherein the calibration process comprises computing a corrected parameter for at least one intrinsic parameter associated with the first sensor or the second sensor.

15. At least one non-transitory storage media storing instructions that, when executed by at least one processor, cause the at least one processor to:
receive sensor data associated with a detected distance to at least one location from a first sensor, wherein the detected distance is to at least one location associated with a substantially reflective portion of at least one object;
generate a first predicted aggregate location associated with the substantially reflective portion based on the sensor data associated with the detected distance to the at least one location;
receive sensor data associated with detected intensity values of the at least one location, wherein the detected intensity values are light intensity values at the at least one location associated with the substantially reflective portion of the at least one object;
generate a second predicted aggregate location associated with the reflective potion based on the sensor data associated with the detected intensity values of the at least one location; and
determine an alignment error value based on the first predicted aggregate location and the second predicted aggregate location.

16. The at least one non-transitory storage media of claim 15 , wherein the first predicted aggregate location is generated using a first algorithm and the second predicted aggregate location is generated using a second algorithm, the method further comprising modifying at least one of the first algorithm and the second algorithm when the alignment error value is greater than a first alignment error threshold.

17. The at least one non-transitory storage media of claim 16 , further comprising modifying at least one of the first algorithm and the second algorithm such that the generated first predicted aggregate location and the generated second predicted aggregate location are substantially aligned.

18. The at least one non-transitory storage media of claim 15 , wherein the sensor data associated with the detected distance

Assignees

• Motional Ad Llc

Inventors

• Aguiar Mauro
• Weinstein Noam
• Wolff Eric
• Sapuan Matthias
• Chang Hsun-Hsien
• Robbel Philipp
• Di Cicco Maurilio
• Ozbilgin Guchan
• Roy Bishwamoy Sinha
• Yang Yifan
• Bhagat Akshay
• Butterworth David
• Eick Andrew J
• Sharma Alok
• Wei Junqing

Classifications

• G05D1/81

  Handing over between on-board automatic and on-board manual control · CPC title

• G02B6/0055

  Reflecting element, sheet or layer · CPC title

• G06T7/80

  Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration · CPC title

• G05D1/0088

  characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours (using knowledge based models G06N5/00) · CPC title

• G01S17/89

  for mapping or imaging · CPC title