Calibrating sensors mounted on an autonomous vehicle

Patent metadata

Field	Value
Publication number	US-11375119-B2
Application number	US-201916538802-A
Country	US
Kind code	B2
Filing date	Aug 12, 2019
Priority date	Oct 19, 2017
Publication date	Jun 28, 2022
Grant date	Jun 28, 2022

Abstract

A system calibrates one or more sensors mounted to an autonomous vehicle. From the one or more sensors, the system identifies a primary sensor and a secondary sensor. The system determines a reference angle for the primary sensor, and based on that reference angle for the primary sensor, a scan-start time representing a start of a scan and a scan-end time representing an end of a scan. The system receives, from the primary sensor, a primary set of scan data recorded from the scan-start time to the scan-end time. The system receives, from the secondary sensor, a secondary set of sensor data recorded from the scan-start time to the scan-end time. The system calibrates the primary and secondary sensors by determining a relative transform for transforming points between the first set of scan data and the second set of scan data.

First claim

What is claimed is:

1. A non-transitory computer readable storage medium having instructions for calibrating sensors mounted on a vehicle encoded thereon that, when executed by a processor, cause the processor to:
identify, from a plurality of sensors mounted on the vehicle, a primary sensor and a secondary sensor;
obtain a reference angle for the primary sensor;
determine, based on the reference angle of the primary sensor, a scan-start time representing a start of a scan and a scan-end time representing an end of a scan;
receive, from the primary sensor, a primary set of scan data recorded from the scan-start time to the scan-end time;
receive, from the secondary sensor, a secondary set of scan data, recorded from the scan-start time to the scan-end time;
align the primary set of scan data with the secondary set of scan data based on one or more of the scan-start time and the scan-end time; and
send signals to controls of the vehicle based on the alignment of the primary set of scan data and the secondary set of scan data.

2. The non-transitory computer readable storage medium of claim 1 , further comprising instructions that cause the processor to:
obtain a history of scan data recorded by the secondary sensor as the primary sensor periodically rotates past the reference angle; and
at each scan-start time and scan-end time received from the primary sensor, segment the history of scan data.

3. The non-transitory computer readable storage medium of claim 1 , wherein the instructions for aligning the primary set of scan data and the secondary set of scan data further cause the processor to:
identify a timestamp assigned to a portion of the secondary set of scan data associated with the reference angle; and
align the primary set of scan data with the secondary set of scan data based on the timestamp.

4. The non-transitory computer readable storage medium of claim 1 , wherein the scan-end time representing the end of a first rotation by the primary sensor is a scan-start time for a second rotation by the secondary sensor.

5. The non-transitory computer readable storage medium of claim 1 , wherein the primary sensor and the secondary sensor are light detection and ranging (LIDAR) sensors.

6. The non-transitory computer readable storage medium of claim 1 , wherein the instructions for aligning the primary set of scan data and the secondary set of scan data further cause the processor to:
identify one or more shared features recorded in both a segment of the primary set of scan data and a segment of the secondary set of scan data; and
align the segment of the primary set of scan data with the segment of the secondary set of scan data by overlapping the shared features.

7. The non-transitory computer readable storage medium of claim 1 , further comprising instructions that cause the processor to:
obtain an initial orientation of the secondary sensor, wherein the aligning the primary set of scan data with the secondary set of scan data is further based on the initial orientation.

8. A computer-implemented method for calibrating sensors mounted on a vehicle, the method comprising:
identifying, from a plurality of sensors mounted on the vehicle, a primary sensor and a secondary sensor;
obtaining a reference angle for the primary sensor;
determining, based on the reference angle of the primary sensor, a scan-start time representing a start of a scan and a scan-end time representing an end of a scan;
receiving, from the primary sensor, a primary set of scan data recorded from the scan-start time to the scan-end time;
receiving, from the secondary sensor, a secondary set of scan data, recorded from the scan-start time to the scan-end time;
aligning the primary set of scan data with the secondary set of scan data based on one or more of the scan-start time and the scan-end time; and
sending signals to controls of the vehicle based on the alignment of the primary set of scan data and the secondary set of scan data.

9. The computer-implemented method of claim 8 , further comprising:
obtaining a history of scan data recorded by the secondary sensor as the primary sensor periodically rotates past the reference angle; and
at each scan-start time and scan-end time received from the primary sensor, segmenting the history of scan data.

10. The computer-implemented method of claim 8 , wherein aligning the primary set of scan data and the secondary set of scan data further comprises:
identifying a timestamp assigned to a portion of the secondary set of scan data associated with the reference angle; and
align the primary set of scan data with the secondary set of scan data based on the timestamp.

11. The computer-implemented method of claim 8 , wherein the scan-end time, representing the end of a first rotation by the primary sensor, is a scan-start time for a second rotation by the secondary sensor.

12. The computer-implemented method of claim 8 , wherein the primary sensor and the secondary sensor are light detection and ranging (LIDAR) sensors.

13. The computer-implemented method of claim 8 , wherein aligning the primary set of scan data and the secondary set of scan data further comprises:
identifying one or more shared features recorded in both a segment of the primary set of scan data and a segment of the secondary set of scan data; and
aligning the segment of the primary set of scan data with the segment of the secondary set of scan data by overlapping the shared features.

14. The computer-implemented method of claim 8 , further comprising:
obtaining an initial orientation of the secondary sensor, wherein the aligning the primary set of scan data with the secondary set of scan data is further based on the initial orientation.

15. A computer system for calibrating sensors mounted on a vehicle, the computer system comprising:
a processor; and
a computer readable storage medium having instructions for calibrating sensors mounted on an vehicle encoded thereon that, when executed by the processor, cause the processor to:
identify, from a plurality of sensors mounted on the vehicle, a primary sensor and a secondary sensor;
obtain a reference angle for the primary sensor;
determine, based on the reference angle of the primary sensor, a scan-start time representing a start of a scan and a scan-end time representing an end of a scan;
receive, from the primary sensor, a primary set of scan data recorded from the scan-start time to the scan-end time;
receive, from the secondary sensor, a secondary set of scan data, recorded from the scan-start time to the scan-end time;
align the primary set of scan data with the secondary set of scan data based on one or more of the scan-start time, and the scan-end time; and

send signals to controls of the vehicle based on the alignment of the primary set of scan data and the secondary set of scan data.

16. The computer system of claim 15 , further comprising instructions that cause the processor to:
obtain a history of scan data recorded by the secondary sensor as the primary sensor periodically rotates past the reference angle; and
at each scan-start time and scan-end time received from the primary sensor, segment the history of scan data.

17. The computer system of claim 15 , wherein the instructions for aligning the primary set of scan data and the secondary set of scan data further cause the processor to:
identify a timestamp assigned to a portion of the secondary set of scan data recorded associated with the reference angle; and
align the primary set of scan data with the secondary set of scan data ba

Assignees

• Deepmap Inc
• Nvidia Corp

Inventors

• Wheeler Mark Damon
• Yang Lin

Classifications

• H04N23/689

  Motion occurring during a rolling shutter mode · CPC title

• G01S7/497 Primary

  Means for monitoring or calibrating · CPC title

• H04N23/60

  Control of cameras or camera modules · CPC title

• H04N23/90

  Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums · CPC title

• H04N23/54

  Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils · CPC title