Vehicle sensor calibration and localization

Patent metadata

Field	Value
Publication number	US-10983199-B2
Application number	US-201715674853-A
Country	US
Kind code	B2
Filing date	Aug 11, 2017
Priority date	Aug 11, 2017
Publication date	Apr 20, 2021
Grant date	Apr 20, 2021

Abstract

Perception sensors of a vehicle can be used for various operating functions of the vehicle. A computing device may receive sensor data from the perception sensors, and may calibrate the perception sensors using the sensor data, to enable effective operation of the vehicle. To calibrate the sensors, the computing device may project the sensor data into a voxel space, and determine a voxel score comprising an occupancy score and a residual value for each voxel. The computing device may then adjust an estimated position and/or orientation of the sensors, and associated sensor data, from at least one perception sensor to minimize the voxel score. The computing device may calibrate the sensor using the adjustments corresponding to the minimized voxel score. Additionally, the computing device may be configured to calculate an error in a position associated with the vehicle by calibrating data corresponding to a same point captured at different times.

First claim

What is claimed is:

1. A system comprising:
one or more processors; and
one or more computer readable storage media communicatively coupled to the one or more processors and storing instructions that are executable by the one or more processors to:
receive an indication that a first sensor or a second sensor of a plurality of sensors of a vehicle is out of alignment;
withhold first sensor data associated with the first sensor or second sensor data associated with the second sensor from an input to an operational control system of the vehicle;
receive the first sensor data as a first dataset from the first sensor on the vehicle and the second sensor data as a second dataset from the second sensor on the vehicle;
project the first dataset and the second dataset into a voxel space, wherein the voxel space comprises a three-dimensional representation comprising a plurality of voxels;
identify occupied voxels of the plurality of voxels in the voxel space;
calculate a residual value associated with individual voxels of the occupied voxels in the voxel space, wherein the residual value is based at least in part on information associated with the three-dimensional representation of the individual voxels of the occupied voxels;
adjust a position of the first dataset or the second dataset in the voxel space to optimize a magnitude of the residual value;
determine a sensor transformation corresponding to an optimized voxel score of the voxel space;
calibrate at least one of the first sensor or the second sensor based at least in part on the sensor transformation; and
responsive to a calibration of the at least one of the first sensor or the second sensor, send the first sensor data or the second sensor data to the operational control system.

2. The system of claim 1 , wherein the instructions are further executable by the one or more processors to:
determine an occupancy penalty for each voxel, wherein an occupancy score is determined based at least in part on the occupancy penalty; and
minimize the occupancy score for each voxel in the voxel space by reprojecting datapoints corresponding to at least a subset of the second dataset based at least in part on the sensor transformation.

3. The system of claim 1 , wherein the first sensor and the second sensor are a same sensor.

4. The system of claim 1 , wherein the instructions are further executable by the one or more processors to:
determine the residual value based on one or more of a smallest eigenvalue resulting from an eigenvalue decomposition or a distance of datapoints in the first dataset from a plane representing at least the second dataset when determining the sensor transformation.

5. The system of claim 1 , wherein the instructions are further executable by the one or more processors to:
receive input from at least one of a motion sensor, a navigation sensor, an image sensor, or a depth sensor;
determine a trajectory of the vehicle based at least in part on the input; and
determine a region of interest based at least in part on the trajectory of the vehicle,
wherein the first dataset or the second dataset is collected in the region of interest.

6. The system of claim 5 , wherein the region of interest comprises a central location and a time window centered on the central location, wherein the central location is identified based on at least one of:
a yaw rate of the vehicle;
a radius of curvature of a path of the vehicle;
a position of the vehicle; or
a speed of the vehicle.

7. The system of claim 1 , wherein the instructions are further executable by the one or more processors to:
calculate drift error associated with a position of the vehicle; or
identify an error associated with a trajectory of the vehicle.

8. The system of claim 1 , wherein the instructions are further executable by the one or more processors to:
reduce a size of the voxels in the voxel space; and
increase a number of voxels which populate the voxel space.

9. The system of claim 1 , wherein the instructions are further executable by the one or more processors to:
calculate a second voxel score associated with the voxel space; and
determine a second sensor transformation corresponding to a second optimized voxel score,
wherein the calibration of the first sensor and the second sensor is based at least in part on the second sensor transformation.

10. A method comprising:
receiving an indication that a first sensor of a plurality of sensors of a vehicle is out of alignment;
withholding first sensor data associated with the first sensor from an input to an operational control system of the vehicle;
receiving a dataset from the plurality of sensors of the vehicle;
projecting the dataset into a voxel space;
perturbing the first sensor data of the dataset to reduce a residual value associated with the voxel space, wherein the residual value is based at least in part on information associated with the voxel space;
determining a sensor transformation to the first sensor data that corresponds to a minimized residual value associated with the voxel space;
calibrating the first sensor of the plurality of sensors based at least in part on the sensor transformation; and
responsive to a calibration of the first sensor, sending the first sensor data to the operational control system.

11. The method of claim 10 , wherein respective residual values for respective voxels of the voxel space comprise a sum of respective occupancy scores and respective residual values, the respective residual values being associated with one of an eigenvalue or a distance to a plane.

12. The method of claim 10 , further comprising:
receiving an indication that the vehicle is located at a reference point at a first time;
capturing a first collective dataset at the first time, the first collective dataset comprising a first metaspin associated with at least a portion of the plurality of sensors and corresponding to an environment at the reference point;
receiving an indication that the vehicle is located at the reference point at a second time;
capturing a second collective dataset at the second time, the second collective dataset comprising a second metaspin associated with the at least a portion of the plurality of sensors and corresponding to the environment at the reference point;
projecting the first collective dataset and the second collective dataset into the voxel space corresponding to the environment at the reference point;
perturbing one or more dimensions of the second collective dataset to minimize a residual value associated with the voxel space; and
determining a positioning error between the first time and the second time.

13. The method of claim 10 , further comprising:
calculating a trajectory of the vehicle based on at least one of motion sensor data or navigation sensor data;
determining that the trajectory corresponds to a region of interest; and
identifying a time window associated with the region of interest,
wherein the first sensor data comprises data collected in the region of interest.

14. The method of claim 13 , wherein a determination that the trajectory corresponds to the region of interest is based on at least one of:
a yaw rate of the vehicle;
a radius of curvature of a path of the vehicle;
a position of the vehicle; or
a speed of the vehicle.

15. The method of claim 10 , further comprising:
determining that a size of voxels in the voxel space is not a minimum size;
reducing the size of voxels in the voxel space;
increasing a number of voxels in the voxel space; and
iteratively determining the sensor transformat

Assignees

• Zoox Inc

Inventors

• Adams Derek
• Baldwin Ian
• Douillard Bertrand Robert
• Levinson Jesse Sol

Classifications

• G01S17/89

  for mapping or imaging · CPC title

• G01S17/931

  of land vehicles · CPC title

• G01S7/4972 Primary

  Alignment of sensor · CPC title

• G01S17/42

  Simultaneous measurement of distance and other co-ordinates (indirect measurement G01S17/46) · CPC title

• G01C25/00

  Manufacturing, calibrating, cleaning, or repairing instruments or devices referred to in the other groups of this subclass (testing, calibrating or compensating compasses G01C17/38) · CPC title