Calibration for autonomous vehicle operation

Patent metadata

Field	Value
Publication number	US-10832502-B2
Application number	US-201815877899-A
Country	US
Kind code	B2
Filing date	Jan 23, 2018
Priority date	Nov 4, 2015
Publication date	Nov 10, 2020
Grant date	Nov 10, 2020

Abstract

Various embodiments relate generally to autonomous vehicles and associated mechanical, electrical and electronic hardware, computer software and systems, and wired and wireless network communications to provide an autonomous vehicle fleet as a service. In particular, a method may include receiving data associated with a sensor measurement of a perceived object, determining a label associated with the perceived object based on an initial calibration, retrieving log file data associated with the label, determining a calibration parameter associated with the sensor measurement based on the retrieved log file data, and storing the calibration parameter in association with a sensor associated with the sensor measurement. Sensors may be calibrated on the fly while the autonomous vehicle is in operation using one or more other sensors and/or fused data from multiple types of sensors.

First claim

What is claimed:

1. A system, comprising:
an image sensor disposed about an autonomous vehicle and configured to sense an environment and capture image data;
a LIDAR sensor configured to capture LIDAR data; and
one or more processors communicatively coupled to the image sensor and the LIDAR sensor, the one or more processors being configured to perform operations comprising:
receiving a signal from the image sensor, the signal comprising data representing a sensor measurement of the environment;
determining, based at least in part on the data, that the image sensor is miscalibrated, wherein determining that the image sensor is miscalibrated comprises:
determining, based at least in part on the LIDAR data, a set of detected edges associated with an object in the environment; and
determining region data associated with the set of detected edges;

generating an expected sensor measurement, the expected sensor measurement associated with the image sensor;
generating a calibration parameter, associated with the image sensor, based at least in part on the expected sensor measurement; and
modifying a parameter of the image sensor based at least in part on the calibration parameter and a parameter associated with the region data.

2. The system of claim 1 ,
wherein the LIDAR sensor is a first LIDAR sensor,
wherein the system further comprises a plurality of additional LIDAR sensors, and
the operations further comprising determining that the first LIDAR sensor is miscalibrated, and wherein determining that the first LIDAR sensor is miscalibrated comprises:
determining a first sensor measurement of the first LIDAR sensor comprising a first distance or a first reflectivity value;
determining a plurality of additional sensor measurements of the plurality of additional LIDAR sensors comprising a second distance or a second reflectivity value, the first sensor measurement and the plurality of additional sensor measurements being associated with an object in the environment; and
determining a difference between the first sensor measurement and at least one measurement of the plurality of additional sensor measurements, and

the operations further comprising modifying a parameter of the first LIDAR sensor, based at least in part on the difference.

3. The system of claim 1 ,
wherein modifying the parameter of the image sensor comprises adjusting a focus parameter or a lens parameter of the image sensor.

4. The system of claim 1 ,
wherein the system further comprises an IMU configured to capture IMU sensor data and a GPS configured to capture GPS sensor data, and
wherein generating the expected sensor measurement comprises:
fusing at least two of the image data, the LIDAR data, the IMU sensor data, or the GPS sensor data to generate fused data;
receiving, based at least in part on the fused data, map data, the map data indicative of an object in the environment having known properties; and
generating the expected sensor measurement based at least in part on the object.

5. The system of claim 1 , wherein generating the calibration parameter comprises:
receiving additional data from a log file, a map, or an additional sensor disposed about the system; and
generating, based at least in part on the additional data, the calibration parameter, wherein the calibration parameter is configured to alter an intrinsic value associated with the image sensor.

6. The system of claim 5 , wherein the additional data comprises an image captured by a camera, a velocity of the system captured by an odometry sensor, a position captured by a global positioning (GPS) sensor, audio data captured by a microphone, sound navigation and ranging (SONAR) data captured by a SONAR sensor, radio detection and ranging (RADAR) data captured by a RADAR sensor, direction of gravity data captured by an inertial measurement unit (IMU), or a direction of travel data.

7. The system of claim 1 , wherein the operations further comprise:
generating, based at least in part on the calibration parameter, a vehicle trajectory, the vehicle trajectory configured to cause the system to traverse a portion of the environment.

8. The system of claim 1 , wherein the operations further comprise:
generating a first probabilistic model; and
generating, based at least in part on a heuristic rule, a second probabilistic model,
wherein generating the calibration parameter comprises using the first probabilistic model, and
wherein modifying the image sensor comprises using the second probabilistic model.

9. A method, comprising:
receiving a plurality of sensor measurements from a plurality of sensors disposed about a vehicle, wherein the plurality of sensors comprise a LIDAR sensor configured to capture LIDAR sensor data and an image sensor configured to capture image sensor data;
identifying, based at least in part on the plurality of sensor measurements, an anomalous sensor measurement of the plurality of sensor measurements associated with the image sensor of the plurality of sensors, wherein identifying the anomalous sensor measurement comprises:
determining a set of edges in the LIDAR sensor data;
associating, as region data, at least a portion of the image sensor data with the LIDAR sensor data; and
identifying a portion of the region data that is anomalous;

determining, based at least in part on the anomalous sensor measurement, that the image sensor is miscalibrated;
generating an expected sensor measurement, associated with the image sensor; and
generating, based at least in part on the expected sensor measurement, a calibration parameter for calibrating the image sensor.

10. The method of claim 9 , further comprising:
generating fused sensor data using data received from the LIDAR sensor and the image sensor, the fused sensor data indicative of a localization of the vehicle; and
generating a probabilistic map of an environment based at least in part on the fused sensor data, the probabilistic map including a plurality of labeled objects and a probability score assigned to at least one object of the plurality of labeled objects, the plurality of labeled objects being associated with object parameters,
wherein identifying the anomalous sensor measurement comprises comparing data received from the image sensor or the LIDAR sensor with the object parameters.

11. The method of claim 10 , wherein generating the probabilistic map comprises:
accessing historical sensor data corresponding to the plurality of sensors; and
generating the probability score based at least in part on the historical sensor data and the anomalous sensor measurement.

12. The method of claim 9 , wherein identifying the portion of the region data that is anomalous comprises
identifying a portion of the region data that is blurry.

13. The method of claim 9 , wherein the LIDAR sensor is configured to capture a first sensor measurement and an additional LIDAR sensor is configured to capture a second sensor measurement,
the first sensor measurement and the second sensor measurement associated with a distance of an object to the vehicle or a reflectivity of the object, and
wherein identifying the anomalous sensor measurement comprises comparing the first sensor measurement and the second sensor measurement.

14. The method of claim 9 , wherein the vehicle is an autonomous vehicle, the method further comprising:
generating, based on the calibration parameter, a trajectory configured to cause the autonomous vehicle to traverse an environment, and
transmitting the trajectory to the autonomous vehicle.

15. One or more non-transitory computer readable m

Assignees

• Zoox Inc

Inventors

• Levinson Jesse Sol
• Sibley Gabriel Thurston
• Douillard Bertrand Robert

Classifications

• B60Q1/508

  specific to vehicles driving in fleets or convoys · CPC title

• B60Q1/507

  specific to autonomous vehicles · CPC title

• B60Q1/30

  for indicating rear of vehicle, e.g. by means of reflecting surfaces · CPC title

• G01S15/86

  Combinations of sonar systems with lidar systems; Combinations of sonar systems with systems not using wave reflection · CPC title

• G01S15/931

  of land vehicles · CPC title