Lidar and camera data fusion for automated vehicle

Patent metadata

Field	Value
Publication number	US-10630913-B2
Application number	US-201715499076-A
Country	US
Kind code	B2
Filing date	Apr 27, 2017
Priority date	Apr 27, 2017
Publication date	Apr 21, 2020
Grant date	Apr 21, 2020

Abstract

A data-fusion system that fuses lidar-data and camera-data for an automated vehicle includes a camera, a lidar, and a controller. The camera renders an image of an object proximate to a host-vehicle. The lidar detects a distance and a direction to the object based on a reflected-signal of light reflected by the object. The controller is in communication with the camera and the lidar. The controller is configured to determine a reflectivity-characteristic of the object based on the image and the reflected-signal, and adjust a detection-characteristic of the lidar when the reflectivity-characteristic of the object makes it difficult for the lidar to detect the distance and the direction to the object.

First claim

We claim:

1. A system comprising:
a camera of a host-vehicle that renders an image of an object proximate to the host-vehicle;
a light detection and ranging (lidar) sensor of the host-vehicle that transmits light-energy that impinges the object;
a controller in communication with the camera and the lidar, said controller configured to:
determine, using the image, a reflection-fraction of the light-energy that impinged on the object and is reflected by the object towards the lidar;
compare the reflection-fraction with a fraction-threshold; and
in accordance with a result of the comparing, adjust a detection-characteristic of the lidar.

2. The system in accordance with claim 1 , wherein the reflection-fraction is also determined based on a signal-strength of the reflected light-energy detected by the lidar.

3. The system in accordance with claim 1 , wherein the reflection-fraction is less than the fraction-threshold when a surface of the object is characterized as non-reflective.

4. The system in accordance with claim 1 , wherein the reflection-fraction is less than the fraction-threshold when a color of the object is black.

5. The system in accordance with claim 1 , wherein the reflection-fraction is less than the fraction-threshold when a surface of the object is characterized as highly-reflective.

6. The system in accordance with claim 1 , wherein the reflection-fraction is less than the fraction-threshold when a surface of the object is chrome.

7. The system in accordance with claim 1 , wherein the detection-characteristic includes a detector-gain of the lidar, and the detector-gain is increased when the reflection-fraction is less than the fraction-threshold.

8. The system in accordance with claim 1 , wherein the detection-characteristic includes a detection-threshold of the lidar, and the detection-threshold is decreased when the reflection-fraction is less than the fraction-threshold.

9. The system in accordance with claim 1 , wherein the reflection-fraction is determined using a strength of the reflected light-energy detected in the image.

10. The system in accordance with claim 1 , wherein the fraction threshold is adjusted according to environmental conditions.

11. The system in accordance with claim 1 , wherein adjusting the detection-characteristic of the lidar includes increasing a detector-gain or a detector-threshold of the lidar to better detect a distance or direction to the object.

12. The system in accordance with claim 1 , wherein adjusting the detection-characteristic of the lidar includes adjusting a portion of a field-of-view of the lidar that corresponds to a bounding-box that is placed around the object by an object-detector.

13. A method comprising:
rendering, using a camera of a host-vehicle, an image of an object proximate to the host-vehicle;
transmitting, by a light detection and ranging (lidar) sensor of the host-vehicle, light-energy that impinges the object;
determining, by a controller of the host-vehicle, a reflection-fraction of the light-energy that impinged on the object and is reflected by the object towards the lidar, the reflection-fraction determined using the image;
comparing, by the controller, the reflection-fraction with a fraction-threshold; and
in accordance with the reflection-fraction being less than the fraction-threshold,
adjusting a detection-characteristic of the lidar.

14. The method in accordance with claim 13 , wherein the reflection-fraction is also determined based on a signal-strength of the reflected light-energy.

15. The method in accordance with claim 13 , wherein the reflection-fraction is less than the fraction-threshold when a surface of the object is characterized as non-reflective.

16. The method in accordance with claim 13 , wherein the reflection-fraction is less than the fraction-threshold when a color of the object is black.

17. The method in accordance with claim 13 , wherein the reflection-fraction is less than the fraction-threshold when a surface of the object is characterized as highly-reflective.

18. The method in accordance with claim 13 , wherein the reflection-fraction is less than the fraction-threshold when a surface of the object is chrome.

19. The method in accordance with claim 13 , wherein the detection-characteristic includes a detector-gain of the lidar, and the detector-gain is increased when the reflection-fraction is less than the fraction-threshold.

20. The method in accordance with claim 13 , wherein the detection-characteristic includes a detection-threshold of the lidar, and the detection-threshold is decreased when the reflection-fraction is less than the fraction-threshold.

21. The method in accordance with claim 13 , wherein the reflection-fraction is determined using a strength of the reflected light-energy detected in the image.

22. The method in accordance with claim 13 , wherein the fraction threshold is adjusted according to environmental conditions.

23. The method in accordance with claim 13 , wherein adjusting the detection-characteristic of the lidar includes increasing a detector-gain or a detector-threshold of the lidar to better detect a distance or direction to the object.

24. The method in accordance with claim 13 , wherein adjusting the detection-characteristic of the lidar includes adjusting a portion of a field-of-view of the lidar that corresponds to a bounding-box that is placed around the object by an object-detector.

Assignees

• Aptiv Tech Ltd

Inventors

• Wei Junqing
• Kim Junsung
• Lee Jong Ho

Classifications

• G01S17/42

  Simultaneous measurement of distance and other co-ordinates (indirect measurement G01S17/46) · CPC title

• G01S17/931 Primary

  of land vehicles · CPC title

• G05D1/0236

  in combination with a laser (lasers per se H01S) · CPC title

• G01S7/493

  Extracting wanted echo signals · CPC title

• G01S7/481

  Constructional features, e.g. arrangements of optical elements · CPC title