Operating a motor vehicle with stereo camera and LIDAR

Patent metadata

Field	Value
Publication number	US-12236629-B2
Application number	US-202017037127-A
Country	US
Kind code	B2
Filing date	Sep 29, 2020
Priority date	Sep 30, 2019
Publication date	Feb 25, 2025
Grant date	Feb 25, 2025

Abstract

A request signal that indicates a quality for a determination of an orientation of a road user is received. The orientation of the road user is determined based on a) image data when the request signal indicates the quality is below a predetermined quality for the determination of the orientation of the road user, or b) on LIDAR data and image data when the request signal indicates the quality is the predetermined quality.

First claim

The invention claimed is:

1. A method for operating a motor vehicle, comprising:
capturing sensor data that includes image data, via a stereo camera system, and light detection and ranging (LIDAR) data, via a LIDAR sensor;
receiving a request signal that indicates a quality of the sensor data for a determination of an orientation of a road user; and
determining the orientation of the road user based on a relevant portion of the LIDAR data and the image data when the request signal indicates the quality of the sensor data is at or above the predetermined quality; wherein the relevant portion of the LIDAR data is determined based on the image data.

2. The method according to claim 1 , further comprising inputting the image data into a machine learning program that outputs the orientation of the road user.

3. The method according to claim 2 , further comprising training the machine learning program with training data based on Computer-Assisted Design (CAD) data sets.

4. The method according to claim 1 , further comprising;
inputting the LIDAR data to an iterative closest point algorithm that outputs an orientation of the road user based on LIDAR data;
inputting the image data into a machine learning program that outputs an orientation of the road user based on image data; and
combining the outputs of the iterative closest point algorithm and the machine learning program to determine the orientation of the road user.

5. The method according to claim 1 , further comprising determining the orientation of the road user additionally based on receiving, from another road user, the orientation of the road user.

6. The method according to claim 1 , further comprising actuating a vehicle actuator based on the orientation of the road user.

7. The method according to claim 1 , further comprising determining the quality based on an operation of a motor vehicle.

8. The method according to claim 1 , wherein the quality is a measure of a quantity of data.

9. The method according to claim 1 , wherein the road user is a motor vehicle.

10. A system for a motor vehicle, comprising:
a stereo camera system configured to capture image data;
a light detection and ranging (LIDAR) sensor configured to capture LIDAR data; and
a control device configured to:
receive a request signal that indicates a quality of the sensor data for a determination of an orientation of a road user; and
determine the orientation of the road user based on a relevant portion of the LIDAR data and the image data when the request signal indicates the quality of the sensor data is at or above the predetermined quality; wherein the relevant portion of the LIDAR data is determined based on the image data.

11. The system according to claim 10 , wherein the control device is further configured to input the image data into a machine learning program that outputs the orientation of the road user.

12. The system according to claim 11 , wherein the control device is further configured to train the machine learning program with training data based on Computer-Assisted Design (CAD) data.

13. The system according to claim 10 , wherein the control device is further configured to:
input the LIDAR data to an iterative closest point algorithm that outputs an orientation of the road user based on LIDAR data;
input the image data into a machine learning program that outputs an orientation of the road user based on image data; and
combine the outputs of the iterative closest point algorithm and the machine learning program to determine the orientation of the road user.

14. The system according to claim 10 , wherein the control device is further configured to determine the orientation of the road user additionally based on receiving, from another road user, the orientation of the road user.

15. The system according to claim 10 , wherein the control device is further configured to actuate a vehicle actuator based on the orientation of the road user.

16. The system according to claim 10 , wherein the control device is further configured to determine the quality based on an operation of the system.

17. The system according to claim 10 , wherein the quality is a measure of a quantity of data.

18. The system according to claim 10 , wherein the road user is a second motor vehicle.

19. A method for operating a motor vehicle, comprising:
capturing sensor data that includes image data, via a stereo camera system, and light detection and ranging (LIDAR) data, via a LIDAR sensor;
receiving a request signal that indicates a quality of the sensor data for a determination of an orientation of a road user; and
determining the orientation of the road user based on the LIDAR data and the image data when the request signal indicates the quality of the sensor data is at or above the predetermined quality by
inputting the LIDAR data to an iterative closest point algorithm that outputs an orientation of the road user based on LIDAR data;
inputting the image data into a machine learning program that outputs an orientation of the road user based on image data; and
combining the outputs of the iterative closest point algorithm and the machine learning program to determine the orientation of the road user.

20. The method according to claim 19 , further comprising actuating a vehicle actuator based on the orientation of the road user.

Assignees

• Ford Global Tech Llc

Inventors

• Aslandere Turgay Isik
• Bitsanis Evangelos
• Marbaix Michael
• Chevalier Alain Marie Roger
• Stefan Frederic

Classifications

• G06N3/09

  Supervised learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G05D1/2435

  Extracting 3D information · CPC title

• G05D1/247

  using signals provided by artificial sources external to the vehicle, e.g. navigation beacons · CPC title

• G05D1/227

  Handing over between remote control and on-board control; Handing over between remote control arrangements · CPC title