Automatically collecting training data for object recognition with 3D lidar and localization

Patent metadata

Field	Value
Publication number	US-10262234-B2
Application number	US-201715495653-A
Country	US
Kind code	B2
Filing date	Apr 24, 2017
Priority date	Apr 24, 2017
Publication date	Apr 16, 2019
Grant date	Apr 16, 2019

Abstract

In one embodiment, 3D LIDAR data points are collected using a 3D LIDAR device mounted on an ADV, while the ADV is driving within a predetermined proximity. GPS information associated with a number of objects that are located and moving within the proximity surrounding the ADV. The GPS information of the objects may include a location, a speed, and a heading direction of the objects captured at a particular point in time. The objects are associated with at least some of the LIDAR data points based on the GPS information of the objects. The 3D LIDAR data points are then labeled based on a type of the objects, wherein the labeled 3D LIDAR data points are utilized to train a machine-learning algorithm or model to be utilized for object recognition by ADVs.

First claim

What is claimed is:

1. A computer-implemented method for training object recognition of an autonomous driving vehicle, the method comprising:
receiving three-dimensional (3D) light detection and range (LIDAR) data points using a 3D LIDAR device installed within an autonomous driving vehicle (ADV), wherein the 3D LIDAR data points include respective coordinates of the 3D LIDAR data points and respective timestamps of capturing the 3D LIDAR data points;
respectively receiving global positioning system (GPS) messages from a set of objects located within a predetermined proximity surrounding the ADV, wherein each of the GPS messages includes a location of a corresponding object and a timestamp of capturing the location;
for each of the received GPS messages,
searching within the 3D LIDAR data points to match the GPS message with at least one of the 3D LIDAR data points by matching the timestamp of the GPS message and the timestamps of the at least one of the 3D LIDAR data points, and matching the location of the object included in the GPS message with the coordinates of the at least one of the 3D LIDAR data points; and
labeling the at least one of the 3D LIDAR data points based on a type of object specified within the GPS message;

wherein the labeled 3D LIDAR data points are utilized to train a machine-learning algorithm for a plurality of ADVs to recognize other objects in real-time.

2. The method of claim 1 , wherein each of the objects is equipped with a GPS receiver to receive GPS signals from a satellite or a basestation, the GPS signals including geographic location information of the objects.

3. The method of claim 2 , wherein receiving the GPS messages of the objects comprises receiving wireless messages from the objects, wherein the wireless messages include GPS information of the objects.

4. The method of claim 3 , wherein each of the objects is equipped with a wireless transmitter to transmit a wireless message representing the GPS signals to the ADV.

5. The method of claim 3 , wherein the wireless messages further include an object identifier (ID) identifying a type of a corresponding object.

6. The method of claim 5 , wherein the type of an object is one of a vehicle, a pedestrian, or a static obstacle.

7. The method of claim 1 , wherein the objects are associated with the 3D LIDAR data points based on the coordinates of the 3D LIDAR data points and the GPS messages of the objects.

8. The method of claim 1 , wherein each of the 3D LIDAR data points includes information representing a light strength of a reflection from a light beam aimed at an object from the 3D LIDAR device.

9. A non-transitory machine-readable medium having instructions stored therein, which when executed by a processor, cause the processor to perform operations, the operations comprising:
receiving three-dimensional (3D) light detection and range (LIDAR) data points using a 3D LIDAR device installed within an autonomous driving vehicle (ADV), wherein the 3D LIDAR data points include respective coordinates of the 3D LIDAR data points and respective timestamps of capturing the 3D LIDAR data points;
respectively receiving global positioning system (GPS) messages from a set of objects located within a predetermined proximity surrounding the ADV, wherein each of the GPS messages includes a location of a corresponding object and a timestamp of capturing the location;
for each of the received GPS messages,
searching within the 3D LIDAR data points to match the GPS message with at least one of the 3D LIDAR data points by matching the timestamp of the GPS message and the timestamps of the at least one of the 3D LIDAR data points, and matching the location of the object included in the GPS message with the coordinates of the at least one of the 3D LIDAR data points; and
labeling the at least one of the 3D LIDAR data points based on a type of object specified within the GPS message;

wherein the labeled 3D LIDAR data points are utilized to train a machine-learning algorithm for a plurality of ADVs to recognize other objects in real-time.

10. The machine-readable medium of claim 9 , wherein each of the objects is equipped with a GPS receiver to receive GPS signals from a satellite or a basestation, the GPS signals including geographic location information of the objects.

11. The machine-readable medium of claim 10 , wherein receiving the GPS messages of the objects comprises receiving wireless messages from the objects, wherein the wireless messages include GPS information of the objects.

12. The machine-readable medium of claim 11 , wherein each of the objects is equipped with a wireless transmitter to transmit a wireless message representing the GPS signals to the ADV.

13. The machine-readable medium of claim 11 , wherein the wireless messages further include an object identifier (ID) identifying a type of a corresponding object.

14. The machine-readable medium of claim 13 , wherein the type of an object is one of a vehicle, a pedestrian, or a static obstacle.

15. The machine-readable medium of claim 9 , wherein the objects are associated with the 3D LIDAR data points based on the coordinates of the 3D LIDAR data points and the GPS messages of the objects.

16. The machine-readable medium of claim 9 , wherein each of the 3D LIDAR data points includes information representing a light strength of a reflection from a light beam aimed at an object from the 3D LIDAR device.

17. A data processing system, comprising:
a processor; and
a memory coupled to the processor to store instructions, which when executed by the processor, cause the processor to perform operations, the operations including
receiving three-dimensional (3D) light detection and range (LIDAR) data points using a 3D LIDAR device installed within an autonomous driving vehicle (ADV), wherein the 3D LIDAR data points include respective coordinates of the 3D LIDAR data points and respective timestamps of capturing the 3D LIDAR data points;
respectively receiving global positioning system (GPS) messages from a set of objects located within a predetermined proximity surrounding the ADV, wherein each of the GPS messages includes a location of a corresponding object and a timestamp of capturing the location;
for each of the received GPS messages,
searching within the 3D LIDAR data points to match the GPS message with at least one of the 3D LIDAR data points by matching the timestamp of the GPS message and the timestamps of the at least one of the 3D LIDAR data points, and matching the location of the object included in the GPS message with the coordinates of the at least one of the 3D LIDAR data points; and
labeling the at least one of the 3D LIDAR data points based on a type of object specified within the GPS message;

wherein the labeled 3D LIDAR data points are utilized to train a machine-learning algorithm for a plurality of ADVs to recognize other objects in real-time.

18. The system of claim 17 , wherein each of the objects is equipped with a GPS receiver to receive GPS signals from a satellite or a basestation, the GPS signals including geographic location information of the objects.

19. The system of claim 18 , wherein receiving the GPS messages of the objects comprises receiving wireless messages from the objects, wherein the wireless messages include GPS information of the objects.

20. The system of claim 19 , wherein each of the objects is equipped with a wireless transmitter to transmit a wireless message representing the GPS signals to the ADV.

21. Th

Assignees

• Baidu Usa Llc

Inventors

• Li Dong
• Tao Jiaming
• Hu Jiangtao
• Jiang Yifei

Classifications

• G06V10/774

  Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06F18/214

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G01S7/4802 Primary

  using analysis of echo signal for target characterisation; Target signature; Target cross-section · CPC title

• G01S17/89

  for mapping or imaging · CPC title

• G01S19/14

  specially adapted for specific applications · CPC title