Method for determining anchor boxes for training neural network object detection models for autonomous driving

Patent metadata

Field	Value
Publication number	US-11055540-B2
Application number	US-201916457820-A
Country	US
Kind code	B2
Filing date	Jun 28, 2019
Priority date	Jun 28, 2019
Publication date	Jul 6, 2021
Grant date	Jul 6, 2021

Abstract

In one embodiment, a set of bounding box candidates are plotted onto a 2D space based on their respective dimension (e.g., widths and heights). The bounding box candidates are clustered on the 2D space based on the distribution density of the bounding box candidates. For each of the clusters of the bounding box candidates, an anchor box is determined to represent the corresponding cluster. A neural network model is trained based on the anchor boxes representing the clusters. The neural network model is utilized to detect or recognize objects based on images and/or point clouds captured by a sensor (e.g., camera, LIDAR, and/or RADAR) of an autonomous driving vehicle.

First claim

What is claimed is:

1. A computer-implemented method for determining anchor boxes for training a neural network object detection model for autonomous driving, the method comprising:
plotting a plurality of bounding boxes in a two-dimensional (2D) space based on their respective dimensions;
clustering the plurality of bounding boxes into one or more clusters of the bounding boxes based on a distribution density of the bounding boxes on the 2D space;
for each of the clusters of bounding boxes, determining an anchor box to represent the corresponding cluster;
determining whether a distribution of the bounding boxes assigned to the anchor boxes satisfies a predetermined condition;
in response to determining that the predetermined condition has not been satisfied, adjusting a dimension of at least one of the anchor boxes on the 2D space; and
training a neural network model for detecting objects using the anchor boxes, wherein the neural network model is utilized to detect objects based on at least one of an image or a point cloud captured by a sensor of an autonomous driving vehicle.

2. The method of claim 1 , wherein an X axis of the 2D space represents width of bounding boxes and a Y axis of the 2D space represents heights of the bounding boxes.

3. The method of claim 1 , wherein clustering the bounding boxes and determining the anchor box to represent each of the clusters comprise:
for each of the bounding boxes, calculating a matching degree between the bounding box and each of the anchor boxes; and
assigning the bounding box to one of the anchor boxes that has a largest overlapped area.

4. The method of claim 3 , further comprising:
iteratively performing calculating a matching degree between each bounding box and each anchor box and assigning each bounding box to one of the anchor boxes, until the predetermined condition is satisfied.

5. The method of claim 4 , wherein the predetermined condition requires a number of bounding boxes assigned to each of the anchor boxes is greater than a predetermined threshold.

6. The method of claim 4 , wherein the predetermined condition requires that each of the anchor boxes falls within a dense area of the bounding boxes on the 2D space.

7. The method of claim 4 , wherein the predetermined condition requires that an aspect ratio of each of the bounding boxes assigned to a particular anchor box is within a predetermined range.

8. The method of claim 3 , wherein the matching degree between the bounding box and each anchor box is determined based on an overlapped area between the bounding box and each anchor box.

9. The method of claim 8 , wherein calculating matching degree between the bounding box and each anchor box comprises calculating an intersection-to-union (IOU) ratio between the bounding box and each anchor box to represent the matching degree.

10. A non-transitory machine-readable medium having instructions stored therein, which when executed by a processor, cause the processor to perform operations, the operations comprising:
plotting a plurality of bounding boxes in a two-dimensional (2D) space based on their respective dimensions;
clustering the plurality of bounding boxes into one or more clusters of the bounding boxes based on a distribution density of the bounding boxes on the 2D space;
for each of the clusters of bounding boxes, determining an anchor box to represent the corresponding cluster;
determining whether a distribution of the bounding boxes assigned to the anchor boxes satisfies a predetermined condition;
in response to determining that the predetermined condition has not been satisfied, adjusting a dimension of at least one of the anchor boxes on the 2D space; and
training a neural network model for detecting objects using the anchor boxes, wherein the neural network model is utilized to detect objects based on at least one of an image or a point cloud captured by a sensor of an autonomous driving vehicle.

11. The machine-readable medium of claim 10 , wherein an X axis of the 2D space represents width of bounding boxes and a Y axis of the 2D space represents heights of the bounding boxes.

12. The machine-readable medium of claim 10 , wherein clustering the bounding boxes and determining the anchor box to represent each of the clusters comprise:
for each of the bounding boxes, calculating a matching degree between the bounding box and each of the anchor boxes; and
assigning the bounding box to one of the anchor boxes that has a largest overlapped area.

13. The machine-readable medium of claim 12 , wherein the operations further comprise:
iteratively performing calculating a matching degree between each bounding box and each anchor box and assigning each bounding box to one of the anchor boxes, until the predetermined condition is satisfied.

14. The machine-readable medium of claim 13 , wherein the predetermined condition requires a number of bounding boxes assigned to each of the anchor boxes is greater than a predetermined threshold.

15. The machine-readable medium of claim 13 , wherein the predetermined condition requires that each of the anchor boxes falls within a dense area of the bounding boxes on the 2D space.

16. The machine-readable medium of claim 13 , wherein the predetermined condition requires that an aspect ratio of each of the bounding boxes assigned to a particular anchor box is within a predetermined range.

17. The machine-readable medium of claim 12 , wherein the matching degree between the bounding box and each anchor box is determined based on an overlapped area between the bounding box and each anchor box.

18. The machine-readable medium of claim 17 , wherein calculating the matching degree between the bounding box and each anchor box comprises calculating an intersection-to-union (IOU) ratio between the bounding box and each anchor box to represent the matching degree.

19. A data processing system, comprising:
a processor; and
a memory coupled to the processor to store instructions, which when executed by the processor, cause the processor to perform operations, the operations including
plotting a plurality of bounding boxes in a two-dimensional (2D) space based on their respective dimensions,
clustering the plurality of bounding boxes into one or more clusters of the bounding boxes based on a distribution density of the bounding boxes on the 2D space,

for each of the clusters of bounding boxes, determining an anchor box to represent the corresponding cluster,
determining whether a distribution of the bounding boxes assigned to the anchor boxes satisfies a predetermined condition,
in response to determining that the predetermined condition has not been satisfied, adjusting a dimension of at least one of the anchor boxes on the 2D space; and
training a neural network model for detecting objects using the anchor boxes, wherein the neural network model is utilized to detect objects based on at least one of an image or a point cloud captured by a sensor of an autonomous driving vehicle.

20. The system of claim 19 , wherein an X axis of the 2D space represents width of bounding boxes and a Y axis of the 2D space represents heights of the bounding boxes.

Assignees

• Baidu Usa Llc

Inventors

• Tsoi Ka Wai
• Choe Tae Eun
• Guo Yuliang
• Chen Guang
• Zhang Weide

Classifications

• G06V10/774

  Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06V10/762

  using clustering, e.g. of similar faces in social networks · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06V20/56 Primary

  exterior to a vehicle by using sensors mounted on the vehicle · CPC title