Estimating two-dimensional object bounding box information based on bird's-eye view point cloud

Patent metadata

Field	Value
Publication number	US-2020082560-A1
Application number	US-201916380273-A
Country	US
Kind code	A1
Filing date	Apr 10, 2019
Priority date	Sep 7, 2018
Publication date	Mar 12, 2020
Grant date	—

Abstract

Upon receiving a set of two-dimensional data points representing an object in an environment, a bounding box estimator estimates a bounding box vector representative of a two-dimensional version of the object that is represented by the two-dimensional data points.

First claim

What is claimed is:

1 . A method for estimating a 2D object bounding box for an object, the method comprising:
receiving, at a bounding box estimator, a 3D point cloud that is representative of a particular object; processing, at the bounding box estimator, the 3D point cloud to generate a set of unordered two-dimensional data points, the set of unordered two-dimensional data points representing a projection of 3D point cloud to a bird's eye view (BEV) of the space surrounding the DAR sensor that includes the particular object; and generating, by the bounding box estimator, a bounding box vector for the object, the bounding box vector including:
a value representative of a width for a bounding box for the object;
a value representative of a length for the bounding box;
a value representative of an orientation angle for the bounding box;
and
values representative of a center for the bounding box.

2 . The method of claim 1 wherein each data point in the set of two-dimensional data points includes a first coordinate value and a second coordinate value.

3 . The method of claim 2 further comprising:
determining, by the bounding box estimator, a first mean of the first coordinate values;
subtracting, by the bounding box estimator, the first mean from the first coordinate values to produce first mean-reduced values;
determining, by the bounding box estimator, a second mean of the second coordinate values; and
subtracting, by the bounding box estimator, the second mean from the second coordinate values to produce second mean-reduced values.

4 . The method of claim 3 , further comprising:
receiving, at a first neural network of the bounding box estimator configured for feature extraction, the first mean-reduced values and the second mean-reduced values; and generating, by the first neural network, an extracted feature vector from the first mean-reduced values and the second mean-reduced values; estimating, using an orientation-estimating sub-network of a second neural network of the bounding box estimator configured for bounding box regression, a bounding box orientation vector for the bounding box; estimating, using a size-estimating sub-network of the second neural network configured for bounding box regression, a size vector for the bounding box; and estimating, using a center-estimating sub-network of the second neural network configured for bounding box regression, a center vector for the bounding box.

5 . The method of claim 4 wherein the bounding box has an orientation angle and the bounding box orientation vector comprises:
a cosine value representative of a cosine function applied to a doubling of the orientation angle; and
a sine value representative of a sine function applied to the doubling of the orientation angle.

6 . The method of claim 5 further comprising:
concatenating, to the extracted feature vector, the bounding box orientation vector and the size vector to generate a concatenated vector; and
estimating, using the center-estimating sub-network configured for bounding box regression, the center of the bounding box based on the concatenated vector.

7 . The method of claim 4 , further comprising:
training the first neural network configured and the second neural network together to update the weights and biases of the first neural network and second neural network until an overall loss function for the bounding box estimator is optimized.

8 . The method of claim 1 , further comprising:
receiving, at a data analysis system, sensor data comprising a point cloud captured by a DAR sensor; processing, at the data analysis system, the sensor data to generate one of more 3D point clouds, each 3D point cloud representative of an object detected by the DAR; providing, by the data analysis system, each of the one or more 3D point clouds to the bounding box estimator.

9 . A processing unit comprising:
electronic storage storing computer readable instructions defining a bounding box estimator; a processor configured to execute the instructions to:
receive sensor data provided by a detection and ranging (DAR) sensor, the sensor data comprising a 3D point cloud that is representative of a particular object;
process the sensor data to generate one or more sets of unordered two-dimensional data points, each set of unordered two-dimensional data points representing a projection of 3D point cloud to a bird's eye view (BEV) of the space surrounding the DAR sensor that includes the particular object; and
generate an estimated bounding box vector for the object, the bounding box vector including:
a value representative of a width for the bounding box;
a value representative of a length for the bounding box;
a value representative of an orientation angle for the bounding box; and
values representative of a center for the bounding box.

10 . The processing unit of claim 8 , wherein each data point in each set of two-dimensional data points includes a first coordinate value and a second coordinate value.

11 . The processing unit of claim 9 wherein the bounding box estimator comprises:
a mean pool configured to, for the set of two-dimensional data points:
determine a first mean of the first coordinate values; and
determine a second mean of the second coordinate values; and

a subtraction function configured to, for the set of two-dimensional data points:
subtract the first mean from the first coordinate values to produce first mean-reduced values; and
subtract the second mean from the second coordinate values to produce second mean-reduced values.

12 . The processing unit of claim 10 wherein the bounding box estimator comprises:
a first neural network configured to receive the first mean-reduced values and the second mean-reduced values and generate an extracted feature vector from the first mean-reduced values and the second mean-reduced values;
a second neural network comprising an orientation-estimating sub-network configured to estimate, based on the extracted feature vector, a bounding box orientation vector for the bounding box.

13 . The processing unit of claim 12 wherein the orientation-estimating sub-network comprises:
a first fully connected layer configured to receive the extracted feature vector;
a second fully connected layer configured to receive output from the first fully connected layer; and
a third fully connected layer configured to:
receive output from the second fully connected layer; and
generate the bounding box orientation vector.

14 . The processing unit of claim 12 , wherein the second neural network further comprises, a size-estimating sub-network configured to estimate, based on the extracted feature vector, a size vector for the bounding box, the size vector including the width information and the length information.

15 . The processing unit of claim 14 , wherein the size-estimating sub-network comprises:
a first fully connected layer configured to receive the extracted feature vector; a second fully connected layer configured to receive output from the first fully connected layer; and a third fully connected layer configured to:
receive output from the second fully connected layer; and
generate the size vector.

16 . The processing unit of claim 14 , wherein the second neural network further comprises a center-estimating sub-network configured to estimate, based on the extracted feature vector, the bounding box orientation vector and the size vector, a center vector for the bounding box, the center vector expressed in a mean-reduced coordinate system.

Assignees

• Nezhadarya Ehsan
• Liu Yang
• Liu Bingbing

Inventors

• Nezhadarya Ehsan
• Liu Yang
• Liu Bingbing

Classifications

• G06T2210/56

  Particle system, point based geometry or rendering · CPC title

• G06T2210/12

  Bounding box · CPC title

• G06T15/10 Primary

  Geometric effects · CPC title

• G06T7/73 Primary

  using feature-based methods · CPC title

• G06T15/20

  Perspective computation · CPC title