Estimating two-dimensional object bounding box information based on bird's-eye view point cloud

Patent metadata

Field	Value
Publication number	US-10970871-B2
Application number	US-201916380273-A
Country	US
Kind code	B2
Filing date	Apr 10, 2019
Priority date	Sep 7, 2018
Publication date	Apr 6, 2021
Grant date	Apr 6, 2021

Abstract

Upon receiving a set of two-dimensional data points representing an object in an environment, a bounding box estimator estimates a bounding box vector representative of a two-dimensional version of the object that is represented by the two-dimensional data points.

First claim

What is claimed is:

1. A method for estimating a 2D object bounding box for an object, the method comprising:
receiving a 3D point cloud that is representative of a particular object;
processing the 3D point cloud to generate a set of unordered two-dimensional data points, the set of unordered two-dimensional data points representing a projection of 3D point cloud to a bird's eye view (BEV) of a space surrounding a detecting and ranging (DAR) sensor that includes the particular object, wherein each data point in the set of unordered two-dimensional data points includes a first coordinate value and a second coordinate value;
generating a set two-dimensional mean-reduced data points from the set of unordered two-dimensional data points by, for each data point in the set of unordered two-dimensional data points:
subtracting a first mean from the first coordinate value to generate a first mean-reduced value;
determining a second mean of the second coordinate values;
subtracting the second mean from the second coordinate values to generate a second mean-reduced value;
generating a two-dimensional mean-reduced data point comprising the first mean-reduced values and the second mean-reduced values;

generating, by a first neural network configured for feature extraction, a feature vector based on the set of mean-reduced data points; and
generating, by a second neural network configured for bounding box regression, a bounding box vector representative of a two-dimensional bounding box for the object based on the feature vector, the bounding box vector including a value representative of a width for a bounding box for the object, a value representative of a length for the bounding box, a value representative of an orientation angle for the bounding box, and values representative of a center for the bounding box;
wherein generating, by the second neural network configured for bounding box regression, the bounding box vector for the object based on the feature vector comprises:
estimating, using an orientation-estimating sub-network of the second neural network configured for bounding box regression, a bounding box orientation vector for the bounding box based on the feature vector, the bounding box orientation vector defining the value representative of an orientation angle for the bounding box;
estimating, using a size-estimating sub-network of the second neural network configured for bounding box regression, a size vector for the bounding box based on the feature vector, the size vector defining the value representative of the width for a bounding box for the object and the value representative of a length for the bounding box;
concatenating, to the feature vector, the bounding box orientation vector and the size vector to generate a concatenated vector; and
estimating, using a center-estimating sub-network of the second neural network configured for bounding box regression, a center vector for the bounding box based on the concatenated vector, the center vector defining the values representative of a center for the bounding box.

2. The method of claim 1 , wherein the bounding box has an orientation angle and the bounding box orientation vector comprises:
a cosine value representative of a cosine function applied to a doubling of the orientation angle; and
a sine value representative of a sine function applied to the doubling of the orientation angle.

3. The method of claim 1 , further comprising:
training the first neural network and the second neural network together to update the weights and biases of the first neural network and second neural network until an overall loss function is optimized.

4. The method of claim 1 , further comprising:
receiving, at a data analysis system, sensor data comprising the 3D point cloud captured by the DAR sensor;
processing, at the data analysis system, the sensor data to generate one or more 3D point clouds, each 3D point cloud representative of an object detected by the DAR sensor.

5. A processing unit comprising:
electronic storage storing computer readable instructions;
a processor configured to execute the instructions to:
receive sensor data provided by a detection and ranging (DAR) sensor, the sensor data comprising a 3D point cloud that is representative of a particular object;
process the sensor data to generate one or more sets of unordered two-dimensional data points, each set of unordered two-dimensional data points representing a projection of 3D point cloud to a bird's eye view (BEV) of the space surrounding the DAR sensor that includes the particular object, wherein each data point in the set of unordered two-dimensional data points includes a first coordinate value and a second coordinate value;
generate a set two-dimensional mean-reduced data points from the set of unordered two-dimensional data points by, for each data point in the set of unordered two-dimensional data points:
subtracting a first mean from the first coordinate value to generate a first mean-reduced value;
determining a second mean of the second coordinate values;
subtracting the second mean from the second coordinate values to generate a second mean-reduced value;
generating a two-dimensional mean-reduced data point comprising the first mean-reduced values and the second mean-reduced values;

generate, by a first neural network configured for feature extraction, a feature vector based on the set of mean-reduced data points; and
generate, by a second neural network configured for bounding box regression, a bounding box vector representative of a two-dimensional bounding box for the object based on the feature vector, the bounding box vector including a value representative of a width for a bounding box for the object, a value representative of a length for the bounding box, a value representative of an orientation angle for the bounding box, and values representative of a center for the bounding box;

wherein the second neural network configured for bounding box regression, to generate the bounding box vector for the object based on the feature vector, is configured to:
estimate, using an orientation-estimating sub-network of the second neural network configured for bounding box regression, a bounding box orientation vector for the bounding box based on the feature vector, the bounding box orientation vector defining the value representative of an orientation angle for the bounding box;
estimate, using a size-estimating sub-network of the second neural network configured for bounding box regression, a size vector for the bounding box based on the feature vector, the size vector defining the value representative of the width for a bounding box for the object and the value representative of a length for the bounding box;
concatenate, to the feature vector, the bounding box orientation vector and the size vector to generate a concatenated vector; and
estimate, using a center-estimating sub-network of the second neural network configured for bounding box regression, a center vector for the bounding box based on the concatenated vector, the center vector defining the values representative of a center for the bounding box.

6. The processing unit of claim 5 , wherein the orientation-estimating sub-network comprises:
a first fully connected layer configured to receive the feature vector;
a second fully connected layer configured to receive output from the first fully connected layer; and
a third fully connected layer configured to:
receive output from the second fully connected layer; and
generate the bounding box orientation vector.

7. The processing unit of claim 5 , wherein the size-estimating sub-network comprises:
a first fully connected layer configured to receive the feature vector;
a second fully connected lay

Assignees

• Nezhadarya Ehsan
• Liu Yang
• Liu Bingbing
• Huawei Tech Co Ltd

Inventors

• Nezhadarya Ehsan
• Liu Yang
• Liu Bingbing

Classifications

• G06N3/045

  Combinations of networks · CPC title

• G06N3/0499

  Feedforward networks · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title

• G06N3/08

  Learning methods · CPC title