Ground plane estimation in a computer vision system

Patent metadata

Field	Value
Publication number	US-11763568-B2
Application number	US-202017115079-A
Country	US
Kind code	B2
Filing date	Dec 8, 2020
Priority date	Jan 5, 2016
Publication date	Sep 19, 2023
Grant date	Sep 19, 2023

Abstract

Estimation of the ground plane of a three dimensional (3D) point cloud based modifications to the random sample consensus (RANSAC) algorithm is provided. The modifications may include applying roll and pitch constraints to the selection of random planes in the 3D point cloud, using a cost function based on the number of inliers in the random plane and the number of 3D points below the random plane in the 3D point cloud, and computing a distance threshold for the 3D point cloud that is used in determining whether or not a 3D point in the 3D point cloud is an inlier of a random plane.

First claim

What is claimed is:

1. A non-transitory medium storing instructions that, when executed by at least one processor, cause the at least one processor to estimate a ground plane in a three-dimensional (3D) point cloud in a computer vision system, the instructions comprising instructions to:
prune a 3D point cloud generated based on a plurality of two-dimensional (2D) frames captured by a camera to remove 3D points in the 3D point cloud not likely to be on any ground plane to generate a pruned 3D point cloud;
identify a plane in the pruned 3D point cloud from three randomly selected non-collinear 3D points in the pruned 3D point cloud and computing parameters of the plane that pass through the three randomly selected non-collinear 3D points, wherein roll and pitch of the plane are within roll and pitch ranges of the camera;
compute a cost function of the plane, wherein the cost function is based on a difference between a number of inliers of the plane and a number of 3D points in the 3D point cloud below the plane;
compare the cost function of the plane with a saved previously computed best cost function for a previously identified plane; and
save, when the cost function is better than the previously computed best cost function, the cost function as the best cost function, the plane as the best plane, and the inliers for the plane.

2. The non-transitory medium of claim 1 , wherein the instructions further comprise an instruction to execute the identify, compute, compare, and save instructions for a set number of iterations.

3. The non-transitory medium of claim 2 , further comprising an instruction to apply a least squares algorithm to the inliers of the saved best plane to estimate the ground plane.

4. The non-transitory medium of claim 1 , wherein the instructions further comprise an instruction to:
determine at least one of a position and distance of an object in a field of view of the camera based on the best plane.

5. A non-transitory medium storing instructions that, when executed by at least one processor, cause the at least one processor to estimate a ground plane in a three-dimensional (3D) point cloud in a computer vision system, the instructions comprising instructions to:
prune a 3D point cloud generated based on a plurality of two-dimensional (2D) frames captured by a camera to remove 3D points in the 3D point cloud not likely to be on any ground plane to generate a pruned 3D point cloud;
identify a plane in the pruned 3D point cloud from three randomly selected non-collinear 3D points in the pruned 3D point cloud and computing parameters of the plane that pass through the three randomly selected non-collinear 3D points, wherein roll and pitch of the lane are within roll and pitch ranges of the camera,
compute a cost function of the plane, wherein the cost function is based on a difference between a number of inliers of the plane and a number of 3D points in the 3D point cloud below the plane,
compare the cost function of the plane with a saved previously computed best cost function for a previously identified plane; and
save, when the cost function is better than the previously computed best cost function, the cost function as the best cost function, the plane as the best plane, and the inliers for the plane,
wherein each 3D point of the 3D point cloud has a height coordinate value indicating a distance above or below a ground plane, and the instruction to prune the 3D point cloud comprises an instruction to sort the 3D points in the 3D point cloud according to the height coordinate values of the 3D points and eliminate all 3D points from the 3D point cloud not included in a set range of the height coordinate values.

6. The non-transitory medium of claim 5 , wherein the set range is a top percentage of height coordinate values.

7. A computer vision system comprising:
an image capture component configured to receive a three-dimensional (3D) point cloud generated based on a plurality of two-dimensional (2D) frames of a scene; and
a processor, the processor configured to:
prune a 3D point cloud generated based on a plurality of two-dimensional (2D) frames captured by a camera of the image capture component to remove 3D points in the 3D point cloud not likely to be on any ground plane to generate a pruned 3D point cloud;
identify a plane in the pruned 3D point cloud from three randomly selected non-collinear 3D points in the pruned 3D point cloud and computing parameters of the plane that pass through the three randomly selected non-collinear 3D points, wherein roll and pitch of the plane are within roll and pitch ranges of the camera;
compute a cost function of the plane, wherein the cost function is based on a difference between a number of inliers of the plane and a number of 3D points in the 3D point cloud below the plane;
compare the cost function of the plane with a saved previously computed best cost function for a previously identified plane; and
save, when the cost function is better than the previously computed best cost function, the cost function as the best cost function, the plane as the best plane, and the inliers for the plane.

8. The computer vision system of claim 7 , wherein the processor is further configured to perform the identify, compute, compare, and save operations for a set number of iterations.

9. The computer vision system of claim 8 , wherein the processor is further configured to apply a least squares algorithm to the inliers of the saved best plane to estimate the ground plane.

10. The computer vision system of claim 7 , wherein the processor comprises at least one general purpose processor and at least one digital signal processor.

11. The computer vision system of claim 10 , further comprising:
memory;
a vision processor;
a display; and
an interconnect to which the at least one general purpose processor, the at least one digital signal processor, the memory, the vision processor, and the display are coupled.

12. The computer vision system of claim 11 , further comprising input and output ports coupled to the interconnect.

13. The computer vision system of claim 7 , wherein the processor is further configured to determine at least one of a position and distance of an object in a field of view of the camera based on the best plane.

14. A computer vision system comprising:
an image capture component configured to receive a three-dimensional (3D) point cloud generated based on a plurality of two-dimensional (2D) frames of a scene; and
a processor, the processor configured to:
prune a 3D point cloud generated based on a plurality of two-dimensional (2D) frames captured by a camera of the image capture component to remove 3D points in the 3D point cloud not likely to be on any ground plane to generate a pruned 3D point cloud,
identify a plane in the pruned 3D point cloud from three randomly selected non-collinear 3D points in the pruned 3D point cloud and computing parameters of the plane that pass through the three randomly selected non-collinear 3D points, wherein roll and pitch of the plane are within roll and pitch ranges of the camera;
compute a cost function of the plane, wherein the cost function is based on a difference between a number of inliers of the plane and a number of 3D points in the 3D point cloud below the plane;
compare the cost function of the plane with a saved previously computed best cost function for a previously identified plane; and
save, when the cost function is better than the previously computed best cost function, the cost function as the best cost function, the plane as the best plane, and the inliers for the plane,

wherein each 3D point

Assignees

• Texas Instruments Inc

Inventors

• Nagori Soyeb
• Kumar Poorna
• Mathew Manu
• Viswanath Prashanth Ramanathpur
• Poddar Deepak Kumar

Classifications

• G06V20/56 Primary

  exterior to a vehicle by using sensors mounted on the vehicle · CPC title

• G06T2207/10021

  Stereoscopic video; Stereoscopic image sequence · CPC title

• G06T2207/20076

  Probabilistic image processing · CPC title

• G06T2207/30244

  Camera pose · CPC title

• G06T7/77

  using statistical methods · CPC title