Enhanced three-dimensional point cloud rendering

Patent metadata

Field	Value
Publication number	US-10410406-B2
Application number	US-201715443804-A
Country	US
Kind code	B2
Filing date	Feb 27, 2017
Priority date	Feb 27, 2017
Publication date	Sep 10, 2019
Grant date	Sep 10, 2019

Abstract

A method of rendering a three-dimensional point cloud in a two-dimensional display includes inputting the three-dimensional point cloud that includes three-dimensional coordinates of a set of points, creating a depth buffer for the three-dimensional point cloud that includes depth data for the set of points from a viewpoint location. The method further includes determining a foreground depth buffer by, for each respective pixel area of the two-dimensional display, determining a foreground depth by detecting a closest point to the viewpoint location among a subset of the set of points corresponding to the respective pixel area, and assigning a depth of the closest point as the foreground depth for the respective pixel area. The method further includes filtering the depth buffer to obtain a filtered depth buffer by removing points that are not in the foreground, and outputting the filtered depth buffer to the two-dimensional display.

First claim

What is claimed is:

1. A method of rendering a three-dimensional point cloud in a two-dimensional display, the method comprising:
inputting the three-dimensional point cloud including three-dimensional coordinates of a set of points representing surfaces of one or more objects;
creating a depth buffer for the three-dimensional point cloud, the depth buffer including depth data for the set of points from a viewpoint location;
determining a foreground depth buffer, the foreground depth buffer including a foreground depth for each respective pixel area of the two-dimensional display, wherein the foreground depth buffer is determined by:
applying an opening operation of a mathematical morphology algorithm to the depth buffer to remove background points from the set of points to obtain a plurality of foreground points; and
applying a closing operation of the mathematical morphology algorithm to reconnect the plurality of foreground points with each other;

filtering the depth buffer to obtain a filtered depth buffer by, for each respective pixel area of the two-dimensional display:
comparing a depth of each respective point corresponding to the respective pixel area to a foreground depth of the respective pixel area; and
removing the respective point from the depth buffer upon determining that the depth of the respective point is greater than the foreground depth of the respective pixel area; and

outputting the filtered depth buffer to the two-dimensional display for displaying a two-dimensional image of the three-dimensional point cloud from the viewpoint location.

2. The method of claim 1 , wherein the depth buffer resides in a graphics processing unit (GPU).

3. The method of claim 2 , wherein determining the foreground depth buffer and filtering the depth buffer are performed using the GPU.

4. The method of claim 3 , wherein determining the foreground depth buffer and filtering the depth buffer are performed on-the-fly for real-time navigation of the three-dimensional point cloud.

5. The method of claim 1 , wherein the three-dimensional point cloud includes color data for each respective point of the set of points, and the method further comprising:
creating a color buffer for the three-dimensional point cloud using the color data for each respective point;
filtering the color buffer to obtain a filtered color buffer by, for each respective pixel area of the two-dimensional display, removing the respective point from the color buffer upon determining that the depth of the respective point is greater than the foreground depth of the respective pixel area; and
outputting the filtered color buffer to the two-dimensional display along with the filtered depth buffer for displaying the two-dimensional image of the three-dimensional point cloud.

6. A method of rendering a three-dimensional point cloud in a two-dimensional display, the method comprising:
inputting the three-dimensional point cloud including three-dimensional coordinates of a set of points representing surfaces of one or more objects;
creating a depth buffer for the three-dimensional point cloud, the depth buffer including depth data for the set of points from a viewpoint location;
determining a foreground depth buffer, the foreground depth buffer including a foreground depth for each respective pixel area of the two-dimensional display, wherein the foreground depth buffer is determined by:
applying an opening operation of a mathematical morphology algorithm to the depth buffer to remove background points from the set of points to obtain a plurality of foreground points; and
applying a closing operation of the mathematical morphology algorithm to reconnect the plurality of foreground points with each other;

filtering the depth buffer to obtain a filtered depth buffer by, for each respective pixel area of the two-dimensional display:
comparing a depth of each respective point corresponding to the respective pixel area to a foreground depth of the respective pixel area; and
removing the respective point from the depth buffer upon determining that the depth of the respective point is greater than the foreground depth of the respective pixel area;

performing interpolation among remaining points in the filtered depth buffer to obtain an interpolated depth buffer; and
outputting the interpolated depth buffer to the two-dimensional display for displaying a two-dimensional image of the three-dimensional point cloud from the viewpoint location.

7. The method of claim 6 , wherein the interpolation is performed using one of a linear interpolation algorithm, a polynomial interpolation algorithm, a morphological processing algorithm, or an inpainting technique.

8. The method of claim 6 , wherein the three-dimensional point cloud includes color data for each respective point of the set of points, and the method further comprising:
creating a color buffer for the three-dimensional point cloud using the color data for each respective point;
filtering the color buffer to obtain a filtered color buffer by, for each respective pixel area of the two-dimensional display, removing the respective point from the color buffer upon determining that the depth of the respective point is greater than the foreground depth of the respective pixel area;
performing interpolation among remaining points in the filtered color buffer to obtain an interpolated color buffer; and
outputting the interpolated color buffer to the two-dimensional display along with the interpolated depth buffer for displaying the two-dimensional image of the three-dimensional point cloud.

9. The method of claim 8 , wherein the interpolation among remaining points in the filtered color buffer is performed in parallel to performing interpolation among remaining points in the filtered depth buffer.

10. A method of rendering a three-dimensional point cloud in a two-dimensional display, the method comprising:
inputting the three-dimensional point cloud including three-dimensional coordinates of a set of points representing surfaces of one or more objects, the three-dimensional point cloud further including color data for each respective point of the set of points;
creating a depth buffer for the three-dimensional point cloud, the depth buffer including depth data for the set of points from a viewpoint location;
creating a color buffer for the three-dimensional point cloud using the color data for each respective point;
segmenting the depth buffer and the color buffer to obtain a segmented depth buffer and a segmented color buffer based on at least one of color, depth, intensity, or orientation, wherein each of the segmented depth buffer and the segmented color buffer includes one or more segmented regions, each segmented region representing a single object of the one or more objects represented by the set of points; and
outputting the segmented depth buffer and the segmented color buffer to the two-dimensional display for displaying a two-dimensional image of the three-dimensional point cloud from the viewpoint location.

11. The method of claim 10 , wherein segmenting the depth buffer and the color buffer is based on a predetermined depth criterion.

12. The method of claim 10 , wherein segmenting the depth buffer and the color buffer is based on orientation, and wherein each segmented region has a respective orientation.

13. The method of claim 10 , wherein segmenting the depth buffer and the color buffer is based on a predetermined intensity criterion.

14. The method of claim 10 , further comprising:
assigning a respective color for each segmented region.

15. The method of claim 10 ,

Assignees

• Trimble Ab

Inventors

• Serna Andrés
• Chaperon Thomas

Classifications

• G06T15/40 Primary

  Hidden part removal · CPC title

• G06T2210/56

  Particle system, point based geometry or rendering · CPC title

• G06T15/405 Primary

  using Z-buffer · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title