Enhanced three-dimensional point cloud rendering

Patent metadata

Field	Value
Publication number	US-2018247447-A1
Application number	US-201715443804-A
Country	US
Kind code	A1
Filing date	Feb 27, 2017
Priority date	Feb 27, 2017
Publication date	Aug 30, 2018
Grant date	—

Abstract

A method of rendering a three-dimensional point cloud in a two-dimensional display includes inputting the three-dimensional point cloud that includes three-dimensional coordinates of a set of points, creating a depth buffer for the three-dimensional point cloud that includes depth data for the set of points from a viewpoint location. The method further includes determining a foreground depth buffer by, for each respective pixel area of the two-dimensional display, determining a foreground depth by detecting a closest point to the viewpoint location among a subset of the set of points corresponding to the respective pixel area, and assigning a depth of the closest point as the foreground depth for the respective pixel area. The method further includes filtering the depth buffer to obtain a filtered depth buffer by removing points that are not in the foreground, and outputting the filtered depth buffer to the two-dimensional display.

First claim

What is claimed is:

1 . A method of rendering a three-dimensional point cloud in a two-dimensional display, the method comprising:
inputting the three-dimensional point cloud including three-dimensional coordinates of a set of points representing surfaces of one or more objects; creating a depth buffer for the three-dimensional point cloud, the depth buffer including depth data for the set of points from a viewpoint location; determining a foreground depth buffer by, for each respective pixel area of the two-dimensional display, determining a foreground depth by detecting a closest point to the viewpoint location among a subset of the set of points corresponding to the respective pixel area, and assigning a depth of the closest point as the foreground depth for the respective pixel area; filtering the depth buffer to obtain a filtered depth buffer by, for each respective pixel area of the two-dimensional display:
comparing a depth of each respective point corresponding to the respective pixel area to a foreground depth of the respective pixel area; and
removing the respective point from the depth buffer upon determining that the depth of the respective point is greater than the foreground depth of the respective pixel area; and
outputting the filtered depth buffer to the two-dimensional display for displaying a two-dimensional image of the three-dimensional point cloud from the viewpoint location.

2 . The method of claim 1 , wherein the depth buffer resides in a graphics processing unit (GPU).

3 . The method of claim 2 , wherein determining the foreground depth buffer and filtering the depth buffer are performed using the GPU.

4 . The method of claim 3 , wherein determining the foreground depth buffer and filtering the depth buffer are performed on-the-fly for real-time navigation of the three-dimensional point cloud.

5 . The method of claim 1 , wherein determining the foreground depth buffer is performed using a Mathematical Morphology algorithm.

6 . The method of claim 5 , wherein determining the foreground depth buffer comprises applying an opening operation and a closing operation of the Mathematical Morphology algorithm.

7 . The method of claim 1 , wherein the three-dimensional point cloud includes color data for each respective point of the set of points, and the method further comprising:
creating a color buffer for the three-dimensional point cloud using the color data for each respective point; filtering the color buffer to obtain a filtered color buffer by, for each respective pixel area of the two-dimensional display, removing the respective point from the color buffer upon determining that the depth of the respective point is greater than the foreground depth of the respective pixel area; and outputting the filtered color buffer to the two-dimensional display along with the filtered depth buffer for displaying the two-dimensional image of the three-dimensional point cloud.

8 . The method of claim 1 , wherein the subset of the set of points corresponding to the respective pixel area comprises zero point, one point, or a plurality of points.

9 . A method of rendering a three-dimensional point cloud in a two-dimensional display, the method comprising:
inputting the three-dimensional point cloud including three-dimensional coordinates of a set of points representing surfaces of one or more objects; creating a depth buffer for the three-dimensional point cloud, the depth buffer including depth data for the set of points from a viewpoint location; determining a foreground depth buffer by, for each respective pixel area of the two-dimensional display, determining a foreground depth by detecting a closest point among a subset of the set of points corresponding to the respective pixel area, and assigning a depth of the closest point as the foreground depth for the respective pixel area; filtering the depth buffer to obtain a filtered depth buffer by, for each respective pixel area of the two-dimensional display:
comparing a depth of each respective point corresponding to the respective pixel area to a foreground depth of the respective pixel area; and
removing the respective point from the depth buffer upon determining that the depth of the respective point is greater than the foreground depth of the respective pixel area;
performing interpolation among remaining points in the filtered depth buffer to obtain an interpolated depth buffer; and outputting the interpolated depth buffer to the two-dimensional display for displaying a two-dimensional image of the three-dimensional point cloud from the viewpoint location.

10 . The method of claim 9 , wherein determining the foreground depth buffer is performed using a Mathematical Morphology algorithm.

11 . The method of claim 9 , wherein interpolation is performed using one of a linear interpolation algorithm, a polynomial interpolation algorithm, a morphological processing algorithm, or an inpainting technique.

12 . The method of claim 9 , wherein the three-dimensional point cloud includes color data for each respective point of the set of points, and the method further comprising:
creating a color buffer for the three-dimensional point cloud using the color data for each respective point; filtering the color buffer to obtain a filtered color buffer by, for each respective pixel area of the two-dimensional display, removing the respective point from the color buffer upon determining that the depth of the respective point is greater than the foreground depth of the respective pixel area; performing interpolation among remaining points in the filtered color buffer to obtain an interpolated color buffer; and outputting the interpolated color buffer to the two-dimensional display along with the interpolated depth buffer for displaying the two-dimensional image of the three-dimensional point cloud.

13 . The method of claim 12 , wherein performing interpolation among remaining points in the filtered color buffer is performed in parallel to performing interpolation among remaining points in the filtered depth buffer.

14 . The method of claim 9 , wherein the subset of the set of points corresponding to the respective pixel area comprises zero point, one point, or a plurality of points.

15 . A method of rendering a three-dimensional point cloud in a two-dimensional display, the method comprising:
inputting the three-dimensional point cloud including three-dimensional coordinates of a set of points representing surfaces of one or more objects, the three-dimensional point cloud further including color data for each respective point of the set of points; creating a depth buffer for the three-dimensional point cloud, the depth buffer including depth data for the set of points from a viewpoint location; creating a color buffer for the three-dimensional point cloud using the color data for each respective point; segmenting the depth buffer and the color buffer to obtain a segmented depth buffer and a segmented color buffer based on at least one of color, depth, intensity, or orientation, wherein each of the segmented depth buffer and the segmented color buffer includes one or more segmented regions; and outputting the segmented depth buffer and the segmented color buffer to the two-dimensional display for displaying a two-dimensional image of the three-dimensional point cloud from the viewpoint location.

16 . The method of claim 15 , wherein segmenting the depth buffer and the color buffer is based on depth, and wherein each segmented region has a substantially similar depth.

17 .

Assignees

• Trimble Ab

Inventors

• Serna Andrés
• Chaperon Thomas

Classifications

• G06T2210/56

  Particle system, point based geometry or rendering · CPC title

• G06T15/40 Primary

  Hidden part removal · CPC title

• G06T15/405 Primary

  using Z-buffer · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title