Hierarchical point cloud compression with smoothing

Patent metadata

Field	Value
Publication number	US-11508095-B2
Application number	US-202117163194-A
Country	US
Kind code	B2
Filing date	Jan 29, 2021
Priority date	Apr 10, 2018
Publication date	Nov 22, 2022
Grant date	Nov 22, 2022

Abstract

A system comprises an encoder configured to compress attribute information for a point cloud and/or a decoder configured to decompress compressed attribute for the point cloud. To compress the attribute information, multiple levels of detail are generated based on spatial information. Also, attribute values are predicted based on the level of details. A decoder follows a similar prediction process based on level of details. Also, attribute correction values may be determined to correct predicted attribute values and may be used by a decoder to decompress a point cloud compressed using level of detail attribute compression. In some embodiments, an update operation is performed to smooth attribute correction values taking into account an influence factor of respective points in a given level of detail on attributes in other levels of detail.

First claim

What is claimed is:

1. A non-transitory, computer-readable, medium storing program instructions, that when executed on or across one or more processors, cause the one or more processors to:
determine a plurality of levels of detail for encoding attribute values for a plurality of points in three-dimensional (3D) space, wherein different levels of detail include different sub-sets of the plurality of points;
for respective points of a first level of detail or points of one or more additional levels of detail, determine a predicted attribute value for the respective point based on predicted or assigned attributes values for neighboring points in a same level of detail as the respective point;
for respective points of the first level of detail or the points of the one or more additional levels of detail, determine an attribute correction value for the respective point, based on comparing a predicted attribute value for the respective point to attribute information for the point prior to compression;
apply an update operation to smooth the attribute correction values, wherein the update operation takes into account relative influences of the attributes of the points of a given level of detail on attribute values of points included in other levels of detail; and
encode the updated attribute correction values for the first level of detail and the one or more additional levels of detail.

2. The non-transitory, computer-readable, medium of claim 1 , wherein to apply the update operation, the program instructions, when executed on or across the one or more processors, cause the one or more processors to:
determine edge distance for edges between a vertex corresponding to a respective point for which an attribute value is being predicted and vertices for the neighboring points being used in predicting the attribute value for the respective point,
wherein attribute values of the neighboring points with shorter edges to the respective point are given a greater edge distance weighting than attribute values of points with longer edges to the respective point.

3. The non-transitory, computer-readable, medium of claim 2 , wherein the program instructions, when executed on or across the one or more processors, further cause the one or more processors to:
quantize the attribute correction values or the updated attribute correction values, wherein the quantization differs based on edge distance weightings determined based on the edge distances using a relationship known by an encoder performing the encoding and a decoder that decodes the encoded point cloud.

4. The non-transitory, computer-readable, medium of claim 3 , wherein the quantization differs based on edge distance weighting by an exponential relationship, and
wherein the program instructions, when executed on or across the one or more processors, further cause the one or more processors to:
encode, in a bit stream with the updated attribute correction values, an exponential coefficient for determining the quantization to be applied.

5. The non-transitory, computer-readable, medium of claim 2 , wherein the program instructions, when executed on or across the one or more processors, further cause the one or more processors to:
quantize edge distance weightings for the edge distances, wherein edge distance weightings with less weights are quantized to a greater degree than edge distance weightings with greater weights.

6. The non-transitory, computer-readable, medium of claim 2 , wherein the program instructions, when executed on or across the one or more processors, further cause the one or more processors to:
disregard, when predicting the attribute values, influences of points in another level of detail with a path of edge distances that adds up to be greater than a threshold distance from a given point for which an attribute value is being predicted.

7. The non-transitory, computer-readable, medium of claim 2 , wherein the weights of edges for the points are determined recursively based on assigning an initial same edge weight and iteratively updating the edge weights by traversing the points in an order from highest level of detail to lowest level of detail, wherein the update is determined based on applying an update function that takes into account the edge weights of a set of neighboring points that neighbor a point for which the edge weight is being updated.

8. The non-transitory, computer-readable, medium of claim 1 , wherein the program instructions, when executed on or across the one or more processors, further cause the one or more processors to:
assign an attribute value to at least one point of the first level of detail; and
encode the assigned attribute value for the at least one point in the first level of detail in a bit stream with the updated attribute correction values for first level of detail.

9. A device, comprising:
a memory storing program instructions; and
one or more processors, wherein the program instructions, when executed on or across the one or more processors, cause the one or more processors to:
determine a plurality of levels of detail for encoding attribute values for a plurality of points in three-dimensional (3D) space, wherein different levels of detail include different sub-sets of the plurality of points;
for respective points of a first level of detail or points of one or more additional levels of detail, determine a predicted attribute value for the respective point based on predicted or assigned attributes values for neighboring points in a same level of detail as the respective point;
for respective points of the first level of detail or the points of the one or more additional levels of detail, determine an attribute correction value for the respective point, based on comparing a predicted attribute value for the respective point to attribute information for the point prior to compression;
apply an update operation to smooth the attribute correction values, wherein the update operation takes into account relative influences of the attributes of the points of a given level of detail on attribute values of points included in other levels of detail; and
encode the updated attribute correction values for the first level of detail and the one or more additional levels of detail.

10. The device of claim 9 , further comprising:
one or more sensors configured to capture the plurality of points in 3D space, wherein respective ones of the points comprise spatial information for the point and attribute information for the point.

11. The device of claim 10 , wherein the one or more sensors comprise:
a LIDAR system;
a 3D camera; or
a 3D scanner.

12. The device of claim 9 , wherein to apply the update operation, the program instructions, when executed on or across the one or more processors, cause the one or more processors to:
determine edge distance for edges between a vertex corresponding to a respective point for which an attribute value is being predicted and vertices for the neighboring points being used in predicting the attribute value for the respective point,
wherein attribute values of the neighboring points with shorter edges to the respective point are given a greater edge distance weighting than attribute values of points with longer edges to the respective point.

13. The device of claim 12 , wherein the program instructions, when executed on or across the one or more processors, further cause the one or more processors to:
quantize the attribute correction values or the updated attribute correction values, wherein the quantization differs based on edge distance weightings determined based on the edge distances using a relationship known

Assignees

• Apple Inc

Inventors

• Mammou Khaled
• Tourapis Alexandros
• Kim Jungsun
• Robinet Fabrice A
• Valentin Valery G
• Su Yeping

Classifications

• G06T17/00

  Three-dimensional [3D] modelling for computer graphics · CPC title

• G06T2210/36

  Level of detail · CPC title

• G06T9/001 Primary

  Model-based coding, e.g. wire frame · CPC title