Quantized depths for projection point cloud compression

Patent metadata

Field	Value
Publication number	US-2020111237-A1
Application number	US-201916593994-A
Country	US
Kind code	A1
Filing date	Oct 5, 2019
Priority date	Oct 5, 2018
Publication date	Apr 9, 2020
Grant date	—

Abstract

A system comprises an encoder configured to compress attribute information and/or spatial information for a point cloud and/or a decoder configured to decompress compressed attribute and/or spatial information for the point cloud. The encoder is configured to convert a point cloud into an image based representation. The encoder packs patch images into an image frame and fills empty spaces in the image frame with a padding. The encoder is also configured to determine quantized minimum depths and/or maximum depths patch images in the image frames, wherein depth information is signaled relative to the quantized minimum depth.

First claim

What is claimed is:

1 . A non-transitory computer-readable medium storing program instructions that, when executed by one or more processors, cause the one or more processors to:
generate, for a point cloud, one or more projected geometry patch images and attribute patch images; pack the generated one or more projected geometry patch images and attribute patch images into one or more image frames, wherein depth values of the geometry patch images are relative to a separately signaled quantized minimum depth for each respective patch corresponding to one of the one or more projected geometry images; encode the one or more image frames.

2 . The non-transitory computer-readable medium of claim 1 , wherein the program instructions, when executed on the one or more processors, cause the one or more processors to:
determine a minimum depth for respective ones of the geometry patch images, wherein respective points included in the respective geometry patch images each have an associated depth value, and wherein the minimum depth for a given geometry patch image is a minimum depth value of a point in the given geometry patch image that has a minimum depth as compared to depth values of other points included in the given geometry patch image.

3 . The non-transitory computer-readable medium of claim 2 , wherein the program instructions, when executed on the one or more processors, cause the one or more processors to:
quantize the determined minimum depths such that minimum depths having depth values in a value interval are assigned a common quantized minimum depth corresponding to the value interval.

4 . The non-transitory computer-readable medium of claim 3 , wherein the program instructions, cause the one or more processors to:
signal the quantized determined minimum depths using a bitdepth that is less than a bitdepth used for signaling the depth values of the points of the geometry patch images and attribute values of the attribute patch images.

5 . The non-transitory computer-readable medium of claim 3 , wherein the program instructions, when executed on the one or more processors, cause the one or more processors to:
adaptively determine a bitdepth for signaling the quantized determined minimum depths; and signal the determined bitdepth to be used for the quantized determined minimum depths.

6 . The non-transitory computer-readable medium of claim 3 , wherein the program instructions cause the one or more processors to:
arithmetically encode the quantized determined minimum depths.

7 . The non-transitory computer-readable medium of claim 3 , wherein the program instructions, when executed on the one or more processors, cause the one or more processors to:
determine for at least one of the determined quantized minimum depths, an offset to be added to, or subtracted from, the determined quantized minimum depth, wherein the offset is determined such that the value interval for the at least one quantized determined minimum depth is more closely centered on a range of depth values of points included in a given patch to which the at least one determined quantized minimum depth corresponds.

8 . The non-transitory computer readable medium of claim 7 , wherein the offset is a fixed-offset applied to the geometry patch images of the one or more image frames.

9 . The non-transitory computer-readable medium of claim 7 , wherein the offset is adaptively determined for respective ones of the geometry patch images and signaled in a bitstream with the encoded information.

10 . The non-transitory computer-readable medium of claim 1 , wherein the program instructions cause the one or more processors to:
determine a maximum depth for respective ones of the geometry patch images, wherein respective points included in the respective geometry patch images each have an associated depth value, and wherein the maximum depth for a given respective geometry patch image is a maximum depth value of a point in the given respective geometry patch image that has a maximum depth as compared to depth values of other points included in the respective given geometry patch image.

11 . The non-transitory computer-readable medium of claim 1 , wherein the program instructions cause the one or more processors to:
signal the determined maximum depth values as delta values relative to the corresponding quantized determined minimum depth values for same patches, wherein a given delta value added to a corresponding quantized determined minimum depth value results in a corresponding quantized determined maximum depth value.

12 . A non-transitory computer-readable medium storing program instructions that, when executed by one or more processors, cause the one or more processors to:
receive a bitstream comprising data for geometry images frames and attribute image frames of a point cloud, wherein the geometry image frames comprise depth information for patches of the point cloud projected onto patch planes and the attribute images frames comprise attribute information for the patches of the point cloud projected onto the patch planes, and wherein the bitstream comprises respective quantized minimum depths for the patches; reconstruct geometry patches and attribute patches for the point cloud using the geometry image frames and the attribute image frames included in the bitstream, wherein reconstructing the geometry patches comprises:
adding a respective quantized minimum depth for respective ones of a plurality of geometry patches to depth values included in the geometry image frames for the respective ones of the geometry patches; and
reconstruct the point cloud using the reconstructed geometry patches and the reconstructed attribute patches.

13 . The non-transitory computer-readable medium of claim 12 , wherein the bitstream further comprises respective maximum depths for the geometry patches, and wherein the program instructions cause the one or more processors to:
clip one or more depth values of one or more of the geometry patches exceeding a corresponding maximum depth for the one or more geometry patches.

14 . The non-transitory computer-readable medium of claim 13 , wherein the respective maximum depths are signaled as absolute values.

15 . The non-transitory computer-readable medium of claim 13 , wherein the respective maximum depths are signaled as delta values in relation to the quantized minimum depths, wherein the program instructions cause the one or more processors to:
add the respective delta values of the maximum depths for respective geometry patches to corresponding quantized minimum depths of the respective geometry patches to reconstruct, at a decoder, the respective maximum depths.

16 . A device, comprising:
a memory storing program instructions; and one or more processors, wherein the program instructions, when executed by the one or more processors, cause the one or more processors to:
generate, for a point cloud, one or more projected geometry patch images and attribute patch images;
pack the generated one or more projected geometry patch images and attribute patch images into one or more image frames, wherein depth values of the geometry patch images are signaled relative to one or more signaled quantized minimum depths; and
encode the one or more image frames.

17 . The device of claim 16 , further comprising:
one or more sensors configured to capture a plurality of points that make up the point cloud, wherein respective ones of the points comprise spatial information for the point and attribute information for the point.

Assignees

• Apple Inc

Inventors

• Tourapis Alexandros
• Robinet Fabrice A
• Kim Jungsun
• Mammou Khaled
• Valentin Valery G
• Su Yeping

Classifications

• G06T9/001 Primary

  Model-based coding, e.g. wire frame · CPC title

• H04N19/597

  specially adapted for multi-view video sequence encoding · CPC title