Conditional recolor for video based point cloud coding

Patent metadata

Field	Value
Publication number	US-11979606-B2
Application number	US-202117495301-A
Country	US
Kind code	B2
Filing date	Oct 6, 2021
Priority date	Jun 4, 2021
Publication date	May 7, 2024
Grant date	May 7, 2024

Abstract

Methods and apparatuses of encoding a video stream using video point cloud coding include obtaining a source point cloud; generating a geometry map and a texture map based on the source point cloud; generating a reconstructed geometry map and a reconstructed texture map using lossy coding without using recoloring, wherein a resolution of the reconstructed geometry map is same as a resolution of the reconstructed texture map; obtaining a point of the source point cloud corresponding to a geometry pixel of the geometry map and a texture pixel of the texture map; selecting a color of the texture pixel based on a color of the obtained point; and generating an encoded video stream based on the selected color, the geometry map, and the texture map.

First claim

What is claimed is:

1. A method of encoding a video stream using video point cloud coding, the method being performed by at least one processor and comprising:
obtaining a source point cloud;
generating a geometry map and a texture map based on the source point cloud;
generating a reconstructed geometry map and a reconstructed texture map using lossy coding,
wherein the reconstructed geometry map and the reconstructed texture map are generated without using recoloring, and
wherein a resolution of the reconstructed geometry map is same as a resolution of the reconstructed texture map;

obtaining a point of the source point cloud corresponding to a geometry pixel of the geometry map and a texture pixel of the texture map;
selecting a color of the texture pixel corresponding to a color of the obtained point,
wherein the texture pixel corresponds to a semi-occupied pixel which is not occupied in an occupancy map and which is occupied in a reconstructed occupancy map, and
wherein the color of the texture pixel is selected based on an average value of a plurality of occupied pixels in a same block of the occupancy map, wherein the block of the occupancy map is a precision with which the occupancy map is coded; and

generating an encoded video stream based on the selected color, the geometry map, and the texture map.

2. The method of claim 1 , further comprising:
generating the occupancy map based on the source point cloud;
generating the reconstructed occupancy map using the lossy coding and without using the recoloring, wherein a resolution of the reconstructed occupancy map is same as the resolution of the reconstructed geometry map and the resolution of the reconstructed texture map,
wherein the color of the texture pixel is selected based on the occupancy map.

3. The method of claim 1 ,
wherein the color of the texture pixel is selected based on a distance-weighted average value of a plurality of nearest neighboring occupied pixels of the same block.

4. The method of claim 1 ,
wherein a color value of the semi-occupied pixel which is not occupied in the occupancy map and which is occupied in the reconstructed occupancy map is determined using the recoloring.

5. The method of claim 4 , wherein a plurality of color values of a plurality of pixels included in a block including the semi-occupied pixel are determined using the recoloring.

6. The method of claim 1 , wherein the point is obtained based on a comparison between the geometry map and the reconstructed geometry map.

7. An apparatus for encoding a video stream using video point cloud coding, the apparatus comprising:
at least one memory configured to store program code; and
at least one processor configured to read the program code and operate as instructed by the program code, the program code comprising:
first obtaining code configured to cause the at least one processor to obtain a source point cloud;
first generating code configured to cause the at least one processor to generate a geometry map and a texture map based on the source point cloud;
second generating code configured to cause the at least one processor to generate a reconstructed geometry map and a reconstructed texture map using lossy coding,
wherein the reconstructed geometry map and the reconstructed texture map are generated without using recoloring, and
wherein a resolution of the reconstructed geometry map is same as a resolution of the reconstructed texture map;

second obtaining code configured to cause the at least one processor to obtain a point of the source point cloud corresponding to a geometry pixel of the geometry map and a texture pixel of the texture map;
selecting code configured to cause the at least one processor to select a color of the texture pixel corresponding to a color of the obtained point,
wherein the texture pixel corresponds to a semi-occupied pixel which is not occupied in an occupancy map and which is occupied in a reconstructed occupancy map, and
wherein the color of the texture pixel is selected based on an average value of a plurality of occupied pixels in a same block of the occupancy map, wherein the block of the occupancy map is a precision with which the occupancy map is coded; and

third generating code configured to cause the at least one processor to generate an encoded video stream based on the selected color, the geometry map, and the texture map.

8. The apparatus of claim 7 , further comprising:
third generating code configured to cause the at least one processor to generate the occupancy map based on the source point cloud;
fourth generating code configured to cause the at least one processor to generate the reconstructed occupancy map using the lossy coding and without using the recoloring, wherein a resolution of the reconstructed occupancy map is same as the resolution of the reconstructed geometry map and the resolution of the reconstructed texture map,
wherein the color of the texture pixel is selected based on the occupancy map.

9. The apparatus of claim 7 ,
wherein the color of the texture pixel is selected based on a distance-weighted average value of a plurality of nearest neighboring occupied pixels of the same block.

10. The apparatus of claim 7 ,
wherein a color value of the semi-occupied pixel which is not occupied in the occupancy map and which is occupied in the reconstructed occupancy map is determined using the recoloring.

11. The apparatus of claim 10 , wherein a plurality of color values of a plurality of pixels included in a block including the semi-occupied pixel are determined using the recoloring.

12. The apparatus of claim 7 , wherein the point is obtained based on a comparison between the geometry map and the reconstructed geometry map.

13. A non-transitory computer-readable medium storing computer instructions encoding a video stream using video point cloud coding that, when executed by at least one processor, cause the at least one processor to:
obtain a source point cloud;
generate a geometry map and a texture map based on the source point cloud;
generate a reconstructed geometry map and a reconstructed texture map using lossy coding,
wherein the reconstructed geometry map and the reconstructed texture map are generated without using recoloring, and
wherein a resolution of the reconstructed geometry map is same as a resolution of the reconstructed texture map;

obtain a point of the source point cloud corresponding to a geometry pixel of the geometry map and a texture pixel of the texture map;
select a color of the texture pixel corresponding to a color of the obtained point,
wherein the texture pixel corresponds to a semi-occupied pixel which is not occupied in an occupancy map and which is occupied in a reconstructed occupancy map, and
wherein the color of the texture pixel is selected based on an average value of a plurality of occupied pixels in a same block of the occupancy map, wherein the block of the occupancy map is a precision with which the occupancy map is coded; and

generate an encoded video stream based on the selected color, the geometry map, and the texture map.

14. The non-transitory computer-readable medium of claim 13 , further comprising:
generate the occupancy map based on the source point cloud;
generate the reconstructed occupancy map using the lossy coding and without using the recoloring, wherein a resolution of the reconstructed occupancy map is same as the resolution of the reconstructed geometry map and the resolution of the reconstructed texture map,
wherein the color of the texture pixel is selected based on the occupancy map.

15

Assignees

• Tencent America LLC

Inventors

• Zhang Xiang
• Gao Wen
• Liu Shan
• Feng Weiwei
• Jian Bing

Classifications

• H04N19/597 Primary

  specially adapted for multi-view video sequence encoding · CPC title

• G06T15/04

  Texture mapping · CPC title

• H04N19/184

  the unit being bits, e.g. of the compressed video stream · CPC title

• G06T2210/56

  Particle system, point based geometry or rendering · CPC title

• G06T9/001 Primary

  Model-based coding, e.g. wire frame · CPC title