Object-based volumetric video coding

Patent metadata

Field	Value
Publication number	US-12230002-B2
Application number	US-202017631301-A
Country	US
Kind code	B2
Filing date	Sep 30, 2020
Priority date	Oct 1, 2019
Publication date	Feb 18, 2025
Grant date	Feb 18, 2025

Abstract

Methods, apparatus, systems and articles of manufacture for object-based volumetric video coding are disclosed. An example apparatus disclosed herein includes a point annotator to receive point cloud data associated with an object and annotate points of the point cloud data with an object identifier of the object. The disclosed example apparatus also includes a projector to project the point cloud data onto projection planes to produce texture images and geometry images. The disclosed example apparatus further includes a patch generator to generate a patch based on the object identifier, the patch including the texture images and the geometry images of the object, the patch associated with the object identifier of the object. The disclosed example apparatus also includes an atlas generator to generate an atlas to include in encoded video data, the atlas including the patch.

First claim

What is claimed is:

1. A video encoder comprising:
interface circuitry;
instructions; and
at least one processor circuit to be programmed by the instructions to:
project first point cloud data associated with a first object identifier onto a first pair of projection layers of a projection plane, the first pair of projection layers to be tagged with the first object identifier to associate the first pair of projection layers with a first object;
project second point cloud data associated with a second object identifier onto a second pair of projection layers of the projection plane, the second pair of projection layers different from the first pair of projection layers, the second object identifier different from the first object identifier, the second pair of projection layers to be tagged with the second object identifier to associate the second pair of projection layers with a second object different than the first object;
generate a patch based on at least one projection layer of the first pair of projection layers, the patch associated with the first object identifier of the first object; and
generate an atlas to include in encoded video data, the atlas including the patch.

2. The video encoder of claim 1 , wherein the projection plane is a first projection plane, and one or more of the at least one processor circuit is to:
project the first point cloud data associated with the first object identifier onto a third pair of projection layers assigned to the first object, the third pair of projection layers associated with a second projection plane; and
project the second point cloud data associated with the second object identifier onto a fourth pair of projection layers assigned to the second object, the fourth pair of projection layers associated with the second projection plane.

3. The video encoder of claim 1 , wherein the first pair of projection layers includes a first near layer and a first far layer assigned to the first object, and the second pair of projection layers includes a second near layer and a second far layer assigned to the second object, the second near layer different from the first near layer, and the second far layer different from the first far layer.

4. The video encoder of claim 1 , wherein one or more of the at least one processor circuit is to order the first pair of projection layers before the second pair of projection layers when the first object is closer to the projection plane than the second object.

5. The video encoder of claim 1 , wherein the patch is a first patch associated with the first object identifier, and one or more of the at least one processor circuit is to generate a second patch based on at least one projection layer of the second pair of projection layers, the second patch associated with the second object identifier.

6. The video encoder of claim 5 , wherein one or more of the at least one processor circuit is to stack the first patch before the second patch when the first object is closer to the projection plane than the second object.

7. The video encoder of claim 1 , wherein one or more of the at least one processor circuit is to annotate points of the first point cloud data with the first object identifier and annotate points of the second point cloud data with the second object identifier.

8. The video encoder of claim 1 , wherein the patch is a first patch, and one or more of the at least one processor circuit is to generate a second patch associated with the second object identifier, the second patch at a lower resolution than the first patch.

9. At least one non-transitory computer readable medium comprising instructions to cause at least one processor circuit to at least:
project first point cloud data associated with a first object identifier onto a first pair of projection layers of a projection plane, the first pair of projection layers to be tagged with the first object identifier to associate the first pair of projection layers with a first object;
project second point cloud data associated with a second object identifier onto a second pair of projection layers of the projection plane, the second pair of projection layers different from the first pair of projection layers, the second object identifier different from the first object identifier, the second pair of projection layers to be tagged with the second object identifier to associate the second pair of projection layers with a second object different than the first object;
generate a patch based on at least one projection layer of the first pair of projection layers, the patch associated with the first object identifier of the first object; and
generate an atlas to include in encoded video data, the atlas including the patch.

10. The at least one non-transitory computer readable medium of claim 9 , wherein the projection plane is a first projection plane, and the instructions are to cause one or more of the at least one processor circuit to:
project the first point cloud data associated with the first object identifier onto a third pair of projection layers assigned to the first object, the third pair of projection layers associated with a second projection plane; and
project the second point cloud data associated with the second object identifier onto a fourth pair of projection layers assigned to the second object, the fourth pair of projection layers associated with the second projection plane.

11. The at least one non-transitory computer readable medium of claim 9 , wherein the first pair of projection layers includes a first near layer and a first far layer assigned to the first object, and the second pair of projection layers includes a second near layer and a second far layer assigned to the second object, the second near layer different from the first near layer, and the second far layer different from the first far layer.

12. The at least one non-transitory computer readable medium of claim 9 , wherein the instructions are to cause one or more of the at least one processor circuit to order the first pair of projection layers before the second pair of projection layers when the first object is closer to the projections plane than the second object.

13. The at least one non-transitory computer readable medium of claim 9 , wherein the patch is a first patch associated with the first object identifier, and the instructions are to cause one or more of the at least one processor circuit to generate a second patch based on at least one projection layer of the second pair of projection layers, the second patch associated with the second object identifier.

14. The at least one non-transitory computer readable medium of claim 13 , wherein the instructions are to cause one or more of the at least one processor circuit to stack the first patch before the second patch when the first object is closer to the projection plane than the second object.

15. A video encoding method comprising:
projecting, by at least one processor circuit programmed by at least one instruction, a first portion of point cloud data associated with an object onto a first pair of projection layers assigned to the object, the first pair of projection layers associated with a first one of a plurality of projection planes, the first pair of projection layers tagged with an identifier of the object to associate the first pair of projection layers with a first object;
projecting, by one or more of the at least one processor circuit, a second portion of point cloud data associated with a background onto a second pair of projection layers assigned to the background, the second pair of projection layers associated with the firs

Assignees

• Intel Corp

Inventors

• Salahieh Basel
• Yeung Fai
• Boyce Jill

Classifications

• G06V10/50

  by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis · CPC title

• H04N19/20

  using video object coding · CPC title

• H04N19/597

  specially adapted for multi-view video sequence encoding · CPC title

• G06T9/00 Primary

  Image coding (bandwidth or redundancy reduction for static pictures H04N1/41; coding or decoding of static colour picture signals H04N1/64; methods or arrangements for coding, decoding, compressing or decompressing digital video signals H04N19/00) · CPC title

• G06T9/001 Primary

  Model-based coding, e.g. wire frame · CPC title