Apparatus, a method and a computer program for volumetric video

Patent metadata

Field	Value
Publication number	US-11523135-B2
Application number	US-201917045845-A
Country	US
Kind code	B2
Filing date	Mar 13, 2019
Priority date	Apr 9, 2018
Publication date	Dec 6, 2022
Grant date	Dec 6, 2022

Abstract

There are disclosed various methods, apparatuses and computer program products for volumetric video encoding and decoding. In some embodiments of a method for encoding, obtaining one or more patches formed from a three-dimensional image information are obtained. The one or more patches represent projection data of at least a part of an object to a projection plane. Priority for at least one of the one or more patches is determined and the one or more patches are projected to a projection plane. Indication of the priority is encoded into or along a bitstream. In some embodiments of a method for decoding, one or more encoded patches formed from a three-dimensional image information are received. Also at least one indication of priority determined for at least one of the one or more patches is received and the patches are reconstructed in the order defined by the at least one indication of priority.

First claim

The invention claimed is:

1. A method comprising:
obtaining one or more patches of one or more objects formed from a three-dimensional scene;
determining priority for at least one of the one or more patches of the one or more objects;
encoding, into or along a bitstream, a one bit object priority update flag that indicates whether object priority update information is present for an object associated with an index;
wherein the one bit object priority update flag having a value equal to zero indicates that object priority information is not present for the object associated with the index; and
encoding, into or along the bitstream, an object priority value that provides an indication of the priority of the object associated with the index.

2. The method according to claim 1 , wherein said determining priority for at least one of the one or more patches comprises one or more of the following:
examining an amount of motion for that part of the object or the three-dimensional scene covered with the at least one of the one or more patches;
examining an amount of high frequency components in that part of the object or the three-dimensional scene covered with the at least one of the one or more patches;
examining a distance between a user and that part of the object or the three-dimensional scene covered with the at least one of the one or more patches;
determining whether the at least one of the one or more patches covers a region of interest in the three-dimensional scene;
receiving the indication of the priority from an input provided with a user;
examining an amount of encoding/decoding complexity of that part of the object or the three-dimensional scene covered with the at least one of the one or more patches; or
examining a bitrate required with encoding/decoding of that part of the object or the three-dimensional scene covered with the at least one of the one or more patches.

3. The method according to claim 1 , further comprising:
encoding an indication of a first priority for a first patch for which the priority has been defined; and
encoding an indication of a second priority when a change in the priority occurs,
wherein the first priority is used for patches succeeding the first patch and preceding the change to the second priority.

4. The method according to claim 3 , further comprising:
indicating the second priority as a value of the second priority or a difference between the second priority and the first priority.

5. The method according to claim 1 , further comprising:
assigning the same priority for each patch representing the same object.

6. The method according to claim 1 , further comprising:
encoding into or along the bitstream the indication of priority into a set of meta data configured to deliver auxiliary patch information configured to be used to perform a transformation of the one or more patches in an atlas from a two-dimensional space to a three-dimensional space in an order indicated with the priority, the meta data comprising the one bit object priority update flag and the object priority value.

7. The method of claim 1 , wherein the lower the object priority value, the higher the priority of the object associated with the index.

8. The method of claim 1 , wherein the object priority update information when present indicates a change in the priority for the object associated with the index.

9. The method of claim 1 , wherein the one bit object priority update flag having the value equal to zero indicates that the object priority update information is not present for the object associated with the index.

10. An apparatus comprising at least one processor and at least one non-transitory memory, said at least one memory stored with code thereon, which when executed with said at least one processor, causes the apparatus to at least:
obtain one or more patches of one or more objects formed from a three-dimensional scene;
determine priority for at least one of the one or more patches of the one or more objects;
encode, into or along a bitstream, a one bit object priority update flag that indicates whether object priority update information is present for an object associated with an index;
wherein the one bit object priority update flag having a value equal to zero indicates that object priority information is not present for the object associated with the index; and
encode, into or along the bitstream, an object priority value that provides an indication of the priority of the object associated with the index.

11. The apparatus according to claim 10 , wherein to determine the priority for at least one of the one or more patches, the apparatus is further caused to perform one or more of the following:
examine an amount of motion for that part of the object or the three-dimensional scene covered with the at least one of the one or more patches;
examine an amount of high frequency components in that part of the object or the three-dimensional scene covered with the at least one of the one or more patches;
examine a distance between a user and that part of the object or the three-dimensional scene covered with the at least one of the one or more patches;
determine whether the at least one of the one or more patches covers a region of interest in the three-dimensional scene;
receive the indication of the priority from an input provided with a user;
examine an amount of encoding/decoding complexity of that part of the object or the three-dimensional scene covered with the at least one of the one or more patches; or
examine a bitrate required with encoding/decoding of that part of the object or the three-dimensional scene covered with the at least one of the one or more patches.

12. The apparatus according to claim 10 , wherein the apparatus is further caused to:
encode an indication of a first priority for a first patch for which the priority has been defined; and
encode an indication of a second priority when a change in the priority occurs,
wherein the apparatus is configured to use the first priority for patches succeeding the first patch and preceding the change to the second priority.

13. The apparatus according to claim 12 , wherein the apparatus is further caused to:
indicate the second priority as a value of the second priority or a difference between the second priority and the first priority.

14. The apparatus according to claim 10 , wherein the apparatus is further caused to:
assign the same priority for each patch representing the same object.

15. The apparatus according to claim 10 , wherein the apparatus is further caused to:
encode into or along the bitstream the indication of priority into a set of meta data configured to deliver auxiliary patch information configured to be used to perform a transformation of the one or more patches in an atlas from a two-dimensional space to a three-dimensional space in an order indicated with the priority, the meta data comprising the one bit object priority update flag and the object priority value.

16. A method comprising:
receiving one or more encoded patches of one or more objects formed from a three-dimensional scene;
receiving an object priority value that provides at least one indication of priority of an object associated with an index, the object comprising at least one of the one or more patches;
receiving a one bit object priority update flag that indicates whether object priority update information is present for the object associated with the index;
wherein the one bit object priority update flag having a value equal to zero indicates that object priority infor

Assignees

• Nokia Technologies Oy

Inventors

• Aflaki Beni Payman
• Schwarz Sebastian

Classifications

• H04N19/115

  Selection of the code volume for a coding unit prior to coding · CPC title

• H04N19/14

  Coding unit complexity, e.g. amount of activity or edge presence estimation (H04N19/146 takes precedence) · CPC title

• H04N13/161

  Encoding, multiplexing or demultiplexing different image signal components (for multi-view video sequence encoding H04N19/597) · CPC title

• H04N19/176

  the region being a block, e.g. a macroblock · CPC title

• H04N19/167

  Position within a video image, e.g. region of interest [ROI] · CPC title