Signaling of multiview tiled volumetric video

Patent metadata

Field	Value
Publication number	US-12413750-B2
Application number	US-202318520421-A
Country	US
Kind code	B2
Filing date	Nov 27, 2023
Priority date	Dec 1, 2022
Publication date	Sep 9, 2025
Grant date	Sep 9, 2025

Abstract

An apparatus includes a communication interface configured to receive a bitstream for a compressed video and a processor operably coupled to the communication interface. The processor is configured to decode the bitstream for the compressed video. The processor is also configured to identify a mapping of view numbers of a plurality of images and a plurality of subpicture identifiers, each of the plurality of subpicture identifiers associated with a defined location in a video frame, wherein the mapping is signaled in the bitstream, and wherein each one of the view numbers is assigned to one image of the plurality of images based on a corresponding one of a plurality of camera viewpoints of a scene. The processor is also configured to instruct a display of at least one image based on at least one of the plurality of images.

First claim

What is claimed is:

1. An apparatus comprising:
a communication interface configured to receive a bitstream for a compressed video; and
a processor operably coupled to the communication interface, the processor configured to:
decode the bitstream for the compressed video;
identify a mapping of view numbers of a plurality of images and a plurality of subpicture identifiers, each of the plurality of subpicture identifiers associated with a defined location in a video frame, wherein the mapping is signaled in the bitstream, and wherein each one of the view numbers is assigned to one image of the plurality of images based on a corresponding one of a plurality of camera viewpoints of a scene; and
instruct a display of at least one image based on at least one of the plurality of images.

2. The apparatus of claim 1 , wherein:
a first image set of the plurality of images is at a first resolution;
a second image set of the plurality of images is at a second resolution; and
the second resolution is a reduced resolution relative to the first resolution.

3. The apparatus of claim 2 , wherein the processor is further configured to identify metadata included in the bitstream that signals a relative resolution scale factor for each image of the plurality of images.

4. The apparatus of claim 2 , wherein the first image set at the first resolution corresponds to camera viewpoints of the plurality of camera viewpoints associated with a user view direction of the scene.

5. The apparatus of claim 4 , wherein:
the first image set and the second image set are compactly packed in the video frame to reduce empty space in the video frame according to a packing order for the view numbers; and
the mapping of the view numbers and the plurality of subpicture identifiers is based on the packing order.

6. The apparatus of claim 5 , wherein the processor is further configured to identify another packing order for another video frame in the bitstream, wherein the other packing order is based on a change in resolution associated with one or more of the view numbers and based on a change in the user view direction.

7. The apparatus of claim 1 , wherein the mapping of the view numbers and the plurality of subpicture identifiers is signaled in the bitstream in a table format.

8. A method comprising:
receiving and decoding a bitstream for a compressed video;
identifying a mapping of view numbers of a plurality of images and a plurality of subpicture identifiers, each of the plurality of subpicture identifiers associated with a defined location in a video frame, wherein the mapping is signaled in the bitstream, and wherein each one of the view numbers is assigned to one image of the plurality of images based on a corresponding one of a plurality of camera viewpoints of a scene; and
instructing a display of at least one image based on at least one of the plurality of images.

9. The method of claim 8 , wherein:
a first image set of the plurality of images is at a first resolution;
a second image set of the plurality of images is at a second resolution; and
the second resolution is a reduced resolution relative to the first resolution.

10. The method of claim 9 , further comprising identifying metadata included in the bitstream that signals a relative resolution scale factor for each image of the plurality of images.

11. The method of claim 9 , wherein the first image set at the first resolution corresponds to camera viewpoints of the plurality of camera viewpoints associated with a user view direction of the scene.

12. The method of claim 11 , wherein:
the first image set and the second image set are compactly packed in the video frame to reduce empty space in the video frame according to a packing order for the view numbers; and
the mapping of the view numbers and the plurality of subpicture identifiers is based on the packing order.

13. The method of claim 12 , further comprising identifying another packing order for another video frame in the bitstream, wherein the other packing order is based on a change in resolution associated with one or more of the view numbers and based on a change in the user view direction.

14. The method of claim 8 , wherein the mapping of the view numbers and the plurality of subpicture identifiers is signaled in the bitstream in a table format.

15. An apparatus comprising:
a communication interface; and
a processor operably coupled to the communication interface, the processor configured to:
obtain a plurality of images each corresponding to one of a plurality of camera viewpoints of a scene;
assign a view number to each image of the plurality of images based on the corresponding one of the plurality of camera viewpoints of the scene;
map the view number of each image of the plurality of images to one of a plurality of subpicture identifiers each associated with a defined location in a video frame;
combine the plurality of images into the video frame according to the subpicture identifiers; and
compress and transmit the video frame as part of a bitstream, wherein the mapping of the view numbers and the plurality of subpicture identifiers is signaled in the bitstream.

16. The apparatus of claim 15 , wherein, to combine the plurality of images into the video frame, the processor is further configured to:
provide a first image set of the plurality of images at a first resolution;
provide a second image set of the plurality of images at a second resolution, wherein the second resolution is a reduced resolution relative to the first resolution; and
store the first image set and the second image set of the plurality of images in the video frame.

17. The apparatus of claim 16 , wherein the processor is further configured to encode metadata in the bitstream signaling a relative resolution scale factor for each image of the plurality of images.

18. The apparatus of claim 16 , wherein the first image set at the first resolution corresponds to camera viewpoints of the plurality of camera viewpoints associated with a user view direction of the scene.

19. The apparatus of claim 18 , wherein, to map the view number of each image of the plurality of images to one of the plurality of subpicture identifiers, the processor is further configured to:
determine a packing order for the view numbers in which the first image set and the second image set are compactly packed in the video frame to reduce empty space in the video frame; and
map the view number of each image of the plurality of images to one of the plurality of subpicture identifiers according to the packing order.

20. The apparatus of claim 19 , wherein the processor is further configured to:
identify a change in the user view direction; and
determine a new packing order for the view numbers based on a change in resolution associated with one or more of the view numbers, where the change in resolution is based on the change in the user view direction.

Assignees

• Samsung Electronics Co Ltd

Inventors

• Budagavi Madhukar
• Joshi Rajan Laxman
• Lim Youngkwon

Classifications

• H04N19/119 Primary

  Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks · CPC title

• H04N19/70

  characterised by syntax aspects related to video coding, e.g. related to compression standards · CPC title

• H04N19/597

  specially adapted for multi-view video sequence encoding · CPC title

• H04N19/184 Primary

  the unit being bits, e.g. of the compressed video stream · CPC title