Codec architecture for multiple layer video coding

Patent metadata

Field	Value
Publication number	US-11627340-B2
Application number	US-202117246025-A
Country	US
Kind code	B2
Filing date	Apr 30, 2021
Priority date	Jul 9, 2012
Publication date	Apr 11, 2023
Grant date	Apr 11, 2023

Abstract

Systems, methods, and instrumentalities are provided to implement video coding system (VCS). The VCS may be configured to receive a video signal, which may include one or more layers (e.g., a base layer (BL) and/or one or more enhancement layers (ELs)). The VCS may be configured to process a BL picture into an inter-layer reference (ILR) picture, e.g., using picture level inter-layer prediction process. The VCS may be configured to select one or both of the processed ILR picture or an enhancement layer (EL) reference picture. The selected reference picture(s) may comprise one of the EL reference picture, or the ILR picture. The VCS may be configured to predict a current EL picture using one or more of the selected ILR picture or the EL reference picture. The VCS may be configured to store the processed ILR picture in an EL decoded picture buffer (DPB).

First claim

What is claimed is:

1. A method comprising:
receiving base layer (BL) coded data, enhancement layer (EL) coded data, and inter-layer prediction (ILP) information, wherein the ILP information is packetized in a network abstraction unit separate from network abstraction units containing the BL coded data and the EL coded data;
reconstructing a BL picture based on the BL coded data;
obtaining an inter-layer reference (ILR) picture based on the ILP information; and
reconstructing an EL picture based on the ILR picture and the EL coded data.

2. The method of claim 1 , wherein the ILP information comprises upsampling filter information.

3. The method of claim 1 , wherein the ILP information comprises ILP parameter data.

4. The method of claim 1 , wherein the ILP information is packetized in a plurality of network abstraction units, each of the network abstraction units being associated with a unit type.

5. The method of claim 1 , wherein the ILP information comprises at least one of upsampling filter information packetized in a network abstraction unit of a first unit type, one or more of disparity compensation parameters packetized in a network abstraction unit of a second unit type, or one or more of inverse tone mapping parameters packetized in a network abstraction unit of a third unit type.

6. The method of claim 1 , further comprising demuxing a video signal to separate the network abstraction unit containing the ILP information from the network abstraction units containing the BL coded data and the EL coded data.

7. The method of claim 1 , wherein the reconstructed EL picture and the ILR picture correspond to a same time instance.

8. The method of claim 1 , wherein the ILR picture is comprised in a set of reference pictures that further comprises a reference EL picture, and reconstructing the EL picture comprises:
predicting a first portion of the EL picture using the ILR picture; and
predicting a second portion of the EL picture using the reference EL picture.

9. A system comprising:
a processor configured to:
receive a video signal comprising base layer (BL) coded data, enhancement layer (EL) coded data, and inter-layer prediction (ILP) information, wherein the ILP information is packetized in a network abstraction unit separate from network abstraction units containing the BL coded data and the EL coded data;
reconstruct a BL picture based on the BL coded data;
obtain an inter-layer reference (ILR) picture based on the ILP information; and
reconstruct an EL picture based on the ILR picture and the EL coded data.

10. The system of claim 9 , wherein the ILP information comprises upsampling filter information.

11. The system of claim 9 , wherein the ILP information comprises ILP parameter data.

12. The system of claim 9 , wherein the reconstructed EL picture and the ILR picture correspond to a same time instance.

13. The system of claim 9 , wherein the network abstraction unit containing the ILP information is in a first mode, and the network abstraction units containing the BL coded data and the EL coded data are in a second mode.

14. The system of claim 9 , further comprising:
a memory communicatively coupled with the processor.

15. A non-transitory computer readable medium having stored thereon instructions executable by one or more processors for performing operations comprising:
receiving base layer (BL) coded data, enhancement layer (EL) coded data, and inter-layer prediction (ILP) information, wherein the ILP information is packetized in a network abstraction unit separate from network abstraction units containing the BL coded data and the EL coded data;
reconstructing a BL picture based on the BL coded data;
obtaining an inter-layer reference (ILR) picture based on the ILP information; and
reconstructing an EL picture based on the ILR picture and the EL coded data.

16. The non-transitory computer readable medium of claim 15 , wherein the ILP information comprises upsampling filter information.

17. The non-transitory computer readable medium of claim 15 , wherein the ILP information comprises ILP parameter data.

18. The non-transitory computer readable medium of claim 15 , wherein the ILP information is packetized in a plurality of network abstraction units, each of the network abstraction units being associated with a unit type.

19. The non-transitory computer readable medium of claim 15 , wherein the ILP information comprises at least one of upsampling filter information packetized in a network abstraction unit of a first unit type, one or more of disparity compensation parameters packetized in a network abstraction unit of a second unit type, or one or more of inverse tone mapping parameters packetized in a network abstraction unit of a third unit type.

20. The non-transitory computer readable medium of claim 15 , having stored thereon instructions executable by one or more processors for performing operations further comprising demuxing a video signal to separate the network abstraction unit containing the ILP information from the network abstraction units containing the BL coded data and the EL coded data.

21. The non-transitory computer readable medium of claim 15 , wherein the reconstructed EL picture and the ILR picture correspond to a same time instance.

22. The non-transitory computer readable medium of claim 15 , wherein the ILR picture is comprised in a set of reference pictures that further comprises a reference EL picture, and reconstructing the EL picture comprises:
predicting a first portion of the EL picture using the ILR picture; and
predicting a second portion of the EL picture using the reference EL picture.

23. A method comprising:
determining enhancement layer (EL) coded data associated with video data;
determining base layer (BL) coded data associated with the video data;
determining inter-layer prediction (ILP) information data associated with the video data; and
generating a bitstream that comprises the BL coded data, the EL coded data, and the ILP information into a bitstream, the ILP information packetized in one or more network abstraction units separate from network abstraction units comprising the BL coded data and the EL coded data.

24. The method of claim 23 , wherein the ILP information comprises upsampling filter information.

25. The method of claim 23 , wherein the ILP information comprises ILP parameter data.

26. The method of claim 23 , wherein the ILP information is packetized in a plurality of network abstraction units, each of the plurality of network abstraction units associated with a unit type.

27. The method of claim 23 , wherein the ILP information comprises at least one of upsampling filter information packetized in a network abstraction unit of a first unit type, one or more of disparity compensation parameters packetized in a network abstraction unit of a second unit type, or one or more of inverse tone mapping parameters packetized in a network abstraction unit of a third unit type.

28. A system comprising:
a processor configured to:
determine enhancement layer (EL) coded data associated with video data;
determine base layer (BL) coded data associated with the video data;
determine inter-layer prediction (ILP) information data associated with the video data; and
generate a bistream that comprises the BL coded data, the EL coded data, and the ILP information into a bit

Assignees

• Vid Scale Inc

Inventors

• Ye Yan
• Mcclellan George W
• He Yong
• Xiu Xiaoyu
• He Yuwen
• Dong Jie
• Bal Can
• Ryu Eun Seok

Classifications

• H04N19/33

  in the spatial domain · CPC title

• H04N19/187

  the unit being a scalable video layer · CPC title

• H04N19/30 Primary

  using hierarchical techniques, e.g. scalability (H04N19/63 takes precedence) · CPC title

• H04N19/36

  Scalability techniques involving formatting the layers as a function of picture distortion after decoding, e.g. signal-to-noise [SNR] scalability · CPC title

• H04N19/31

  in the temporal domain · CPC title