Codec architecture for multiple layer video coding

Patent metadata

Field	Value
Publication number	US-9998764-B2
Application number	US-201313937645-A
Country	US
Kind code	B2
Filing date	Jul 9, 2013
Priority date	Jul 9, 2012
Publication date	Jun 12, 2018
Grant date	Jun 12, 2018

Abstract

Systems, methods, and instrumentalities are provided to implement video coding system (VCS). The VCS may be configured to receive a video signal, which may include one or more layers (e.g., a base layer (BL) and/or one or more enhancement layers (ELs)). The VCS may be configured to process a BL picture into an inter-layer reference (ILR) picture, e.g., using picture level inter-layer prediction process. The VCS may be configured to select one or both of the processed ILR picture or an enhancement layer (EL) reference picture. The selected reference picture(s) may comprise one of the EL reference picture, or the ILR picture. The VCS may be configured to predict a current EL picture using one or more of the selected ILR picture or the EL reference picture. The VCS may be configured to store the processed ILR picture in an EL decoded picture buffer (DPB).

First claim

What is claimed is:

1. A video coding method comprising:
receiving a video signal comprising a base layer (BL) and an enhancement layer (EL);
determining a first scalability type that relates the EL to the BL and a second scalability type that relates the EL to the BL based on the video signal;
reconstructing a BL picture from the BL;
determining, based on the first scalability type, a first picture level inter-layer process for processing the reconstructed BL picture into an inter-layer reference (ILR) picture;
determining, based on the second scalability type, a second picture level inter-layer process for processing the reconstructed BL picture into the ILR picture;
performing the determined first picture level inter-layer process and the determined second picture level inter-layer process on the reconstructed BL picture to generate the ILR picture; and
in response to a determination that a set of reference pictures associated with a current EL picture comprises a reference EL picture and the ILR picture, predicting the current EL picture using the ILR picture and the reference EL picture.

2. The method of claim 1 , further comprising storing the ILR picture in an EL decoded picture buffer (DPB).

3. The method of claim 1 , wherein the ILR picture comprises a non-collocated ILR picture.

4. The method of claim 1 , wherein, in response to a determination that the first scalability type comprises at least one of spatial scalability or chroma format scalability, the first picture level inter-layer process is determined to comprise upsampling.

5. The method of claim 1 , wherein, in response to a determination that the first scalability type comprises at least one of quality scalability, spatial scalability, view scalability, standard scalability, or aspect ratio scalability, the first picture level inter-layer process is determined to comprise denoising.

6. The method of claim 1 , wherein, in response to a determination that the first scalability type comprises view scalability, the first picture level inter-layer process is determined to comprise disparity compensation.

7. The method of claim 1 , wherein, in response to a determination that the first scalability type comprises bit-depth scalability, the first picture level inter-layer process is determined to comprise inverse tone mapping.

8. The method of claim 1 , wherein, in response to a determination that the first scalability type comprises spatial scalability, the first picture level inter-layer process is determined to comprise upsampling, and in response to a determination that the second scalability type comprises bit-depth scalability, the second picture level inter-layer process is determined to comprise inverse tone mapping.

9. The method of claim 1 , wherein the current EL picture and the ILR picture are associated with a first format, and the reconstructed BL picture is associated with a second format.

10. The method of claim 1 , wherein predicting the current EL picture further comprises:
predicting a first portion of the current EL picture using the ILR picture; and
predicting a second portion of the current EL picture using the reference EL picture.

11. The method of claim 1 , the method further comprises:
receiving packetized inter-layer prediction (ILP) information, wherein the first picture level inter-layer process and the second picture level inter-layer process are performed based on the packetized ILP information.

12. The method of claim 11 , wherein the packetized ILP information comprises at least one of upsampling filter information, denoising information, one or more of display compensation parameters, or one or more of inverse tone mapping parameters.

13. The method of claim 12 , further comprising:
in response to a determination that the first scalability type comprises at least one of spatial scalability or chroma format scalability, extracting the upsampling filter information from the video signal, wherein the first picture level inter-layer process is performed using the extracted upsampling filter information in response to a determination that the first scalability type comprises at least one of the spatial scalability or the chroma format scalability.

14. The method of claim 12 , the method further comprising, in response to a determination that the fist scalability type comprises bit-depth scalability, extracting the one or more of inverse tone mapping parameters from the video signal, wherein the first picture level inter-layer process is performed using the extracted one or more of inverse tone mapping parameters in response to a determination that the first scalability type comprises the bit-depth scalability.

15. The method of claim 11 , wherein the packetized ILP information is received via a separate network abstraction layer (NAL) unit from a NAL carrying BL information or EL information.

16. A video coding method comprising:
receiving a video signal comprising a base layer (BL) and an enhancement layer (EL);
determining a scalability type that relates the EL to the BL based on the video signal;
reconstructing a BL picture from the BL;
determining, based on the scalability type, a picture level inter-layer process for processing the reconstructed BL picture into an inter-layer reference (ILR) picture;
performing the determined picture level inter-layer process on the reconstructed BL picture to generate the ILR picture; and
in response to a determination that a set of reference pictures associated with a current EL picture comprises a reference EL picture and the ILR picture, predicting a first portion of the current EL picture using the ILR picture and predicting a second portion of the current EL picture using the reference EL picture.

17. The method of claim 16 , further comprising storing the ILR picture in an EL decoded picture buffer (DPB).

18. The method of claim 16 , wherein the ILR picture comprises a non-collocated ILR picture.

19. The method of claim 16 , the method further comprises:
determining a second scalability type that relates the EL to the BL based on the video signal;
determining, based on the second scalability type, a second picture level inter-layer process for processing the reconstructed BL picture into the ILR picture; and
performing the second picture level inter-layer process on the reconstructed BL picture to generate the ILR picture.

20. The method of claim 16 , the method further comprises:
determining a second scalability type that relates the EL to the BL based on the video signal; and
determining, based on the scalability type and the second scalability type, a combined picture level inter-layer process for processing the reconstructed BL picture into the ILR picture, wherein performing the determined picture level inter-layer process on the reconstructed BL picture to generate the ILR picture comprises performing the combined picture level inter-layer process on the reconstructed BL picture to generate the ILR picture.

21. The method of claim 16 , wherein, in response to a determination that the scalability type comprises at least one of spatial scalability or chroma format scalability, the picture level inter-layer process is determined to comprise upsampling.

22. The method of claim 16 , wherein, in response to a determination that the scalability type comprises bit-depth scalability, the picture level inter-layer process is determined to comprise inverse tone mapping.

23. The method of claim 16 , wher

Assignees

• Vid Scale Inc

Inventors

• Ye Yan
• Mcclellan George W
• He Yong
• Xiu Xiaoyu
• He Yuwen
• Dong Jie
• Bal Can
• Ryu Eun

Classifications

• H04N19/105 Primary

  Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction · CPC title

• H04N19/36

  Scalability techniques involving formatting the layers as a function of picture distortion after decoding, e.g. signal-to-noise [SNR] scalability · CPC title

• H04N19/30 Primary

  using hierarchical techniques, e.g. scalability (H04N19/63 takes precedence) · CPC title

• H04N13/161

  Encoding, multiplexing or demultiplexing different image signal components (for multi-view video sequence encoding H04N19/597) · CPC title

• H04N19/503

  involving temporal prediction (adaptive coding with adaptive selection between spatial and temporal predictive coding H04N19/107; adaptive coding with adaptive selection among a plurality of temporal predictive coding modes H04N19/109) · CPC title