Inter-layer residual prediction

Patent metadata

Field	Value
Publication number	US-10764592-B2
Application number	US-201213996577-A
Country	US
Kind code	B2
Filing date	Sep 28, 2012
Priority date	Sep 28, 2012
Publication date	Sep 1, 2020
Grant date	Sep 1, 2020

Abstract

Systems, devices and methods are described including performing scalable video coding using inter-layer residual prediction. Inter-layer residual prediction in an enhancement layer coding unit, prediction unit, or transform unit may use residual data obtained from a base layer or from a lower enhancement layer. The residual may be subjected to upsample filtering and/or refinement filtering. The upsample or refinement filter coefficients may be predetermined or may be adoptively determined.

First claim

What is claimed:

1. A method, comprising:
at a scalable video decoder:
accessing, for a block of an enhancement layer (EL) frame of scalable video content, multiple co-located blocks in one of a lower level EL frame or a base layer (BL) frame; and
performing inter-layer residual prediction for a first region of the block of the EL frame responsive to a first block of the co-located blocks in the lower level EL frame or base layer (BL) frame corresponding to the first region being an intra coded block, wherein only regions of the block of the EL frame corresponding to intra coded blocks of the multiple co-located blocks in the lower level EL frame or base layer (BL) frame are inter-layer residual predicted, and wherein performing inter-layer residual prediction for the first region of the block of the EL frame comprises:
determining a residual from the first block of the co-located blocks in the lower level EL frame or the BL frame;
applying an upsample filter and a refining filter to the residual to generate an upsampled and filtered residual; and
determining a predicted residual for the block of the EL frame based at least in part on adding the upsampled and filtered residual to a second residual of the first region of the block of the EL frame.

2. The method of claim 1 , further comprising accessing the residual in memory.

3. The method of claim 1 , wherein accessing the multiple co-located blocks for inter-layer residual prediction is in response to an indicator included in a bitstream received at the scalable video decoder.

4. The method of claim 3 , wherein, in a first state, the indicator specifies that the scalable video decoder is to perform inter-layer residual prediction, and wherein, in a second state, the indicator specifies that the scalable video decoder is not to perform inter-layer residual prediction.

5. The method of claim 4 , wherein the indicator has been placed in one of the first state or the second state based on a rate-distortion cost.

6. The method of claim 3 , further comprising:
determining a second block of the co-located blocks in the lower level EL frame or base layer (BL) frame corresponding to a second region of the block of the EL frame is an inter coded block, wherein inter-layer residual prediction is not performed for the second region responsive to the second block of the co-located blocks being an inter coded block.

7. The method of claim 6 , wherein the upsample filter comprises adaptive upsample coefficients, the refining filter comprises adaptive refining coefficients, and the adaptive upsample coefficients and the adaptive refining coefficients are in the bitstream received at the scalable video decoder.

8. At least one non-transitory machine-readable medium comprising a plurality of instructions that, in response to being executed on a computing device, cause the computing device to perform scalable video decoding by:
accessing, for a block of an enhancement layer (EL) frame of scalable video content, multiple co-located blocks in one of a lower level EL frame or a base layer (BL) frame; and
performing inter-layer residual prediction for a first region of the block of the EL frame responsive to a first block of the co-located blocks in the lower level EL frame or base layer (BL) frame corresponding to the first region being an intra coded block, wherein only regions of the block of the EL frame corresponding to intra coded blocks of the multiple co-located blocks in the lower level EL frame or base layer (BL) frame are inter-layer residual predicted, and wherein performing inter-layer residual prediction for the first region of the block of the EL frame comprises:
determining a residual from the first block of the co-located blocks in the lower level EL frame or the BL frame;
applying an upsample filter and a refining filter to the residual to generate an upsampled and filtered residual; and
determining a predicted residual for the block of the EL frame based at least in part on adding the upsampled and filtered residual to a second residual of the first region of the block of the EL frame.

9. The at least one machine-readable medium of claim 8 , further comprising one or more instructions that, in response to being executed on the computing device, cause the computing device to access the residual in memory.

10. The at least one machine-readable medium of claim 8 , wherein accessing the multiple co-located blocks for inter-layer residual prediction is in response to an indicator included in a bitstream received at the computing device, wherein, in a first state, the indicator specifies that the computing device is to perform inter-layer residual prediction, and wherein, in a second state, the indicator specifies that the computing device is not to perform inter-layer residual prediction.

11. The at least one machine-readable medium of claim 8 , wherein the indicator has been placed in one of the first state or the second state based on a rate-distortion cost.

12. The at least one machine-readable medium of claim 10 , further comprising one or more instructions that, in response to being executed on the computing device, cause the computing device to determine a second block of the co-located blocks in the lower level EL frame or base layer (BL) frame corresponding to a second region of the block of the EL frame is an inter coded block, wherein inter-layer residual prediction is not performed for the second region responsive to the second block of the co-located blocks being an inter coded block.

13. The at least one machine-readable medium of claim 12 , wherein the upsample filter comprises adaptive upsample coefficients, the refining filter comprises one adaptive refining coefficients, and the adaptive upsample coefficients and the adaptive refining coefficients are in the bitstream received at the scalable video decoder.

14. An apparatus, comprising:
memory to store scalable video content; and
circuitry to access, for a block of an enhancement layer (EL) frame of the scalable video content, multiple co-located blocks in one of a lower level EL frame or a base layer (BL) frame, to perform inter-layer residual prediction for a first region of the block of the EL frame responsive to a first block of the co-located blocks in the lower level EL frame or base layer (BL) frame corresponding to the first region being an intra coded block, wherein only regions of the block of the EL frame corresponding to intra coded blocks of the multiple co-located blocks in the lower level EL frame or base layer (BL) frame are inter-layer residual predicted, and wherein to perform inter-layer residual prediction for the first region of the block of the EL frame, the circuitry is to access the memory to obtain a residual, wherein the residual is from the first block of the co-located blocks in the lower level EL frame or the BL frame, to apply an upsample filter and a refining filter to the residual to generate an upsampled and filtered residual, and to determine a predicted residual for the block of the EL frame based on an addition of the upsampled and filtered residual to a second residual of the first region of the block of the EL frame.

15. The apparatus of claim 14 , wherein the circuitry to access the multiple co-located blocks for inter-layer residual prediction is in response to an indicator stored in the memory, wherein, in a first state, the indicator specifies that the circuitry is to perform inter-layer residual prediction, and wherein, in a second state, the indicator specifies that the circuitry is not to perform inter-layer residual prediction.

16. The appa

Assignees

• Intel Corp

Inventors

• Zhang Wenhao
• Chiu Yi-Jen
• Xu Lidong
• Han Yu
• Deng Zhipin
• Cai Xiaoxia

Classifications

• H04N19/30 Primary

  using hierarchical techniques, e.g. scalability (H04N19/63 takes precedence) · CPC title

• H04N19/59 Primary

  involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution · CPC title

• H04N19/61

  in combination with predictive coding · CPC title

• H04N19/463

  by compressing encoding parameters before transmission · CPC title

• H04N19/50

  using predictive coding (H04N19/61 takes precedence) · CPC title