Rate-distortion defined interpolation for video coding based on fixed filter or adaptive filter

Patent metadata

Field	Value
Publication number	US-9967590-B2
Application number	US-42031509-A
Country	US
Kind code	B2
Filing date	Apr 8, 2009
Priority date	Apr 10, 2008
Publication date	May 8, 2018
Grant date	May 8, 2018

Abstract

This disclosure describes filtering techniques applied by an encoder and a decoder during the prediction stage of a video encoding and/or decoding process. The filtering techniques may enhance the accuracy of predictive data used during fractional interpolation, and may improve predictive data of integer blocks of pixels. There are several aspects to this disclosure, including a useful twelve-pixel filter support that may be used for interpolation, techniques that use coefficient symmetry and pixel symmetry to reduce the amount of data needed to be sent between an encoder and a decoder to configure the filter support for interpolation, and techniques for filtering data at integer pixel locations in a manner that is similar to sub-pixel interpolation. Other aspects of this disclosure concern techniques for encoding information in the bitstream to convey the type of filter used, and possibly the filter coefficients used. Predictive coding of filter coefficients is also described.

First claim

The invention claimed is:

1. A method comprising:
generating first interpolated predictive data for encoding of video data based on a first interpolation filter corresponding to a fixed interpolation filter;
generating second interpolated predictive data for encoding of the video data based on a second interpolation filter corresponding to an adaptive interpolation filter;
determining a first quality difference between the video data and the first interpolated predictive data based on one or more pixel values of the video data and the first interpolated predictive data;
calculating a first rate-distortion cost associated with the video data in response to the video data being encoded via the first interpolated predictive data;
determining a second quality difference between the video data and the second interpolated predictive data based on one or more pixel values of the video data and the second interpolated predictive data;
calculating a second rate-distortion cost associated with the video data in response to the video data being encoded via the second interpolated predictive data;
selecting between the first interpolated predictive data and the second interpolated predictive data based on a rate distortion analysis;
encoding the video data based on the selection;
in response to the selection comprising the second interpolated predictive data, determining, for the second interpolation filter, a first set of filter coefficients and a second set of filter coefficients; and
wherein a separate syntax element is defined for each of a plurality of sub-pixel locations to indicate whether the first interpolation filter or the second interpolation filter should be used by a decoder for each of the plurality of sub-pixel locations,
wherein the second set of filter coefficients is encoded based on a residual value, the residual value based on a difference between the second set of filter coefficients and a rotated version of the first set of filter coefficients.

2. The method of claim 1 , wherein the first interpolation filter corresponds to an ITU-T H.264 interpolation filter.

3. The method of claim 1 , wherein the first interpolation filter corresponds to an ITU-T H.264 interpolation filter without intermediate rounding of fractional-pixel values that are used to generate other fractional-pixel values.

4. The method of claim 1 , further comprising encoding a syntax element to indicate the selection, wherein the syntax element comprises a flag that defines whether the first interpolation filter or the second interpolation filter should be used by the decoder.

5. The method of claim 1 , wherein a separate syntax element is defined for integer-pixel locations of the video data to indicate whether the first interpolation filter or the second interpolation filter should be used by the decoder for the integer-pixel locations.

6. The method of claim 1 , wherein the first interpolation filter does not apply any filtering.

7. The method of claim 4 , further comprising transmitting the encoded video data and the syntax element to indicate whether the first interpolated predictive data or the second interpolated predictive data was used to generate the encoded video data.

8. The method of claim 7 , further comprising:
transmitting filter coefficients when the syntax element indicates that the second interpolated predictive data was used to generate the encoded video data; and
transmitting no filter coefficients when the syntax element indicates that the first interpolated predictive data was used to generate the encoded video data.

9. The method of claim 1 , wherein the first number of bits is zero.

10. The method of claim 1 , wherein determining the first and second quality differences is based on one of
a mean squared error (MSE) of the one or more pixel values,
a sum of absolute difference (SAD) of the one or more pixel values, or
a sum of squared difference (SSD) of the one or more pixel values.

11. The method of claim 1 , wherein calculating the second rate-distortion cost further comprises multiplying the second number of bits by a Lagrangian multiplier.

12. An apparatus comprising a video coder that:
generates first interpolated predictive data for encoding of video data based on a first interpolation filter corresponding to a fixed interpolation filter;
generates second interpolated predictive data for encoding of the video data based on a second interpolation filter corresponding to an adaptive interpolation filter;
determines a first quality difference between the video data and the first interpolated predictive data based on one or more pixel values of the video data and the first interpolated predictive data;
calculates a first rate-distortion cost associated with the video data in response to the video data being encoded via the first interpolated predictive data;
determines a second quality difference between the video data and the second interpolated predictive data based on one or more pixel values of the video data and the second interpolated predictive data;
calculates a second rate-distortion cost associated with the video data in response to the video data being encoded via the second interpolated predictive data
selects between the first interpolated predictive data and the second interpolated predictive data based on a rate distortion analysis;
encodes the video data based on the selection;
in response to the selection comprising the second interpolated predictive data, determine, for the second interpolation filter, a first set of filter coefficients and a second set of filter coefficients; and
wherein a separate syntax element is defined for each of a plurality of sub-pixel locations to indicate whether the first interpolation filter or the second interpolation filter should be used by a decoder for each of the plurality of sub-pixel locations,
wherein the second set of filter coefficients is encoded based on a residual value, the residual value based on a difference between the second set of filter coefficients and a rotated version of the first set of filter coefficients.

13. The apparatus of claim 12 , wherein the fixed interpolation filter corresponds to an ITU-T H.264 interpolation filter.

14. The apparatus of claim 12 , wherein the fixed interpolation filter corresponds to an ITU-T H.264 interpolation filter without intermediate rounding of fractional pixel values that are used to generate other fractional pixel values.

15. The apparatus of claim 12 , wherein the video coder is configured to encode a syntax element to indicate the selection, wherein the syntax element comprises a flag that defines whether the first interpolation filter or the second interpolation filter should be used by the decoder.

16. The apparatus of claim 12 , wherein a separate syntax element is defined for integer-pixel locations of the video data to indicate whether the first interpolation filter or the second interpolation filter should be used by the decoder for the integer-pixel locations.

17. The apparatus of claim 12 , wherein the first interpolation filter does not apply any filtering.

18. The apparatus of claim 15 , further comprising a transmitter that transmits the encoded video data and the syntax element to indicate whether the first interpolated predictive data or the second interpolated predictive data was used to generate the encoded video data.

19. The apparatus of claim 18 , wherein the transmitter:
transmits filter coefficients when the syntax element indicates that the second interpolated predictive d

Assignees

• Ye Yan
• Karczewicz Marta
• Qualcomm Inc

Inventors

• Ye Yan
• Karczewicz Marta

Classifications

• H04N19/147

  according to rate distortion criteria (rate-distortion as a criterion for motion estimation H04N19/567) · CPC title

• H04N19/117

  Filters, e.g. for pre-processing or post-processing (sub-band filter banks H04N19/635) · CPC title

• H04N19/82

  involving filtering within a prediction loop · CPC title

• H04N19/46

  Embedding additional information in the video signal during the compression process (H04N19/517, H04N19/68, H04N19/70 take precedence) · CPC title

• H04N19/523 Primary

  with sub-pixel accuracy · CPC title