Frame packing and unpacking between frames of chroma sampling formats with different chroma resolutions

Patent metadata

Field	Value
Publication number	US-9979960-B2
Application number	US-201314027013-A
Country	US
Kind code	B2
Filing date	Sep 13, 2013
Priority date	Oct 1, 2012
Publication date	May 22, 2018
Grant date	May 22, 2018

Abstract

Video frames of a higher-resolution chroma sampling format such as YUV 4:4:4 are packed into video frames of a lower-resolution chroma sampling format such as YUV 4:2:0 for purposes of video encoding. For example, sample values for a frame in YUV 4:4:4 format are packed into two frames in YUV 4:2:0 format. After decoding, the video frames of the lower-resolution chroma sampling format can be unpacked to reconstruct the video frames of the higher-resolution chroma sampling format. In this way, available encoders and decoders operating at the lower-resolution chroma sampling format can be used, while still retaining higher resolution chroma information. In example implementations, frames in YUV 4:4:4 format are packed into frames in YUV 4:2:0 format such that geometric correspondence is maintained between Y, U and V components for the frames in YUV 4:2:0 format.

First claim

We claim:

1. A method comprising:
receiving one or more frames of a higher-resolution chroma sampling format, the one or more frames of the higher-resolution chroma sampling format including sample values of chroma components and sample values of luma components; and
packing the one or more frames of the higher-resolution chroma sampling format into one or more frames of a lower-resolution chroma sampling format, wherein the lower-resolution chroma sampling format has a lower chroma resolution than the higher-resolution chroma sampling format, and wherein the packing includes assigning some of the sample values of the chroma components of the one or more frames in the higher-resolution chroma sampling format to sample values of a luma component of one of the one or more frames of the lower-resolution chroma sampling format.

2. The method of claim 1 further comprising, after the packing:
encoding the one or more frames of the lower-resolution chroma sampling format.

3. The method of claim 2 wherein the packing maintains geometric correspondence between adjacent sample values of the chroma components of the one or more frames of the higher-resolution chroma sampling format as adjacent samples and/or collocated portions of luma and chroma components of the one or more frames of the lower-resolution chroma sampling format, and wherein the encoding exploits the geometric correspondence.

4. The method of claim 3 wherein the encoding exploits the geometric correspondence (a) in encoding operations that include one or more of derivation of motion vectors and derivation of prediction modes, and/or (b) to guide encoding decisions that include one or more of motion estimation, selection of quantization parameters and selection of prediction modes.

5. The method of claim 1 wherein first parts of the one or more frames of the lower-resolution chroma sampling format represent a lower chroma resolution version of the one or more frames of the higher-resolution chroma sampling format, and wherein second parts of the one or more frames of the lower-resolution chroma sampling format, the second parts including the luma component of one of the one or more frames of the lower-resolution chroma sampling format, represent remaining chroma information from the one or more frames of the higher-resolution chroma sampling format.

6. The method of claim 1 further comprising:
signaling metadata that indicates that first parts of the one or more frames of the lower-resolution chroma sampling format represent a lower chroma resolution version of the one or more frames of the higher-resolution chroma sampling format, and indicates that second parts of the one or more frames of the lower-resolution chroma sampling format, the second parts including the luma component of one of the one or more frames of the lower-resolution chroma sampling format, represent remaining chroma information from the one or more frames of the higher-resolution chroma sampling format.

7. The method of claim 1 wherein the packing is consistent with plural design constraints, the plural design constraints including:
maintaining geometric correspondence among the sample values of the chroma components of the one or more frames of the higher-resolution chroma sampling format after the packing; and
embedding a lower chroma resolution version of the one or more frames of the higher-resolution chroma sampling format as part of the one or more frames of the lower-resolution chroma sampling format.

8. The method of claim 1 wherein the luma component of one of the one or more frames of the lower-resolution chroma sampling format is a luma component of an auxiliary view among the one or more frames of the lower-resolution chroma sampling format, and wherein the packing includes, for the chroma components of the one or more frames of the higher-resolution chroma sampling format:
assigning sample values corresponding to every second row or column of the chroma components of the one or more frames of the higher-resolution chroma sampling format to sample values of the luma component of the auxiliary view among the one or more frames of the lower-resolution chroma sampling format;
assigning every second sample value corresponding to other rows or columns of the chroma components of the one or more frames of the higher-resolution chroma sampling format to sample values of chroma components of a main view among the one or more frames of the lower-resolution chroma sampling format; and
assigning other sample values corresponding to the other rows or columns of the chroma components of the one or more frames of the higher-resolution chroma sampling format to sample values of chroma components of the auxiliary view among the one or more frames of the lower-resolution chroma sampling format.

9. The method of claim 1 wherein the packing includes:
anti-alias filtering the sample values of the chroma components of the one or more frames of the higher-resolution chroma sampling format; and
assigning the filtered sample values to sample values of chroma components of a main view among the one or more frames of the lower-resolution chroma sampling format.

10. The method of claim 1 wherein the sample values of the one or more frames of the higher-resolution chroma sampling format have a first bit depth, and wherein sample values of the one or more frames of the lower-resolution chroma sampling format have a second bit depth higher than the first bit depth.

11. The method of claim 1 further comprising:
signaling an indication of whether the sample values of the chroma components of the one or more frames of the higher-resolution chroma sampling format have been filtered as part of the packing.

12. A computing device comprising:
one or more processing units;
volatile memory; and
non-volatile memory and/or storage, the non-volatile memory and/or storage having stored therein computer-executable instructions for causing the computing device, when programmed thereby, to perform operations to reconstruct frames of a higher-resolution chroma sampling format, the operations comprising:
receiving one or more frames of a lower-resolution chroma sampling format, wherein the lower-resolution chroma sampling format has a lower chroma resolution than the higher-resolution chroma sampling format; and
unpacking the one or more frames of the lower-resolution chroma sampling format into one or more frames of the higher-resolution chroma sampling format, the one or more frames of the higher-resolution chroma sampling format including sample values of chroma components and sample values of luma components, wherein the unpacking includes assigning sample values of a luma component of one of the one or more frames of the lower-resolution chroma sampling format to some of the sample values of the chroma components of the one or more frames in the higher-resolution chroma sampling format.

13. The computing device of claim 12 wherein the operations further comprise, before the receiving and the unpacking:
decoding the one or more frames of the lower-resolution chroma sampling format.

14. The computing device of claim 12 wherein first parts of the one or more frames of the lower-resolution chroma sampling format represent a lower chroma resolution version of the one or more frames of the higher-resolution chroma sampling format, and wherein second parts of the one or more frames of the lower-resolution chroma sampling format, the second parts including the luma component of one of the one or more frames of the lower-resolution chroma sampling format, represent remaining chroma information from the one or more frames of

Assignees

• Microsoft Corp
• Microsoft Technology Licensing Llc

Inventors

• Zhu Lihua
• Sankuratri Sridhar
• Kumar B Anil
• Wu Yongjun
• Kanumuri Sandeep
• Sadhwani Shyam
• Sullivan Gary J

Classifications

• H04N19/88

  involving rearrangement of data among different coding units, e.g. shuffling, interleaving, scrambling or permutation of pixel data or permutation of transform coefficient data among different blocks · CPC title

• H04N19/70

  characterised by syntax aspects related to video coding, e.g. related to compression standards · CPC title

• H04N21/234327

  by decomposing into layers, e.g. base layer and one or more enhancement layers · CPC title

• H04N19/80

  Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation (H04N19/635, H04N19/86 take precedence) · CPC title

• H04N19/186 Primary

  the unit being a colour or a chrominance component · CPC title