Parallel processing of video frames during video encoding

Patent metadata

Field	Value
Publication number	US-11871018-B2
Application number	US-202117451964-A
Country	US
Kind code	B2
Filing date	Oct 22, 2021
Priority date	Sep 2, 2021
Publication date	Jan 9, 2024
Grant date	Jan 9, 2024

Abstract

Disclosed are techniques for compressing data of an image. Intermediate pixels may be determined. Each location of the image may be associated with a block of a plurality of blocks of a first size and a block of a plurality of blocks of a second size. For each block of the first size and of the second size, a first cost for a first mode and a second cost for a second mode may be determined in parallel using the intermediate pixels. A final mode and a final block size may be selected for each location of the image using the first cost and the second cost for each of a respective block of the first size and a respective block of the second size associated with a corresponding location. Final pixels may be determined, and a representation of the image may be obtained based on the final pixels.

First claim

What is claimed is:

1. A method for compressing data of an image, the method comprising:
determining intermediate pixels;
associating each location of the image with a block of a plurality of blocks of a first size and a block a plurality of blocks of a second size;
determining, in parallel, using the intermediate pixels, for each block of the first size and for each block of the second size, a first cost for a first mode and a second cost for a second mode;
selecting, for each location of the image, i) a final mode from at least the first mode and the second mode and ii) a final block size from at least the first size and the second size, using the first cost and the second cost for each of a respective block of the first size and a respective block of the second size associated with a corresponding location;
determining final pixels based on the selected final mode and the selected final block size for each location of the image; and
obtaining, based on the final pixels, a representation of the image.

2. The method of claim 1 , wherein determining the intermediate pixels comprises:
partitioning the image into a plurality of blocks of a third size;
processing, sequentially, each block of the third size, to determine a third cost for the first mode and a fourth cost for the second mode, wherein determining the third cost and the fourth cost uses a set of intermediate pixels obtained for a previously processed block;
selecting, for each block of the third size, an intermediate mode based on a comparison of the third cost for the first mode and the fourth cost for the second mode; and
obtaining, using the selected intermediate mode, a set of intermediate pixels for a corresponding block of the third size.

3. The method of claim 1 , wherein determining the intermediate pixels comprises:
partitioning the image into a plurality of blocks of a third size; and
selecting, for each of the blocks of the third size, a corresponding block of source pixels from the image.

4. The method of claim 1 , wherein determining the first cost for the first mode and the second cost for the second mode comprises, for each of the first mode and the second mode:
generating a temporary block prediction based on a respective mode and a set of intermediate pixels; and
calculating a cost of the respective mode based on a comparison of the temporary block prediction and a corresponding block of source pixels of the image.

5. The method of claim 1 , wherein determining the final pixels comprises, for each location of the image:
generating an initial predicted block based on the selected final mode, the selected final block size, and a set of previously determined final pixels associated with one or more neighbor locations; and
deriving a final pixel block by transforming the initial predicted block using a compression algorithm.

6. The method of claim 5 , wherein the compression algorithm is based on one of AV1, HEVC, VVC, or VP9 encoding standards.

7. The method of claim 1 , wherein the first mode is one of a plurality of intra-prediction modes and the second mode is an inter-prediction mode.

8. The method of claim 7 , wherein the inter-prediction mode approximates:
the respective block of the first size with a first reference block of the first size in a first reference image; and
the respective block of the second size with a second reference block of the second size in a second reference image.

9. A system comprising:
a memory; and
a processor, coupled to the memory, to perform operations comprising:
determining intermediate pixels for an image;
associating each location of the image with a block of a plurality of blocks of a first size and a block a plurality of blocks of a second size;
determining, in parallel, using the intermediate pixels, for each block of the first size and for each block of the second size, a first cost for a first mode and a second cost for a second mode;
selecting, for each location of the image, i) a final mode from at least the first mode and the second mode and ii) a final block size from at least the first size and the second size, using the first cost and the second cost for each of a respective block of the first size and a respective block of the second size associated with a corresponding location;
determining final pixels based on the selected final mode and the selected final block size for each location of the image; and
obtaining, based on the final pixels, a representation of the image.

10. The system of claim 9 , wherein determining the intermediate pixels comprises:
partitioning the image into a plurality of blocks of a third size;
processing, sequentially, each block of the third size, to determine a third cost for the first mode and a fourth cost for the second mode, wherein determining the third cost and the fourth cost uses a set of intermediate pixels obtained for a previously processed block;
selecting, for each block of the third size, an intermediate mode based on a comparison of the third cost for the first mode and the fourth cost for the second mode; and
obtaining, using the selected intermediate mode, a set of intermediate pixels for a corresponding block of the third size.

11. The system of claim 9 , wherein determining the first cost for the first mode and the second cost for the second mode comprises, for each of the first mode and the second mode:
generating a temporary block prediction based on a respective mode and a set of intermediate pixels; and
calculating a cost of the respective mode based on a comparison of the temporary block prediction and a corresponding block of source pixels of the image.

12. The system of claim 9 , wherein determining the final pixels comprises, for each location of the image:
generating an initial predicted block based on the selected final mode, the selected final block size, and a set of previously determined final pixels associated with one or more neighbor locations; and
deriving a final pixel block by transforming the initial predicted block using a compression algorithm.

13. The system of claim 12 , wherein the compression algorithm is based on one of AV1, HEVC, VVC, or VP9 encoding standards.

14. The system of claim 9 , wherein the first mode is one of a plurality of intra-prediction modes and the second mode is an inter-prediction mode.

15. A non-transitory computer-readable medium comprising instructions, which when executed by a processor cause the processor to perform operations comprising:
determining intermediate pixels for an image;
associating each location of the image with a block of a plurality of blocks of a first size and a block a plurality of blocks of a second size;
determining, in parallel, using the intermediate pixels, for each block of the first size and for each block of the second size, a first cost for a first mode and a second cost for a second mode;
selecting, for each location of the image, i) a final mode from at least the first mode and the second mode and ii) a final block size from at least the first size and the second size, using the first cost and the second cost for each of a respective block of the first size and a respective block of the second size associated with a corresponding location;
determining final pixels based on the selected final mode and the selected final block size for each location of the image; and
obtaining, based on the final pixels, a representation of the image.

16. The non-transitory computer-readable medium of claim 15 , wherein determining the intermediate pixels comprises:
partitioning the image into a plural

Assignees

• Nvidia Corp

Inventors

• Chen Jianjun
• Feng Wei
• He Xi
• Han Sangeun
• Srinivasan Ranga Ramanujam

Classifications

• H04N19/436 Primary

  using parallelised computational arrangements · CPC title

• H04N19/105

  Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction · CPC title

• H04N19/14

  Coding unit complexity, e.g. amount of activity or edge presence estimation (H04N19/146 takes precedence) · CPC title

• H04N19/176 Primary

  the region being a block, e.g. a macroblock · CPC title

• H04N19/103

  Selection of coding mode or of prediction mode · CPC title