Selective template matching in video coding

Patent metadata

Field	Value
Publication number	US-12149729-B2
Application number	US-202418597907-A
Country	US
Kind code	B2
Filing date	Mar 6, 2024
Priority date	Aug 31, 2018
Publication date	Nov 19, 2024
Grant date	Nov 19, 2024

Abstract

In some embodiments, a method selects a first template of pixels that is outside of a first block and a second template of pixels that is outside of the first block. Then, the method selects a first region of pixels that are inside the first block and a second region of pixels that are inside the first block. A first weight is calculated based on the first template of pixels and the first region of pixels and a second weight is calculated based on the second template of pixels and the second region of pixels. The first weight and the second weight are used in a motion prediction search for a second block.

First claim

What is claimed is:

1. A method comprising:
receiving an encoded bitstream for a first block, wherein a second block is signaled in the encoded bitstream as a reference for the first block;
searching for a third block to use as the reference for the first block, wherein searching comprises:
applying a first weight to a first template of pixels for the first block to generate a weighted first template of pixels for the first block and applying the first weight to a first template of pixels for the third block to generate a weighted first template of pixels for the third block;
applying a second weight to a second template of pixels for the first block to generate a weighted second template of pixels for the first block and applying the second weight to a second template of pixels for the third block to generate a weighted second template of pixels for the third block; and
comparing the first weighted template of pixels for the first block to the first weighted template of pixels for the third block and comparing the second weighted template of pixels for the first block to the second weighted template of pixels for the third block to determine whether the third block is more similar to the first block than the second block.

2. The method of claim 1 , wherein searching for the third block to use as the reference for the first block comprises:
applying the first weight to a first template of pixels for a plurality of third blocks to generate a weighted first template of pixels for respective third blocks in the plurality of third blocks;
applying the second weight to a second template of pixels for the plurality of third blocks to generate a weighted second template of pixels for respective third blocks in the plurality of third blocks;
comparing the first weighted template of pixels for respective third blocks in the plurality of third blocks to the first weighted template of pixels for the first block and comparing the second weighted template of pixels for respective third blocks in the plurality of third blocks to the second weighted template of pixels for the first block; and
selecting one of the plurality of third blocks based on the comparing.

3. The method of claim 2 , wherein:
the one of the plurality of third blocks is determined to be more similar to the first block than the second block.

4. The method of claim 1 , wherein:
the first template of pixels for the first block includes pixels to a left of the first block,
the second template of pixels for the first block includes pixels to a top of the first block,
the first template of pixels for the third block includes pixels to the left of the third block, and
the second template of pixels for the third block includes pixels to the top of the third block.

5. The method of claim 1 , wherein:
the first weight is determined by:
selecting a first region of pixels that is inside the first block, wherein the first template of pixels for the first block and first region of pixels are similar sizes of a height and a width;
calculating the first weight based on the first template of pixels for the first block and the first region of pixels; and
the second weight is determined by:
selecting a second region of pixels that is inside the first block, wherein the second template of pixels for the first block and the second region of pixels are similar sizes of a height and a width
calculating the second weight based on the second template of pixels for the first block and the second region of pixels.

6. The method of claim 5 , wherein:
the first template of pixels for the first block includes pixels that neighbor the pixels in the first region of pixels, and
the second template of pixels for the first block includes pixels that neighbor the pixels in the second region of pixels.

7. The method of claim 5 , wherein:
calculating the first weight comprises calculating a first difference between pixels in the first template of pixels for the first block and the first region of pixels; and
calculating the second weight comprises calculating a second difference between pixels in the second template of pixels for the first block and the second region of pixels.

8. The method of claim 7 , wherein:
calculating the first weight and the second weight comprises calculating a weighted distribution of the first difference and the second difference.

9. The method of claim 1 , wherein the first weight and the second weight are included in the encoded bitstream.

10. The method of claim 1 , wherein the first weight and the second weight are calculated and not included in the encoded bitstream.

11. The method of claim 1 , wherein:
the first weight and the second weight are used to select the third block to use as the reference for decoding the first block.

12. The method of claim 1 , wherein:
the first block is in a first frame of a video, and
the second block is in a second frame of the video.

13. The method of claim 1 , further comprising:
searching a search area for the third block to use as the reference for the first block being decoded.

14. The method of claim 1 , wherein:
the first block, the first template of pixels for the first block, and the second template of pixels for the first block are in a same frame,
the third block, the first template of pixels for the third block, and the second template of pixels for the third block are in another frame.

15. The method of claim 1 , wherein:
the first weight and the second weight are different values,
values of the first weight and the second weight are determined based on:
a first difference between pixels in the first template of pixels for the first block and a first region of pixels inside the first block; and
a second difference between pixels in the second template of pixels for the first block and a second region of pixels inside the first block.

16. The method of claim 15 , wherein:
the first template of pixels for the first block and first region of pixels are similar sizes of a height and a width, and
the second template of pixels for the first block and second region of pixels are similar sizes of a height and a width.

17. The method of claim 1 , wherein:
the first block is being decoded, and
the third block is searched for as a new reference block to use to decode the first block.

18. The method of claim 1 , wherein:
the first template of pixels for the first block and first template of pixels for the third block are similar sizes of a height and a width, and
the second template of pixels for the first block and second template of pixels for the third block are similar sizes of a height and a width.

19. A non-transitory computer-readable storage medium containing instructions, that when executed, control a computer system to be operable for:
receiving an encoded bitstream for a first block, wherein a second block is signaled in the encoded bitstream as a reference for the first block;
searching for a third block to use as the reference for the first block, wherein searching comprises:
applying a first weight to a first template of pixels for the first block to generate a weighted first template of pixels for the first block and applying the first weight to a first template of pixels for the third block to generate a weighted first template of pixels for the third block;
applying a second weight to a second template of pixels for the first block to generate a weighted second template of pixels for the first block and applying the second weight to a second template of pixels

Assignees

• Hulu Llc

Inventors

• Zhang Wenhao
• Fu Deliang
• Liu Chen
• Liu Xiaobo

Classifications

• H04N19/105 Primary

  Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction · CPC title

• H04N19/182

  the unit being a pixel · CPC title

• H04N19/176

  the region being a block, e.g. a macroblock · CPC title

• H04N19/157

  Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter · CPC title

• H04N19/57

  Motion estimation characterised by a search window with variable size or shape · CPC title