Sub-block based motion candidate selection and signaling

Patent metadata

Field	Value
Publication number	US-12069239-B2
Application number	US-202117321802-A
Country	US
Kind code	B2
Filing date	May 17, 2021
Priority date	Nov 22, 2018
Publication date	Aug 20, 2024
Grant date	Aug 20, 2024

Abstract

Devices, systems and methods for digital video coding, which include sub-block based inter prediction methods, are described. An exemplary method for video processing includes determining, for a conversion between a current block of video and a bitstream representation of the video, a maximum number of candidates in a sub-block based merge candidate list and/or whether to add sub-block based temporal motion vector prediction (SbTMVP) candidates to the sub-block based merge candidate list based on whether temporal motion vector prediction (TMVP) is enabled for use during the conversion or whether a current picture referencing (CPR) coding mode is used for the conversion, and performing, based on the determining, the conversion.

First claim

The invention claimed is:

1. A method of video processing, comprising:
inserting, for a conversion between a current block of a video and a bitstream of the video that uses a sub-block merge candidate list, a non-affine merge padding candidate or an affine merge padding candidate into the sub-block merge candidate list when the sub-block merge candidate list is not filled, wherein whether to insert the non-affine merge padding candidate or the affine merge padding candidate is based on whether an affine usage flag in the bitstream is on or off, and wherein the affine usage flag is used to indicate whether an affine mode is enabled and to determine which one of the non-affine merge padding candidate or the affine merge padding candidate is inserted into the sub-block merge candidate list; and
performing, subsequent to the inserting, the conversion,
wherein the non-affine merge padding candidate is a default non-affine merge padding candidate, or the affine merge padding candidate is a default affine merge padding candidate,
wherein the non-affine merge padding candidate is a zero non-affine candidate, or the affine merge padding candidate is a zero affine candidate, and
wherein, when the affine usage flag is off, the affine merge padding candidate is not inserted into the sub-block merge candidate list or the non-affine merge padding candidate is inserted into the sub-block merge candidate list.

2. The method of claim 1 , further comprising:
determining motion vectors using a rule that determines that the motion vectors are derived from one or more motion vectors of a block covering a corresponding position in a collocated picture.

3. The method of claim 2 , wherein the one or more motion vectors comprise MV0 and MV1 that represent motion vectors in reference list 0 and reference list 1, respectively, and wherein the motion vectors to be derived comprise MV0′ and MV1′ that represent motion vectors in the reference list 0 and the reference list 1, respectively.

4. The method of claim 3 , wherein MV0′ and MV1′ are derived based on MV0 in case that the collocated picture is in the reference list 0.

5. The method of claim 3 , wherein MV0′ and MV1′ are derived based on MV1 in case that the collocated picture is in the reference list 1.

6. The method of claim 1 , wherein the conversion decodes the current block from the bitstream.

7. The method of claim 1 , wherein the conversion encodes the current block into the bitstream.

8. The method of claim 1 , wherein performing the conversion comprises parsing the bitstream based on one or more decoding rules.

9. A video processing apparatus comprising a processor and a non-transitory memory with instructions thereon, wherein the instructions upon execution by the processor, cause the processor to:
insert, for a conversion between a current block of a video and a bitstream of the video that uses a sub-block merge candidate list, a non-affine merge padding candidate or an affine merge padding candidate into the sub-block merge candidate list when the sub-block merge candidate list is not filled, wherein whether to insert the non-affine merge padding candidate or the affine merge padding candidate is based on whether an affine usage flag in the bitstream is on or off, and wherein the affine usage flag is used to indicate whether an affine mode is enabled and to determine which one of the non-affine merge padding candidate or the affine merge padding candidate is inserted into the sub-block merge candidate list; and
perform, subsequent to the inserting, the conversion,
wherein the non-affine merge padding candidate is a default non-affine merge padding candidate, or the affine merge padding candidate is a default affine merge padding candidate,
wherein the non-affine merge padding candidate is a zero non-affine candidate, or the affine merge padding candidate is a zero affine candidate, and
wherein, when the affine usage flag is off, the affine merge padding candidate is not inserted into the sub-block merge candidate list or the non-affine merge padding candidate is inserted into the sub-block merge candidate list.

10. The video processing apparatus of claim 9 , wherein the instructions upon execution by the processor, cause the processor to:
determine motion vectors using a rule that determines that the motion vectors are derived from one or more motion vectors of a block covering a corresponding position in a collocated picture.

11. A non-transitory computer-readable recording medium storing a bitstream of a video which is generated by a method performed by a video processing apparatus, wherein the method comprises:
inserting a non-affine merge padding candidate or an affine merge padding candidate into a sub-block merge candidate list when the sub-block merge candidate list is not filled, wherein whether to insert the non-affine merge padding candidate or the affine merge padding candidate is based on whether an affine usage flag in the bitstream is on or off, and wherein the affine usage flag is used to indicate whether an affine mode is enabled and to determine which one of the non-affine merge padding candidate or the affine merge padding candidate is inserted into the sub-block merge candidate list; and
generating the bitstream based on the inserting,
wherein the non-affine merge padding candidate is a default non-affine merge padding candidate, or the affine merge padding candidate is a default affine merge padding candidate,
wherein the non-affine merge padding candidate is a zero non-affine candidate, or the affine merge padding candidate is a zero affine candidate, and
wherein, when the affine usage flag is off, the affine merge padding candidate is not inserted into the sub-block merge candidate list or the non-affine merge padding candidate is inserted into the sub-block merge candidate list.

12. The method of claim 1 , wherein the non-affine merge padding candidate is not inserted into the sub-block merge candidate list in case that the affine usage flag in the bitstream is on.

13. The method of claim 1 , wherein the affine merge padding candidate is inserted into the sub-block merge candidate list in case that the affine usage flag is on.

Assignees

• Beijing Bytedance Network Tech Co Ltd
• Bytedance Inc

Inventors

• Zhang Kai
• Zhang Li
• Liu Hongbin
• Xu Jizheng
• Wang Yue

Classifications

• H04N19/46

  Embedding additional information in the video signal during the compression process (H04N19/517, H04N19/68, H04N19/70 take precedence) · CPC title

• H04N19/139

  Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability · CPC title

• H04N19/17

  the unit being an image region, e.g. an object · CPC title

• H04N19/52 Primary

  by predictive encoding · CPC title

• H04N19/189

  characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding · CPC title