Interactions between neural network-based intra prediction modes and regular intra prediction modes

Patent metadata

Field	Value
Publication number	US-12587640-B2
Application number	US-202218288763-A
Country	US
Kind code	B2
Filing date	Apr 28, 2022
Priority date	Apr 28, 2021
Publication date	Mar 24, 2026
Grant date	Mar 24, 2026

Abstract

Systems, methods, and instrumentalities are disclosed for performing intra prediction of a luminance coding block and/or a chrominance coding block and related signaling when neural network-based intra prediction is enabled. For a current coding block, information representative of a neural network-based prediction mode and a block context may be obtained. A neural network-based predictor may be generated for the current coding block based on the block context and the neural network-based prediction mode. A non-neural network-based intra prediction mode that corresponds to the generated neural network-based predictor may be determined. A prediction mode for a neighboring block may be determined based on the non-neural network-based intra prediction mode.

First claim

The invention claimed is:

1 . A decoding device comprising:
a processor configured to:
obtain, for a current coding block, information representative of a neural network-based prediction mode and a block context;
generate a neural network-based predictor for the current coding block based on the block context and the neural network-based prediction mode;
determine, based on a plurality of pixels that neighbor the current coding block, a non-neural network-based intra prediction mode that corresponds to the generated neural network-based predictor;
determine an intra prediction index associated with the determined non-neural network-based intra prediction mode;
add the intra prediction index to a most probable mode (MPM) list of a neighboring block of the current coding block;
determine a prediction mode for the neighboring block based on the MPM list; and
decode the neighboring block based on the prediction mode.

2 . The device of claim 1 , wherein the current coding block comprises a current luminance block and a current chrominance block, and the generated neural network-based predictor comprises a luminance predictor for the current luminance block, and a chrominance predictor for the current chrominance block.

3 . The device of claim 1 , wherein determining the non-neural network-based intra prediction mode further comprises:
obtaining a plurality of representation probabilities associated with a plurality of non-neural network based intra prediction modes based on the block context and the neural network-based prediction mode; and selecting, from the plurality of non-neural network based intra prediction modes, the determined non-neural network-based intra prediction mode based on the plurality of representation probabilities.

4 . The device of claim 3 , wherein a first representation probability associated with a first non-neural network based intra prediction mode is configured to indicate a probability that using the first non-neural network based intra prediction mode on the current block yields a representation of the generated neural network-based predictor, wherein a higher probability indicates that the representation is more similar to the generated neural network-based predictor when compared to a lower probability, wherein a non-neural network based intra prediction mode associated with a highest representation probability is selected.

5 . The device of claim 1 , wherein the processor is further configured to:
obtain an intra prediction index associated with the determined non-neural network-based intra prediction mode that corresponds to the generated neural network-based predictor; and add the intra prediction index associated with the determined non-neural network-based intra prediction mode to a most probable mode (MPM) list of the neighboring block.

6 . The device of claim 1 , wherein the processor is further configured to:
apply at least one of a primary inverse transform or a secondary inverse transform based on the non-neural network-based intra prediction mode.

7 . The device of claim 1 , wherein the processor is further configured to:
derive a most probable mode (MPM) for the neighboring block based on the determined non-neural network-based intra prediction mode.

8 . A method of video decoding, comprising:
obtaining, for a current coding block, information representative of a neural network-based prediction mode and a block context; generating a neural network-based predictor for the current coding block based on the block context and the neural network-based prediction mode; determining, based on a plurality of pixels that neighbor the current coding block, a non-neural network-based intra prediction mode that corresponds to the generated neural network-based predictor; determining an intra prediction index associated with the determined non-neural network-based intra prediction mode; adding the intra prediction index to a most probable mode (MPM) list of a neighboring block of the current coding block; determining a prediction mode for the neighboring block based on the MPM list; and decoding the neighboring block based on the prediction mode.

9 . The method of claim 8 , wherein the current coding block comprises a current luminance block and a current chrominance block, and the generated neural network-based predictor comprises a luminance predictor for the current luminance block, and a chrominance predictor for the current chrominance block.

10 . The method of claim 8 , wherein determining the non-neural network-based intra prediction mode further comprises:
obtaining a plurality of representation probabilities associated with a plurality of non-neural network based intra prediction modes based on the block context and the neural network-based prediction mode; and selecting, from the plurality of non-neural network based intra prediction modes, the determined non-neural network-based intra prediction mode based on the plurality of representation probabilities.

11 . The method of claim 10 , wherein a first representation probability associated with a first non-neural network based intra prediction mode is configured to indicate a probability that using the first non-neural network based intra prediction mode on the current block yields a representation of the generated neural network-based predictor, wherein a higher probability indicates that the representation is more similar to the generated neural network-based predictor when compared to a lower probability, and wherein a non-neural network based intra prediction mode associated with a highest representation probability is selected.

12 . The method of claim 8 , further comprising:
obtaining an intra prediction index associated with the determined non-neural network-based intra prediction mode that corresponds to the generated neural network-based predictor; and adding the intra prediction index associated with the determined non-neural network-based intra prediction mode to a most probable mode (MPM) list of the neighboring block.

13 . The method of claim 8 , further comprising:
applying at least one of a primary inverse transform or a secondary inverse transform based on the non-neural network-based intra prediction mode.

14 . The method of claim 8 , further comprising:
deriving a most probable mode (MPM) for the neighboring block based on the determined non-neural network-based intra prediction mode.

15 . A method of video encoding, comprising:
obtaining, for a current coding block, information representative of a neural network-based prediction mode and a block context; generating a neural network-based predictor for the current coding block based on the block context and the neural network-based prediction mode; determining, based on a plurality of pixels that neighbor the current coding block, a non-neural network-based intra prediction mode that corresponds to the generated neural network-based predictor; determining an intra prediction index associated with the determined non-neural network-based intra prediction mode; adding the intra prediction index to a most probable mode (MPM) list of a neighboring block of the current coding block; determining a prediction mode for the neighboring block based on the MPM list; and encoding the neighboring block based on the prediction mode.

16 . The method of claim 15 , wherein the current coding block comprises a current luminance block and a current chrominance block, and the generated neural network-based predictor comprises a luminance predictor for the current luminance block, and a

Assignees

• Interdigital Ce Patent Holdings Sas

Inventors

• Dumas Thierry
• GALPIN Franck
• Bordes Philippe
• Le Léannec Fabrice
• Robert Antoine

Classifications

• H04N19/593

  involving spatial prediction techniques · CPC title

• H04N19/196

  being specially adapted for the computation of encoding parameters, e.g. by averaging previously computed encoding parameters (processing of motion vectors H04N19/513) · CPC title

• H04N19/176

  the region being a block, e.g. a macroblock · CPC title

• H04N19/159

  Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction · CPC title

• H04N19/463

  by compressing encoding parameters before transmission · CPC title