Method for video processing, electronic device, and storage medium

Patent metadata

Field	Value
Publication number	US-12531989-B2
Application number	US-202418444824-A
Country	US
Kind code	B2
Filing date	Feb 19, 2024
Priority date	Aug 20, 2021
Publication date	Jan 20, 2026
Grant date	Jan 20, 2026

Abstract

A video processing method, an electronic device and a storage medium are provided. The method includes: a target frame is determined in a video to be processed; at least two first energy images corresponding to the target frame are determined based on at least two preset macro block sizes respectively, each first energy image represents alternating current energy of first macro block(s) corresponding to a respective macro block size and the first macro block(s) are obtained by segmenting the target frame based on the respective macro block size; a first energy map corresponding to the target frame is determined based on the first energy images, the first energy map represents energy distribution in the target frame; and an adaptive quantization parameter corresponding to the target frame is determined based on the first energy map and the target frame is encoded using the adaptive quantization parameter.

First claim

The invention claimed is:

1 . A method for video processing, comprising:
determining a target frame in a video to be processed; respectively determining at least two first energy images corresponding to the target frame based on at least two preset macro block sizes, each of the at least two first energy images representing alternating current energy of at least one first macro block corresponding to a respective one of the at least two preset macro block sizes, wherein the at least one first macro block is obtained by segmenting the target frame based on the respective one of the at least two preset macro block sizes; determining a first energy map corresponding to the target frame based on the at least two first energy images, wherein the first energy map represents energy distribution in the target frame; determining, based on the first energy map, an adaptive quantization parameter corresponding to the target frame and encoding the target frame by using the adaptive quantization parameter; and respectively determining at least two second energy images corresponding to the target frame based on the at least two preset macro block sizes, each of the at least two second energy images representing alternating energy of at least one second macro block corresponding to a respective one of the at least two preset macro block sizes, wherein the at least one second macro block is obtained by shifting and segmenting the target frame based on the respective one of the at least two preset macro block sizes; wherein determining the first energy map corresponding to the target frame based on the at least two first energy images comprises:
determining a second mean value of the at least two first energy images and the at least two second energy images at a same pixel position; and
determining the first energy map based on second mean values corresponding to all pixel positions.

2 . The method of claim 1 , wherein respectively determining the at least two first energy images corresponding to the target frame based on the at least two preset macro block sizes comprises:
for each of the at least two preset macro block sizes, segmenting the target frame based on the preset macro block size to obtain a plurality of first macro blocks; determining alternating current energy of each of the plurality of first macro blocks; and determining a corresponding first energy image based on the plurality of first macro blocks obtained by segmenting the target frame based on a same macro block size, pixel values in the corresponding first energy image being respective alternating current energy of the plurality of first macro blocks.

3 . The method of claim 2 , wherein determining the alternating current energy of each of the plurality of first macro blocks comprises:
determining the alternating current energy corresponding to the first macro block based on variance of all pixel values and a number of pixels in the first macro block.

4 . The method of claim 1 , wherein respectively determining the at least two second energy images corresponding to the target frame based on the at least two preset macro block sizes comprises:
performing, based on each of the at least two preset macro block sizes, shifting processing on the target frame, to obtain a plurality of shifted frames; and determining respective second energy images of the plurality of shifted frames respectively corresponding to the at least two preset macro block sizes.

5 . The method of claim 4 , wherein performing, based on each of the at least two preset macro block sizes, the shifting processing on the target frame to obtain the plurality of shifted frames comprises:
scaling, based on a preset scaling ratio, the at least two preset macro block sizes to obtain corresponding shifting sizes; and performing, based on each of the corresponding shifting sizes, shifting processing on the target frame to obtain the plurality of shifted frames.

6 . The method of claim 5 , wherein performing, based on each of the corresponding shifting sizes, the shifting processing on the target frame to obtain the plurality of shifted frames comprises:
for each of the corresponding shifting sizes, adding a quantity of pixel rows and pixel columns at two adjacent sides of the target frame respectively by copying the two adjacent sides, the quantity of the added pixel rows and pixel columns corresponding to the shifting size, and copying a pixel at an intersection position of the two adjacent sides to a blank area between the added pixel rows and pixel columns, to obtain a corresponding candidate shifted frame; and cropping a quantity of pixel rows and pixel columns at two sides not being copied of the candidate shifted frame to obtain a corresponding shifted frame, the quantity of the cropped pixel rows and pixel columns corresponding to the shifting size.

7 . The method of claim 4 , wherein determining the respective second energy images of the plurality of shifted frames respectively corresponding to the at least two preset macro block sizes comprises:
segmenting, based on each of the at least two preset macro block sizes, a shifted frame corresponding to the preset macro block size to obtain a plurality of second macro blocks; determining alternating current energy of each of the plurality of second macro blocks; and determining a corresponding second energy image based on the plurality of second macro blocks corresponding to a same shifted frame, pixel values in the corresponding second energy image being respective alternating current energy of the plurality of second macro blocks.

8 . The method of claim 1 , wherein determining, based on the first energy map, the adaptive quantization parameter corresponding to the target frame and encoding the target frame by using the adaptive quantization parameter comprises:
determining a second energy map corresponding to the first energy map by means of average pooling; and determining, based on the second energy map, the adaptive quantization parameter corresponding to the target frame and encoding the target frame by using the adaptive quantization parameter.

9 . The method of claim 8 , wherein determining the second energy map corresponding to the first energy map by means of the average pooling comprises:
determining a target macro block size; and performing, by taking the target macro block size as a window and a stride, the average pooling on the first energy map to obtain the second energy map.

10 . The method of claim 8 , wherein determining, based on the second energy map, the adaptive quantization parameter corresponding to the target frame and encoding the target frame by using the adaptive quantization parameter comprises:
determining a histogram mapping table corresponding to the second energy map; mapping the second energy map based on the histogram mapping table to obtain the adaptive quantization parameter corresponding to the target frame; and inputting the adaptive quantization parameter and the target frame into a video encoder for performing, based on the corresponding adaptive quantization parameter, video encoding on the target frame.

11 . The method of claim 1 , wherein the at least two preset macro block sizes are set based on a fixed ratio.

12 . The method of claim 1 , wherein determining the target frame in the video to be processed comprises:
determining the target frame from the video to be processed in sequence in a timeline order.

13 . An electronic device, comprising:
a processor; and a memory for storing instructions that, when executed by the processor, cause the processor to: determine a target frame i

Assignees

• Shanghai Sensetime Intelligent Tech Co Ltd

Inventors

• Xu Tongda
• Gao Chenjian
• Wang Yan
• Yuan Tao
• Qin Hongwei

Classifications

• H04N19/176

  the region being a block, e.g. a macroblock · CPC title

• H04N19/172

  the region being a picture, frame or field · CPC title

• H04N19/14

  Coding unit complexity, e.g. amount of activity or edge presence estimation (H04N19/146 takes precedence) · CPC title

• H04N19/119 Primary

  Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks · CPC title

• H04N19/124 Primary

  Quantisation · CPC title