Method for selecting reference frame, electronic device, and storage medium

Patent metadata

Field	Value
Publication number	US-11490049-B2
Application number	US-202117305872-A
Country	US
Kind code	B2
Filing date	Jul 16, 2021
Priority date	Jul 17, 2020
Publication date	Nov 1, 2022
Grant date	Nov 1, 2022

Abstract

A method for selecting a reference frame, an electronic device, and a storage medium. The method includes: calculating a sum of absolute values of pixel brightness differences of corresponding pixel locations in a current frame and a previous frame in a video; determining frame attribute of the current frame based on the sum of absolute values of pixel brightness differences, the frame attribute including a raw frame and a duplicate frame; counting a number of raw frames in M historical frames previous to the current frame; obtaining a current frame interpolation step size based on the number of raw frames in the M historical frames; obtaining a next frame phase to be interpolated based on a current frame interpolation phase and the current frame interpolation step size; and determining an interpolation reference frame based on the next frame to be interpolated.

First claim

What is claimed is:

1. A method for selecting a reference frame, for frame interpolation processing on frame rate conversion FRC, comprising:
calculating a sum of absolute values of pixel brightness differences of corresponding pixel locations in a current frame and a previous frame in a video;
determining frame attribute of the current frame based on the sum of absolute values of pixel brightness differences, the frame attribute comprising a raw frame and a duplicate frame; counting a number of raw frames in M historical frames previous to the current frame along a timeline, wherein M is an integer greater than 0;
obtaining a current frame interpolation step size based on the number of raw frames in the M historical frames;
obtaining a next frame phase to be interpolated based on a current frame interpolation phase and the current frame interpolation step size; and
determining an interpolation reference frame based on the next frame to be interpolated.

2. The method as claimed in claim 1 , wherein determining the frame attribute of the current frame based on the sum of absolute values of pixel brightness differences comprises:
determining that the current frame is the raw frame in response that the sum of absolute values of pixel brightness differences is greater than a second brightness threshold, the second brightness threshold being an average value of a sum of absolute values of pixel brightness differences of the M historical frames previous to the current frame along the timeline; or
determining that the current frame is the duplicate frame in response that the sum of absolute values of pixel brightness differences is not greater than the second brightness threshold.

3. The method as claimed in claim 1 , wherein obtaining the current frame interpolation step size based on the number of raw frames in the M historical frames comprises:
calculating a ratio of the number of raw frames in the M historical frames and M, as the current frame interpolation step size.

4. The method as claimed in claim 1 , wherein obtaining the next frame phase to be interpolated based on the current frame interpolation phase and the current frame interpolation step size comprises:
calculating a sum of the current frame interpolation phase and the current frame interpolation step size as the next frame phase to be interpolated.

5. The method as claimed in claim 1 , wherein determining the interpolation reference frame based on the next frame to be interpolated comprises:
skipping to a next raw frame of the current reference frame as the interpolation reference frame in response that the next frame to be interpolated is greater than a phase normalization threshold; or
taking the current reference frame as the interpolation reference frame in response that the next frame to be interpolated is less than or equal to the phase normalization threshold.

6. The method as claimed in claim 5 , further comprising:
calculating a phase difference between the next frame to be interpolated and the phase normalization threshold as a new next frame to be interpolated in response that the next frame to be interpolated is greater than the phase normalization threshold.

7. The method as claimed in claim 1 , further comprising:
calculating absolute values of pixel brightness differences of corresponding pixel locations in the current frame and the previous frame, obtaining frame attribute statistical information of N historical frames previous to the current frame along the timeline based on the absolute values of pixel brightness differences, and determining a movie mode of the video based on the frame attribute statistical information, wherein N is an integer greater than 0.

8. The method as claimed in claim 7 , wherein obtaining the frame attribute statistical information of the N historical frames previous to the current frame along the timeline based on the absolute values of pixel brightness differences, and determining the movie mode of the video based on the frame attribute statistical information comprises:
counting a number of pixels whose absolute values of pixel brightness differences are greater than a first brightness threshold in the current frame;
determining that the current frame is the raw frame in response that the number of pixels is greater than a first number threshold, or determining that the current frame is the duplicate frame in response that the number of pixels is not greater than the first number threshold;
counting frame attribute of each historical frame in the N historical frames previous to the current frame along the timeline to obtain a frame attribute sequence of the N historical frames; and
determining the movie mode to which the current frame belongs based on frame attribute sequence of the N historical frames.

9. A non-transitory computer-readable storage medium having stored therein a computer program that, when executed by a processor, causes the processor to perform a method for selecting a reference frame, the method being for frame interpolation processing on frame rate conversion FRC and comprising:
calculating a sum of absolute values of pixel brightness differences of corresponding pixel locations in a current frame and a previous frame in a video;
determining frame attribute of the current frame based on the sum of absolute values of pixel brightness differences, the frame attribute comprising a raw frame and a duplicate frame;
counting a number of raw frames in M historical frames previous to the current frame along a timeline, wherein M is an integer greater than 0;
obtaining a current frame interpolation step size based on the number of raw frames in the M historical frames;
obtaining a next frame phase to be interpolated based on a current frame interpolation phase and the current frame interpolation step size; and
determining an interpolation reference frame based on the next frame to be interpolated.

10. The non-transitory computer-readable storage medium as claimed in claim 9 , wherein determining the frame attribute of the current frame based on the sum of absolute values of pixel brightness differences comprises:
determining that the current frame is the raw frame in response that the sum of absolute values of pixel brightness differences is greater than a second brightness threshold, the second brightness threshold being an average value of a sum of absolute values of pixel brightness differences of the M historical frames previous to the current frame along the timeline; or
determining that the current frame is the duplicate frame in response that the sum of absolute values of pixel brightness differences is not greater than the second brightness threshold.

11. The non-transitory computer-readable storage medium as claimed in claim 9 , wherein obtaining the current frame interpolation step size based on the number of raw frames in the M historical frames comprises:
calculating a ratio of the number of raw frames in the M historical frames and M, as the current frame interpolation step size.

12. The non-transitory computer-readable storage medium as claimed in claim 9 , wherein obtaining the next frame phase to be interpolated based on the current frame interpolation phase and the current frame interpolation step size comprises:
calculating a sum of the current frame interpolation phase and the current frame interpolation step size as the next frame phase to be interpolated.

13. The non-transitory computer-readable storage medium as claimed in claim 9 , wherein determining the interpolation reference frame based on the next frame to be interpolated comprises:
skipping to a next raw frame of the current re

Assignees

• Amlogic Shanghai Co Ltd

Inventors

• Liu Yanling
• Ji Tao
• Wang Chun
• Wang Dongjian
• Chen Xuyun

Classifications

• H04N21/440281 Primary

  by altering the temporal resolution, e.g. by frame skipping (television signal recording using magnetic recording on tape for reproducing at a rate different from the recording rate H04N5/783) · CPC title

• H04N7/0115 Primary

  with details on the detection of a particular field or frame pattern in the incoming video signal, e.g. 3:2 pull-down pattern · CPC title

• H04N7/0127

  by changing the field or frame frequency of the incoming video signal, e.g. frame rate converter · CPC title

• H04N21/8455

  involving pointers to the content, e.g. pointers to the I-frames of the video stream · CPC title

• H04N7/0145 Primary

  the interpolation being class adaptive, i.e. it uses the information of class which is determined for a pixel based upon certain characteristics of the neighbouring pixels · CPC title