System and method for video encoding using constructed reference frame

Patent metadata

Field	Value
Publication number	US-9374596-B2
Application number	US-201213658396-A
Country	US
Kind code	B2
Filing date	Oct 23, 2012
Priority date	Sep 11, 2008
Publication date	Jun 21, 2016
Grant date	Jun 21, 2016

Abstract

Disclosed herein is a method for digital video encoding prediction comprising creating a constructed reference frame using an encoder and compressing a series of source video frames using the constructed reference frame to obtain a bitstream including a compressed digital video signal for a subsequent decoding process. The constructed reference frame is omitted from the series of digital video frames during the subsequent viewing process.

First claim

What is claimed is:

1. A method of encoding a series of source video frames to be transmitted to a decoder to produce a series of decoded video frames, the method comprising:
generating a series of compressed frames by compressing a series of source video frames, such that each source video frame from the series of source video frames is represented by a respective compressed frame from the series of compressed frames, and wherein at least one compressed frame from the series of compressed frames is compressed using a constructed reference frame, wherein compressing the series of source video frames includes:
generating the constructed reference frame, wherein the constructed reference frame is a frame in addition to the series of compressed frames, such that a cardinality of the constructed reference frame and the series of compressed frames is greater than a cardinality of the series of compressed frames, wherein generating the constructed reference frame includes generating the constructed reference frame based on a source video frame from the series of source video frames at a first quality level,
generating a compressed constructed reference frame by compressing the constructed reference frame,
generating an encoded frame representing the source video frame by encoding the source video frame at a second quality level different from the first quality level using the constructed reference frame as a reference frame;

including the compressed constructed reference frame and the series of compressed frames in an output bitstream for transmission to a decoder; and
including the encoded frame in the output bitstream after the compressed constructed reference frame.

2. The method according to claim 1 , wherein generating the constructed reference frame comprises:
using temporal filtering to remove video noise from a plurality of source video frames from the series of source video frames;
calculating an error score for each macroblock of the plurality of source video frames; and
including a macroblock of each source video frame from the plurality of source video frames having the lowest error score in the constructed reference frame.

3. The method according to claim 1 , wherein generating the constructed reference frame comprises:
generating the constructed reference frame using a background of at least one source video frame from the series of source video frames.

4. The method according to claim 3 , wherein generating the constructed reference frame using the background comprises:
calculating a motion vector of each of a plurality of macroblocks over a plurality of source video frames from the series of source video frames, wherein each of the plurality of macroblocks includes a plurality of pixels;
including a macroblock having a motion vector below a predetermined limit in the constructed reference frame.

5. The method according to claim 1 , wherein each source video frame from the series of source video frames has a predetermined size; and wherein the constructed reference frame has a size larger than the predetermined size, and wherein generating the constructed reference frame includes using image super resolution.

6. The method according to claim 5 , wherein compressing the series of source video frames comprises generating a compressed frame from the series of compressed frames by compressing a source video frame from the series of source video frames using the constructed reference frame such that the compressed frame includes high quality boarder regions.

7. The method according to claim 1 , further comprising:
generating a plurality of scores for the series of source video frames by scoring each macroblock within a plurality of source video frames from the series of source video frames based on a scoring criterion, wherein each macroblock includes a plurality of pixels;
computing an overall score for each of the plurality of source video frames based on the plurality of scores; and
selecting one of the plurality of source video frames for creating the constructed reference frame based on the overall scores.

8. The method according to claim 7 , wherein the scoring criterion comprises at least one of a ratio comparing intra-prediction to inter-prediction, a motion vector, a decay in a prediction quality, or a conclusion of a zoom in or a zoom out.

9. A method of decoding an encoded bitstream, the encoded bitstream including a series of encoded video frames and an encoded constructed reference frame, wherein each encoded video frame from the series of encoded video frames represents a respective source video frame from a series of source video frames, the method comprising:
generating a constructed reference frame by decompressing the encoded constructed reference frame, wherein the constructed reference frame is a frame in addition to the series of source video frames, such that a cardinality of the constructed reference frame and the series of source video frames is greater than a cardinality of the series of source video frames; and
generating a series of decoded video frames by decompressing at least a portion of the series of encoded video frames to obtain a series of decompressed video frames, wherein each decompressed video frame from the series of decompressed video frames corresponds with a respective source video frame from the series of source video frames, and wherein at least one encoded video frame from the series of encoded video frames is decompressed using the constructed reference frame, wherein the constructed reference frame has a first encoding quality level and the at least one encoded video frame has a second encoding quality level different from the first encoding quality level.

10. The method according to claim 9 , wherein the constructed reference frame includes:
a plurality of lowest error blocks, wherein the plurality of lowest error blocks are temporally filtered blocks from a plurality of temporally filtered blocks, the plurality of lowest error blocks having respective lowest error scores, and wherein the plurality of temporally filtered blocks are source blocks temporally filtered to remove video noise, from a plurality of source blocks from a plurality of source video frames from the series of source video frames.

11. The method according to claim 9 , wherein the constructed reference frame includes at least a portion of a background of at least one of the series of source video frames.

12. The method according to claim 9 , wherein the constructed reference frame has a size larger than a size of a source video frame from the series of source video frames.

13. The method of claim 9 , wherein generating the series of decoded video frames includes:
determining a sequential order for the series of decoded video frames; and
including the series of decoded video frames in an output bitstream for presentation, such that the constructed reference frame is omitted from the output bitstream.

Assignees

• Bankoski James
• Xu Yaowu
• Wilkins Paul
• Google Inc

Inventors

• Bankoski James
• Xu Yaowu
• Wilkins Paul

Classifications

• H04N19/105

  Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction · CPC title

• H04N19/23

  with coding of regions that are present throughout a whole video segment, e.g. sprites, background or mosaic · CPC title

• H04N19/127

  Prioritisation of hardware or computational resources · CPC title

• H04N19/139

  Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability · CPC title

• H04N19/172

  the region being a picture, frame or field · CPC title