Reducing computational complexity when video encoding uses bi-predictively encoded frames

Patent metadata

Field	Value
Publication number	US-9788010-B2
Application number	US-77314510-A
Country	US
Kind code	B2
Filing date	May 4, 2010
Priority date	May 7, 2009
Publication date	Oct 10, 2017
Grant date	Oct 10, 2017

Abstract

Several techniques aimed at reducing computational complexity when encoding uses bi-predictively encoded frames (B-frames) are implemented in a video encoder. In an embodiment, B-frames are not used as reference frames for encoding P-frames and other B-frames. Non-use of B-frames allows a de-blocking filter used in the video encoder to be switched off when reconstructing encoded B-frames, and use of a lower complexity filter for fractional-resolution motion search for B-frames. In another embodiment, cost functions used in motion estimation for B-frames are simplified to reduce computational complexity. In one more embodiment, fractional pixel refinement in motion search for B-frames is simplified. In yet another embodiment, predictors used in motion estimation for a macro-block in a P-frame are selected from a B-frame that uses a same reference frame as the P-frame.

First claim

What is claimed is:

1. A method of encoding image frames, the method comprising:
encoding, with one or more integrated circuits, each of a plurality of image frames as a bi-predictively encoded frame (B-frame) or a non bi-predictively encoded frame (non B-frame) to form a plurality of encoded image frames;
disabling, with the one or more integrated circuits, a de-blocking filter for all encoded B-frames included in the plurality of encoded image frames;
for each non B-frame in the plurality of image frames, employing a filter with a first number of taps to perform sub-pixel interpolation to determine one or more motion vectors for the respective image frame; and
for each B-frame in the plurality of image frames, employing a filter with a second number of taps less than the first number of taps to perform sub-pixel interpolation to determine the one or more motion vectors for the respective image frame.

2. The method of claim 1 , wherein
each of the motion vectors specifies a displacement of a respective one of a plurality of macro-blocks in the respective image frame with respect to a reference frame.

3. The method of claim 1 , wherein the one or more integrated circuits are included in a video encoder, wherein the video encoder includes a first storage component external to the integrated circuits, and wherein bits representing a reconstructed image frame are written to the first storage component only if the reconstructed image frame is a non-B frame.

4. The method of claim 1 , wherein encoding each of the image frames includes:
encoding an image frame to form an encoded B-frame;
receiving pixel values representing a first macro-block contained in the image frame; and
determining, for the first macro-block, a cost associated with a potential best-match macro-block in a reference frame, the potential best-match macro-block in the reference frame potentially representing a best-match displacement of the first macro-block in the reference frame, wherein the cost contains a similarity indicator component and a motion vector cost component,
wherein determining the cost includes determining the motion-vector cost component based on motion vectors computed for each of a plurality of macro-blocks occurring earlier than the first macro-block in raster scan order in the image frame, as well as a predicted motion vector of an immediately preceding macro-block in raster scan order,
wherein the predicted motion vector represents an estimate of displacement of the immediately preceding macro-block in raster scan order with respect to the reference frame.

5. The method of claim 4 , wherein the cost contains only the similarity indicator component for evaluation of a motion vector predictor for motion estimation for a macro-block in the image frame.

6. The method of claim 5 , wherein the similarity indicator component is computed as a sum of absolute differences (SAD) of corresponding pixel values of pixels in the first macro-block and a potential best-match reference macro-block, and
wherein the B-frame is not used as a reference frame for encoding other frames.

7. A device comprising one or more integrated circuits configured to:
encode each of a plurality of image frames as a bi-predictively encoded frame (B-frame) or a non bi-predictively encoded frame (non B-frame) to form a plurality of encoded image frames;
reconstruct each of the encoded image frames to generate corresponding reconstructed image frames, the reconstructed image frames including reconstructed B-frames and reconstructed non B-frames;
disable a de-blocking filter for all reconstructed B-frames included in the reconstructed image frames;
enable the de-blocking filter for all reconstructed non B-frames included in the reconstructed image frames; and
for each of the plurality of image frames, selectively employ a first filter with a first number of taps or a second filter with a second number of taps to perform sub-pixel interpolation to determine one or more motion vectors for the respective image frame based on whether the respective image frame is a B-frame or a non B-frame.

8. The device of claim 7 , wherein the integrated circuits are further configured to selectively employ the first or second filter by:
for each non B-frame in the plurality of image frames, employing a filter with a first number of taps to perform sub-pixel interpolation to determine the one or more motion vectors for the respective image frame; and
for each B-frame in the plurality of image frames, employing a filter with a second number of taps less than the first number of taps to perform sub-pixel interpolation to determine the one or more motion vectors for the respective image frame.

9. The device of claim 7 , wherein
each of the motion vectors specifies a displacement of a respective one of a plurality of macro-blocks in the respective image frame with respect to a reference frame.

10. The device of claim 7 , wherein the one or more integrated circuits are included in a video encoder, wherein the video encoder includes a first storage component external to the integrated circuits, and wherein bits representing a reconstructed image frame are written to the first storage component only if the reconstructed image frame is a non-B frame.

11. The device of claim 7 , wherein the one or more integrated circuits are further configured to:
encode an image frame to form an encoded B-frame;
receive pixel values representing a first macro-block contained in the image frame; and
determine, for the first macro-block, a cost associated with a potential best-match macro-block in a reference frame, the potential best-match macro-block in the reference frame potentially representing a best-match displacement of the first macro-block in the reference frame, wherein the cost contains a similarity indicator component and a motion vector cost component,
wherein the one or more integrated circuits are further configured to determine the cost at least in part by determining the motion-vector cost component based on motion vectors computed for each of a plurality of macro-blocks occurring earlier than the first macro-block in raster scan order in the image frame, as well as a predicted motion vector of an immediately preceding macro-block in raster scan order,
wherein the predicted motion vector represents an estimate of displacement of the immediately preceding macro-block in raster scan order with respect to the reference frame.

12. The device of claim 11 , wherein the cost contains only the similarity indicator component for evaluation of a motion vector predictor for motion estimation for a macro-block in the image frame.

13. The device of claim 12 ,
wherein the similarity indicator component is computed as a sum of absolute differences (SAD) of corresponding pixel values of pixels in the first macro-block and a potential best-match reference macro-block, and
wherein the B-frame is not used as a reference frame for encoding other frames.

14. A machine readable medium comprising instructions that, when executed by a video encoder, cause the video encoder to:
encode each of a plurality of image frames as a bi-predictively encoded frame (B-frame) or a non bi-predictively encoded frame (non B-frame) to form a plurality of encoded image frames;
reconstruct each of the encoded image frames to generate corresponding reconstructed image frames, the reconstructed image frames including reconstructed B-frames and reconstructed non B-frames;
disable a de-blocking filter for all reconstructed B-frames included in the reconstructed image frames; and
enable the de-blocking filter for all reconstructed

Assignees

• Nagori Soyeb
• Kudana Arun Shankar
• Swami Pramod Kumar
• Texas Instruments Inc

Inventors

• Nagori Soyeb
• Kudana Arun Shankar
• Swami Pramod Kumar

Classifications

• H04N19/61

  in combination with predictive coding · CPC title

• H04N19/523 Primary

  with sub-pixel accuracy · CPC title

• H04N19/172

  the region being a picture, frame or field · CPC title

• H04N19/156

  Availability of hardware or computational resources, e.g. encoding based on power-saving criteria · CPC title

• H04N19/147

  according to rate distortion criteria (rate-distortion as a criterion for motion estimation H04N19/567) · CPC title