Camera parameter-assisted video frame rate up conversion

Patent metadata

Field	Value
Publication number	US-9137569-B2
Application number	US-201113115888-A
Country	US
Kind code	B2
Filing date	May 25, 2011
Priority date	May 26, 2010
Publication date	Sep 15, 2015
Grant date	Sep 15, 2015

Abstract

This disclosure describes techniques for obtaining one or more video capture parameters associated with a video capture device and using the obtained video capture parameters to control a video decoding process. The obtained video capture parameters may be, for example, parameters associated with auto exposure control (AE), auto focus (AF), and auto white balance (AWB), zooming, motion in captured frames, and the like. The video encoding device may embed the video capture parameters in the encoded video bit stream such that a decoder may utilize the camera parameters during frame rate up-conversion.

First claim

What is claimed is:

1. A method for decoding video data comprising:
receiving a bit stream including encoded video data comprising encoded video frame data and embedded data, the video frame data encoded at a first frame rate;
decoding the encoded video data to obtain decoded video frame data;
determining at least one camera parameter from the embedded data; and
up converting the decoded video frame data to a second frame rate based on the at least one camera parameter.

2. The method of claim 1 , wherein the at least one camera parameter used for frame rate up-conversion includes one or more of a zoom factor, auto focus status, lens position, frame luma, auto exposure (AE) convergence status, automatic white balance (AWB) convergence status, global motion information, and frame blurriness information.

3. The method of claim 1 , wherein the embedded data comprises metadata.

4. The method of claim 1 , further comprising:
de-packetizing the at least one camera parameter for each corresponding captured video frame; and
reading the de-packetized at least one camera parameter to up convert the video frame rate.

5. The method of claim 1 , wherein the at least one camera parameter corresponds to zooming information, autofocus information, lens position information, or global motion information of the camera during capture of the at least one video frame.

6. The method of claim 5 , wherein the camera is a wireless telephone.

7. The method of claim 1 , wherein the at least one camera parameter comprises zoom-in information, and wherein up-converting the video frame rate comprises:
decoding the bit stream to generate decoded frames;
scaling the decoded frames based on a factor of the zoom-in information;
predicting motion vectors based on the scaled frames;
interpolating a frame based on the predicted motion vectors.

8. The method of claim 1 , wherein the at least one camera parameter comprises blurriness information, and wherein up-converting the video frame rate comprises:
decoding the bit stream to generate decoded frames;
obtaining the blurriness information from the decoded frames;
selectively determining a motion vector search range based on the blurriness information;
predicting motion vectors from the determined search range; and
interpolating a frame based on the predicted motion vectors.

9. The method of claim 8 , wherein the at least one camera parameter comprises auto-focus information, the method further comprising determining if the decoded frame was captured during a re-focusing operation based on the auto-focus information, and wherein selectively determining the motion vector search range based on the blurriness information is performed when the decoded frame is captured during a re-focusing operation.

10. The method of claim 9 , further comprising interpolating a frame by averaging pixels of neighboring decoded frames when the neighboring decoded frames are not captured during a re-focusing operation.

11. The method of claim 1 , wherein the at least one camera parameter comprises brightness information, and wherein up-converting the video frame rate comprises:
decoding the bit stream to generate a first decoded frame and a second decoded frame;
obtaining the brightness information from the first decoded frame and the second decoded frame;
setting the brightness level of the second decoded frame equal to the brightness level of the first decoded frame based on the brightness information to generate first and second reference frames;
predicting motion vectors of the interpolated frame based on the first and second reference frames;
interpolating a frame based on the predicted motion vectors; and
adjusting brightness of the interpolated frame based on the brightness information.

12. The method of claim 1 , wherein the at least one camera parameter comprises auto-white balance (AWB) information and color gain information, and wherein up-converting the video frame rate comprises:
decoding the bit stream to generate decoded frames;
interpolating a frame based on the decoded frames;
adjusting AWB and color gain of the interpolated frame based on the AWB and color gain information.

13. The method of claim 1 , wherein the at least one camera parameter comprises global motion information, and wherein up-converting the video frame rate comprises:
decoding the bit stream to generate decoded frames;
obtaining the global motion information from the decoded frames;
determining a motion vector search range based on the global motion information;
predicting motion vectors based on the motion vector search range; and
interpolating a frame based on the predicted motion vectors.

14. The method of claim 1 , further comprising displaying the decoded video frames at the second frame rate on an electronic display.

15. A device comprising:
means for receiving a bit stream including encoded video data comprising encoded video frame data and embedded data, the video frame data encoded at a first frame rate;
means for decoding the encoded video data to obtain decoded video frame data;
means for determining at least one camera parameter from the embedded data; and
means for up converting the decoded video frame data to a second frame rate based on the at least one camera parameter.

16. The device of claim 15 , wherein the means for receiving a bit stream and the means for obtaining at least one camera parameter comprises a decoder, and wherein the means for up converting a video frame comprises a processor.

17. The device of claim 15 , wherein the at least one camera parameter used for frame rate up-conversion includes one or more of a zoom factor, auto focus status, lens position, frame luma, auto exposure (AE) convergence status, automatic white balance (AWB) convergence status, global motion information, and frame blurriness information.

18. The device of claim 15 , wherein the embedded data comprises metadata.

19. The device of claim 15 , further comprising:
means for de-packetizing the at least one camera parameter for each corresponding captured video frame; and
means for reading the de-packetized at least one camera parameter to up convert the video frame rate.

20. The device of claim 15 , wherein the at least one camera parameter corresponds to zooming information, autofocus information, lens position information, or global motion information of the camera during capture of the at least one video frame.

21. The device of claim 20 , wherein the camera is a wireless telephone.

22. The device of claim 15 , wherein the at least one camera parameter comprises zoom-in information, and wherein up-converting the video frame rate comprises:
means for decoding the bit stream to generate decoded frames;
means for scaling the decoded frames based on a factor of the zoom-in information;
means for predicting motion vectors based on the scaled frames;
means for interpolating a frame based on the predicted motion vectors.

23. The device of claim 15 , wherein the at least one camera parameter comprises blurriness information, and wherein up-converting the video frame rate comprises:
means for decoding the bit stream to generate decoded frames;
means for obtaining the blurriness information from the decoded frames;
means for selectively determining a motion vector search range based on the blurriness information;
means for predicting motion vectors from the determined search range; and
means for interpolating a frame b

Assignees

• Liang Liang
• Hung Bob R
• Dane Gokce
• Qualcomm Inc

Inventors

• Liang Liang
• Hung Bob R
• Dane Gokce

Classifications

• H04N21/6332

  directed to client · CPC title

• H04N21/2353

  specifically adapted to content descriptors, e.g. coding, compressing or processing of metadata · CPC title

• H04N19/139

  Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability · CPC title

• H04N19/463 Primary

  by compressing encoding parameters before transmission · CPC title

• H04N19/132 Primary

  Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking · CPC title