Automatic video quality enhancement with temporal smoothing and user override

Patent metadata

Field	Value
Publication number	US-9934558-B2
Application number	US-201615259396-A
Country	US
Kind code	B2
Filing date	Sep 8, 2016
Priority date	Jun 14, 2014
Publication date	Apr 3, 2018
Grant date	Apr 3, 2018

Abstract

Technologies for a single-pass process for enhancing video quality with temporal smoothing. The process may include providing for user overrides of automatically enhanced video/frame characteristics and providing substantially immediate previews of enhanced video frames to a user. The process may also include detecting a degree of shakiness in a portion of the video, and performing or recommending stabilization based on the detected shakiness.

First claim

The invention claimed is:

1. A method performed on a computing device, the method for enhancing quality of a series of video frames, the method comprising:
generating, by the computing device based on a characteristic of a first video frame in the series of video frames, a first correction curve for the first video frame in the series of video frames;
adjusting, by the computing device based on a number of additional correction curves that are each for one of the number of additional video frames in the series of video frames, the generated first correction curve for the first video frame in the series of video frames, where the number is greater than one; and
enhancing the quality of the series of video frames by applying, by the computing device, the adjusted correction curve to the characteristic of the first video frame.

2. The method of claim 1 where the additional video frames are located before the first video frame in the series of video frames.

3. The method of claim 1 further comprising weighting the additional video frames.

4. The method of claim 3 where video frames in the series of video frames that are closer to the first video frame are weighted greater than the video frames in the series of video frames that are farther from the first video frame.

5. The method of claim 1 where the number is between 4 and 10.

6. An electronic hardware system comprising:
at least one processor;
memory that is coupled to the at least one processor and that includes computer-executable instructions that, based on execution by the at least one processor, configure the electronic hardware system to perform actions for enhancing quality of a series of video frames, the actions comprising;
generating, based on a characteristic of a first video frame in the series of video frames, a first correction curve for the first video frame in the series of video frames;
adjusting, based on a number of additional correction curves that are each for one of the number of additional video frames in the series of video frames, the generated first correction curve for the first video frame in the series of video frames, where the number is greater than one; and
enhancing the quality of the series of video frames by applying the adjusted correction curve to the characteristic of the first video frame.

7. The electronic hardware system of claim 6 where the additional video frames are located before the first video frame in the series of video frames.

8. The electronic hardware system of claim 6 further comprising a temporal smoother configured to weight the additional video frames.

9. The electronic hardware system of claim 8 where video frames in the series of video frames that are closer to the first video frame are weighted greater than the video frames in the series of video frames that are farther from the first video frame.

10. The electronic hardware system of claim 6 where the number is between 4 and 10.

11. At least one hardware computer-readable medium that includes computer-executable instructions that, based on execution by a computing device, configure the computing device to perform actions for enhancing quality of a series of video frames, the actions comprising:
generating, by the computing device based on a characteristic of a first video frame in the series of video frames, a first correction curve for the first video frame in the series of video frames;
adjusting, by the computing device based on a number of additional correction curves that are each for one of the number of additional video frames in the series of video frames, the generated first correction curve for the first video frame in the series of video frames, where the number is greater than one; and
enhancing the quality of the series of video frames by applying, by the computing device, the adjusted correction curve to the characteristic of the first video frame.

12. The at least one hardware computer-readable medium of claim 11 where the additional video frames are located before the first video frame in the series of video frames.

13. The at least one hardware computer-readable medium of claim 11 , the actions further comprising weighting the additional video frames.

14. The at least one hardware computer-readable medium of claim 13 where video frames in the series of video frames that are closer to the first video frame are weighted greater than the video frames in the series of video frames that are farther from the first video frame.

15. The at least one hardware computer-readable medium of claim 11 where the number is between 4 and 10.

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Suri Nitin
• Ivory Andrew Shaun
• Wang Tzong-Jhy
• Lindbloom Bruce Justin
• Sproule William David

Classifications

• G06T5/40

  using histogram techniques · CPC title

• G06T5/50

  using two or more images, e.g. averaging or subtraction · CPC title

• G06T2207/20182

  Noise reduction or smoothing in the temporal domain; Spatio-temporal filtering · CPC title

• G06T2207/20092

  Interactive image processing based on input by user · CPC title

• G06T2207/20072

  Graph-based image processing · CPC title