Stabilization of low-light video

Patent metadata

Field	Value
Publication number	US-9692972-B2
Application number	US-201514843841-A
Country	US
Kind code	B2
Filing date	Sep 2, 2015
Priority date	Mar 27, 2014
Publication date	Jun 27, 2017
Grant date	Jun 27, 2017

Abstract

In one embodiment, a method includes determining a maximum exposure time for capturing one or more image frames of a video clip. The maximum exposure time represents an exposure time below which the captured image frames are substantially free of motion-blur artifacts and above which the captured image frames exhibit motion-blur artifacts. The method also includes capturing the image frames with an exposure time that is less than or equal to the maximum exposure time.

First claim

What is claimed is:

1. A method comprising:
by a computing device, determining a maximum exposure time for capturing one or more image frames of a video clip, wherein:
the maximum exposure time is determined based at least in part on an amount of motion of the computing device as measured by a motion sensor; and
the maximum exposure time represents an exposure time below which the captured image frames are substantially free of motion-blur artifacts and above which the captured image frames exhibit motion-blur artifacts; and

by the computing device, capturing the image frames, wherein each of the captured image frames has an exposure time that is less than or equal to the maximum exposure time.

2. The method of claim 1 , further comprising:
by the computing device, receiving information indicative of an amount of light present for capturing the image frames;
by the computing device, based on the amount of light present, determining a candidate exposure time for capturing the image frames; and
by the computing device, if the candidate exposure time is less than the maximum exposure time, then capturing the image frames with the exposure time equal to the candidate exposure time, else capturing the image frames with the exposure time equal to the maximum exposure time.

3. The method of claim 1 , wherein the amount of motion is further based on information received from a gyroscope or accelerometer.

4. The method of claim 1 , further comprising:
by the computing device, receiving information from the motion sensor indicating a subsequent amount of motion of the computing device at a subsequent point in time;
by the computing device, adjusting the maximum exposure time based on the received information; and
by the computing device, capturing one or more subsequent image frames of the video clip, wherein the exposure time of the subsequent image frames is less than or equal to the adjusted maximum exposure time.

5. The method of claim 1 , further comprising applying a video-stabilization routine to the video clip based on the amount of motion of the computing device as measured by the motion sensor.

6. The method of claim 1 , further comprising:
by the computing device, receiving orientation information from the motion sensor, the orientation information indicating a plurality of orientations or motions of the computing device; and
by the computing device, applying a low-pass filter to the orientation information to determine a stabilized orientation of the computing device.

7. The method of claim 1 , wherein the capturing the image frames comprises capturing the image frames at a frame rate that is higher than a standard frame rate, further comprising:
combining two or more successive captured image frames into a single frame of a combined-frame video clip, wherein the combined-frame video clip has the standard frame rate.

8. The method of claim 7 , wherein the frame rate for capturing the image frames is four times the standard frame rate, and wherein the combining the two or more successive captured image frames into the single frame comprises combining four successive captured image frames into the single frame.

9. One or more computer-readable non-transitory storage media embodying software that is operable when executed by a computing device to:
determine a maximum exposure time for capturing one or more image frames of a video clip, wherein:
the maximum exposure time is determined based at least in part on an amount of motion of the computing device as measured by a motion sensor; and
the maximum exposure time represents an exposure time below which the captured image frames are substantially free of motion-blur artifacts and above which the captured image frames exhibit motion-blur artifacts; and

capture the image frames, wherein each of the captured image frames has an exposure time that is less than or equal to the maximum exposure time.

10. The one or more computer-readable non-transitory storage media of claim 9 , wherein the software is further operable when executed to:
receive information indicative of an amount of light present for capturing the image frames;
determine, based on the amount of light present, a candidate exposure time for capturing the image frames; and
if the candidate exposure time is less than the maximum exposure time, then capture the image frames with the exposure time equal to the candidate exposure time, else capture the image frames with the exposure time equal to the maximum exposure time.

11. The one or more computer-readable non-transitory storage media of claim 9 , wherein the amount of motion is further based on information received from a gyroscope or accelerometer.

12. The one or more computer-readable non-transitory storage media of claim 9 , wherein the software is further operable when executed to:
receive information from the motion sensor indicating a subsequent amount of motion of the computing device;
adjust the maximum exposure time based on the received information; and
capture one or more subsequent image frames of the video clip, wherein the exposure time of the subsequent image frames is less than or equal to the adjusted maximum exposure time.

13. The one or more computer-readable non-transitory storage media of claim 9 , wherein the software is further operable when executed to apply a video-stabilization routine to the video clip based on the amount of motion of the computing device as measured by the motion sensor.

14. The one or more computer-readable non-transitory storage media of claim 9 , wherein the software is further operable when executed to:
receive orientation information from the motion sensor, the orientation information indicating a plurality of orientations or motions of the computing device; and
apply a low-pass filter to the orientation information to determine a stabilized orientation of the computing device.

15. The one or more computer-readable non-transitory storage media of claim 9 , wherein:
the capture the image frames comprises capture the image frames at a frame rate that is higher than a standard frame rate; and
the software is further operable when executed to combine two or more successive captured image frames into a single frame of a combined-frame video clip, wherein the combined-frame video clip has the standard frame rate.

16. The one or more computer-readable non-transitory storage media of claim 15 , wherein the frame rate for capturing the image frames is four times the standard frame rate, and wherein the combine the two or more successive captured image frames into the single frame comprises combine four successive captured image frames into the single frame.

17. A computing device comprising:
one or more processors; and
a memory coupled to the processors comprising instructions executable by the processors, the processors being operable when executing the instructions to:
determine a maximum exposure time for capturing one or more image frames of a video clip, wherein:
the maximum exposure time is determined based at least in part on an amount of motion of the computing device as measured by a motion sensor; and
the maximum exposure time represents an exposure time below which the captured image frames are substantially free of motion-blur artifacts and above which the captured image frames exhibit motion-blur artifacts; and

capture the image frames, wherein each of the captured image frames has an exposure time that is less than or equal to the maximum exposure time.

18. The computing device of claim 17 , wher

Assignees

• Facebook Inc

Inventors

• Karpenko Alexandre

Classifications

• H04N23/682

  Vibration or motion blur correction · CPC title

• H04N23/73

  by influencing the exposure time · CPC title

• H04N23/684

  performed by controlling the image sensor readout, e.g. by controlling the integration time · CPC title

• H04N23/68 Primary

  for stable pick-up of the scene, e.g. compensating for camera body vibrations · CPC title

• H04N23/6812 Primary

  based on additional sensors, e.g. acceleration sensors · CPC title