Video stabilization

Patent metadata

Field	Value
Publication number	US-9635256-B2
Application number	US-201414230767-A
Country	US
Kind code	B2
Filing date	Mar 31, 2014
Priority date	Sep 26, 2011
Publication date	Apr 25, 2017
Grant date	Apr 25, 2017

Abstract

Method, device and computer program product for transmitting a video signal from a user device includes capturing a plurality of frames of the video signal using a camera at the user device, determining a functional state of the device and selectively stabilizing the video signal prior to transmission based on the functional state.

First claim

The invention claimed is:

1. A method comprising:
capturing frames of a video signal using a camera at a computing device;
monitoring, with a motion sensor of the computing device, motion of the camera;
determining a functional state of the computing device based at least in part on one or more samples that are generated by the motion sensor and describe motion of the camera;
responsive to determining that the computing device is in a first functional state, not selectively stabilizing the video signal;
responsive to determining that the computing device is in a second functional state, selectively stabilizing the video signal in real-time, the selectively stabilizing comprising repeating, for each captured frame of the video signal:
determining a pixel displacement between the captured frame and a previous frame of the video signal;
determining an accumulated pixel displacement based on the determined pixel displacement between the captured frame and the previous frame of the video signal;
determining a filtered accumulated pixel displacement for the captured frame based on a weighted sum of the determined accumulated pixel displacement for the captured frame and a filtered accumulated pixel displacement of the previous frame; and
shifting an image of one or more of the captured frame or the previous frame based on the determined filtered pixel displacement; and

transmitting the video signal, with or without stabilization, from the computing device to an additional computing device.

2. The method of claim 1 , wherein the camera comprises a front-facing camera, and wherein the computing device further includes a rear-facing camera.

3. The method of claim 2 , wherein determining that the device is in a first functional state comprises determining that the front-facing camera is selected and used to capture the frames of the video signal.

4. The method of claim 2 , wherein determining that the device is in a second functional state comprises determining that the rear-facing camera is selected and used to capture the frames of the video signal.

5. The method of claim 1 , wherein determining the functional state comprises:
comparing the monitored motion of the camera to a threshold to determine the degree of motion of the camera.

6. The method of claim 5 , the determining comprising:
determining that the computing device is in the second functional state when the degree of motion of the camera exceeds the threshold; and
determining that the computing device is in the first functional state when the degree of motion of the camera is less than the threshold.

7. The method of claim 5 , wherein the monitoring comprises:
using the motion sensor of the computing device to generate a plurality of samples that describe the motion of the camera;
using the samples to determine a displacement of the camera between successive frames captured by the camera; and
determining a pixel displacement representing motion in the video signal between the successive frames caused by the determined displacement of the camera.

8. The method of claim 7 , further comprising:
comparing the pixel displacement to the threshold, and wherein the determining comprises determining that the computing device is in the second functional state when the pixel displacement exceeds said threshold, and determining that the computing device is in the first functional state when the pixel displacement is less than said threshold.

9. The method of claim 7 , wherein the displacement of the camera comprises an angular displacement of the camera.

10. A computer-readable storage device comprising code stored thereon that, responsive to execution by a processor of a device, performs a method comprising:
capturing frames of a video signal using a video camera at the device;
determining whether a front-facing video camera of the device or a rear-facing video camera of the device was used to capture the frames of the video signal;
responsive to determining that the front-facing video camera of the device captured the frames of the video signal, not stabilizing the video signal;
responsive to determining that the rear-facing video camera of the device captured the frames of the video signal, monitoring motion of the rear-facing video camera;
responsive to determining that the monitored motion of the rear-racing video camera exceeds a threshold degree of motion, stabilizing the video signal based at least in part on the monitored motion of the rear-facing video camera; and
transmitting the video signal, with or without stabilization, from the device to an additional device.

11. The computer-readable storage device of claim 10 , wherein determining that the monitored motion of the rear-facing video camera exceeds a threshold degree of motion comprises determining that an angular displacement of the rear-facing video camera exceeds a threshold amount.

12. The computer-readable storage device of claim 10 , wherein the monitoring motion of the rear-facing video camera is performed by a gyroscope of the device.

13. The computer-readable storage device of claim 10 , wherein stabilizing the video signal comprises determining a pixel displacement between successive frames of the video signal and shifting an image of one or more of the successive frames based on the determined pixel displacement.

14. The computer-readable storage device of claim 10 , wherein the monitoring motion of the rear-facing video camera comprises determining a pixel displacement between successive frames of the video signal and comparing the pixel displacement to a threshold.

15. A computing device comprising:
a front-facing camera configured to capture frames of a video signal;
a rear-facing camera configured to capture frames of the video signal;
at least a memory and a processor configured to perform operations comprising:
capturing frames of the video signal using a video camera of the device;
determining whether the front-facing video camera of the device or the rear-facing camera of the device was used to capture the frames of the video signal;
responsive to determining that the front-facing video camera of the device captured the frames of the video signal, not stabilizing the video signal;
responsive to determining that the rear-facing video camera of the device captured the frames of the video signal, monitoring motion of the rear-facing video camera;
responsive to determining that the monitored motion of the rear-facing video camera exceeds a threshold degree of motion, stabilizing the video signal based at least in part on the monitored motion of the rear-facing video camera; and
transmitting the video signal, with or without stabilization, from the device to an additional device.

16. The computing device of claim 15 , wherein determining that the monitored motion of the rear-facing video camera exceeds a threshold degree of motion comprises determining that an angular displacement of the rear-facing video camera exceeds a threshold amount.

17. The computing device of claim 15 , the device further comprising a gyroscope, wherein the monitoring motion of the rear-facing video camera is performed by the gyroscope.

18. The computing device of claim 15 , wherein stabilizing the video signal comprises determining a pixel displacement between successive frames of the video signal and shifting an image of one or more of the successive frames based on the determined pixel displacement.

19. The computing device of claim 15 , wherein the monitoring motion of the rear-facing video camera

Assignees

• Microsoft Corp
• Skype

Inventors

• Owen Christopher
• Carlsson Pontus

Classifications

• H04N23/6812

  based on additional sensors, e.g. acceleration sensors · CPC title

• H04N23/683

  performed by a processor, e.g. controlling the readout of an image memory · CPC title

• H04N23/682 Primary

  Vibration or motion blur correction · CPC title

• H04N23/681 Primary

  Motion detection · CPC title

• H04N5/23267

  Electricity · mapped topic