Video stabilization

Patent metadata

Field	Value
Publication number	US-10412305-B2
Application number	US-201414242512-A
Country	US
Kind code	B2
Filing date	Apr 1, 2014
Priority date	May 31, 2011
Publication date	Sep 10, 2019
Grant date	Sep 10, 2019

Abstract

Method, device and computer program product for stabilizing a video signal. A plurality of frames of the video signal are captured using a camera. A motion sensor associated with the camera is used to generate a plurality of samples representing motion of the camera. The samples are used to determine a displacement of the camera between a first time and a second time, wherein the first time corresponds to an exposure time midpoint of a first frame of the video signal and the second time corresponds to an exposure time midpoint of a second frame of the video signal. The determined displacement is used to compensate for motion in the video signal between the first and second frames caused by the motion of the camera, to thereby stabilize the video signal.

First claim

What is claimed is:

1. A method of stabilizing a video signal, the method comprising:
capturing a plurality of frames of the video signal using a camera;
determining a pixel displacement representing motion in the video signal between first and second frames of the video signal caused by motion of the camera; and
shifting an image of at least one of the first and second frames in accordance with the pixel displacement to thereby stabilize the video signal, wherein the image comprises a stride value indicating memory space of the image, a plurality of pixel values, a pointer indicating the position of a first pixel of the image, and a width value indicating the width of the image, wherein said shifting of the image comprises adjusting the pointer and the width value without adjusting the stride value.

2. The method of claim 1 wherein the image comprises a plurality of image planes which are each represented by a respective plurality of pixel values, a respective pointer indicating the position of a first pixel of the image plane, and a respective width value indicating the width of the image plane, wherein the image planes are shifted together by adjusting their respective pointers and width values without adjusting the stride value.

3. The method of claim 1 further comprising rounding the determined pixel displacement to an integer-pixel displacement.

4. The method of claim 1 further comprising:
using a motion sensor associated with the camera to generate a plurality of samples representing motion of the camera; and
using the samples to determine a displacement of the camera between a first frame of the video signal and a second frame of the video signal, wherein the determined displacement is used to determine said pixel displacement.

5. The method of claim 1 further comprising filtering the pixel displacement.

6. The method of claim 1 wherein the motion of the camera is rotational motion.

7. The method of claim 1 , wherein shifting the image further comprises keeping the memory space of the image the same without copying data in the memory space for the image when the image is shifted.

8. A device configured to stabilize a video signal, the device comprising:
a camera configured to capture a plurality of frames of the video signal;
one or more processors; and
one or more computer-readable media storing instructions which, when executed by the one or more processors, perform operations comprising:
determining a pixel displacement representing motion in the video signal between first and second frames of the video signal caused by motion of the camera; and
shifting an image of at least one of the first and second frames in accordance with the pixel displacement to thereby stabilize the video signal, wherein the image comprises a stride value indicating memory space of the image, a plurality of pixel values, a pointer indicating the position of a first pixel of the image, and a width value indicating the width of the image, wherein said shifting comprises shifting the image by adjusting the pointer and the width value without adjusting the stride value.

9. The device of claim 8 , wherein the image comprises a plurality of image planes which are each represented by a respective plurality of pixel values, a respective pointer indicating the position of a first pixel of the image plane, and a respective width value indicating the width of the image plane, wherein the image planes are shifted together by adjusting their respective pointers and width values without adjusting the stride value.

10. The device of claim 8 , the operations further comprising rounding the determined pixel displacement to an integer-pixel displacement.

11. The device of claim 8 , the computing device further comprising a motion sensor, and the operations further comprising:
using the motion sensor to generate a plurality of samples representing motion of the camera; and
using the samples to determine a displacement of the camera between a first frame of the video signal and a second frame of the video signal, wherein the determined displacement is used to determine said pixel displacement.

12. The device of claim 8 , the operations further comprising filtering the pixel displacement.

13. The device of claim 8 , wherein the motion of the camera is rotational motion.

14. The device of claim 8 , the operations further comprising keeping the memory space of the image the same without copying data in the memory space for the image when the image is shifted.

15. A computer program product for stabilizing a video signal, the computer program product being stored on a memory of a computing device and configured so as when executed on a processor of the computing device to perform the operations comprising:
capturing a plurality of frames of the video signal using a camera;
determining a pixel displacement representing motion in the video signal between first and second frames of the video signal caused by motion of the camera; and
shifting an image of at least one of the first and second frames in accordance with the pixel displacement to thereby stabilize the video signal, wherein the image comprises a stride value indicating memory space of the image, a plurality of pixel values, a pointer indicating the position of a first pixel of the image, and a width value indicating the width of the image, wherein said shifting of the image comprises adjusting the pointer and the width value without adjusting the stride value.

16. The computer-program product of claim 15 , wherein the image comprises a plurality of image planes which are each represented by a respective plurality of pixel values, a respective pointer indicating the position of a first pixel of the image plane, and a respective width value indicating the width of the image plane, wherein the image planes are shifted together by adjusting their respective pointers and width values without adjusting the stride value.

17. The computer-program product of claim 15 , the operations further comprising rounding the determined pixel displacement to an integer-pixel displacement.

18. The computer-program product of claim 15 , the operations further comprising:
using a motion sensor associated with the camera to generate a plurality of samples representing motion of the camera; and
using the samples to determine a displacement of the camera between a first frame of the video signal and a second frame of the video signal, wherein the determined displacement is used to determine said pixel displacement.

19. The computer-program product of claim 15 , the operations further comprising filtering the pixel displacement.

20. The computer-program product of claim 15 , wherein shifting the image further comprises keeping the memory space of the image the same without copying data in the memory space for the image when the image is shifted.

Assignees

• Skype

Inventors

• Jefremov Andrei
• Zhao David Yuheng
• Carlsson Pontus

Classifications

• H04N23/682 Primary

  Vibration or motion blur correction · CPC title

• H04N25/443

  by reading pixels from selected two-dimensional [2D] regions of the array, e.g. for windowing or digital zooming · CPC title

• H04N23/6812 Primary

  based on additional sensors, e.g. acceleration sensors · CPC title

• H04N23/683 Primary

  performed by a processor, e.g. controlling the readout of an image memory · CPC title

• H04N5/23264 Primary

  Electricity · mapped topic