Received video stabilization

Patent metadata

Field	Value
Publication number	US-9762799-B2
Application number	US-201113339879-A
Country	US
Kind code	B2
Filing date	Dec 29, 2011
Priority date	Oct 14, 2011
Publication date	Sep 12, 2017
Grant date	Sep 12, 2017

Abstract

Method, device and computer program product for stabilizing a received video signal. A plurality of frames of the video signal is captured using a camera. The video signal is transmitted as an encoded bit stream. Displacement of the camera between successive frames is determined from a motion indication in the bit stream. The determined displacement is used to compensate for motion in the video signal between successive frames caused by the motion of the camera, to thereby stabilize the video signal.

First claim

What is claimed is:

1. A method of stabilizing a video signal, the method comprising:
decoding from an encoded bit stream a plurality of frames of a video signal including a reference frame and one or more motion vectors representing a successive frame captured by a camera;
generating the successive frame from the reference frame and the one or more motion vectors;
storing an image of the successive frame in a memory space;
using the one or more motion vectors to determine a displacement of the camera between the reference frame and the successive frame; and
using the determined displacement to compensate for motion in the decoded video signal between the reference frame and the successive frame caused by the motion of the camera, to thereby stabilize the decoded video signal, including:
shifting the image to generate a shifted version of the image by creating an adjusted pointer of the image and an adjusted width value of the image according to the displacement from a region of the image to a corresponding region of the shifted version of the image;
storing the adjusted pointer and adjusted width value; and
retrieving the shifted version of the image from the memory space by utilizing the adjusted pointer and the adjusted width value without changing the memory space of the image and without copying data of the image.

2. The method of claim 1 , wherein the step of determining a displacement of the camera further comprises receiving samples from a motion sensor associated with the camera and provided in the bit stream.

3. The method of claim 1 , wherein said using the determined displacement to compensate for motion in the video signal between the reference frame and the successive frame caused by the motion of the camera further comprises:
filtering the displacement; and
shifting the image of the successive frame in accordance with the filtered displacement to thereby stabilize the video signal.

4. The method of claim 3 , wherein said filtering the displacement comprises:
determining an accumulated pixel displacement based on said determined displacement for the successive frame; and
determining a filtered accumulated pixel displacement for the successive frame based on a weighted sum of the determined accumulated pixel displacement for the successive frame and a filtered accumulated pixel displacement for a previous frame.

5. The method of claim 3 wherein the successive frame is received after a previous frame.

6. The method of claim 3 wherein a previous frame is received after the successive frame, the successive frame having been buffered until the previous frame is received.

7. The method of claim 1 , wherein the image comprises a plurality of image planes, each of the plurality of image planes is represented by a respective pointer and a respective width value, and said shifting the image comprises shifting the plurality of image planes together by adjusting the pointer for each respective image plane by the same amount and by adjusting the width value for each respective image plane by the same amount.

8. The method of claim 1 , further comprising rounding the determined displacement to an integer-pixel displacement.

9. A system for stabilizing a video signal, the system comprising:
one or more processors;
a network receiving module including logic that is executable by the one or more processors to perform operations including receiving an encoded bit stream;
a decoder module including logic that is executable by the one or more processors to perform operations including decoding from the bit stream a plurality of frames of the video signal including successive frames captured by a camera;
a memory space configured to store an image of at least one of the successive frames;
a displacement determining module including logic that is executable by the one or more processors to perform operations including using a motion indication received in the bit stream to determine a displacement of the camera between the successive frames;
a motion compensation module including logic that is executable by the one or more processors to perform operations including:
using the determined displacement to compensate for motion in the decoded video signal between the successive frames caused by the motion of the camera, to thereby stabilize the decoded video signal, by shifting the image in accordance with the displacement including adjusting a width value of the image and adjusting a pointer of the image from a region of the image to a corresponding region of a shifted version of the image;
storing the adjusted pointer and adjusted width value; and
retrieving the shifted version of the image from the memory space by utilizing the adjusted pointer and the adjusted width value without changing the memory space of the image and without copying data of the image.

10. The system of claim 9 , wherein the device system is a mobile device.

11. The system of claim 9 , comprising a display configured to display the decoded, stabilized video signal.

12. The system of claim 9 , wherein the motion compensation module is further configured to use the determined displacement to compensate for motion in the video signal between the successive frames caused by the motion of the camera by performing operations including:
filtering the displacement; and
shifting the image of at least one of the first and second frames in accordance with the filtered displacement to thereby stabilize the video signal.

13. A memory storing a computer program product that is executable on one or more processors to cause the one or more processors to perform operations comprising:
receiving an encoded bit stream;
decoding from the encoded bit stream a plurality of frames of a video signal including successive frames captured by a camera;
storing an image of at least one of the successive frames in a memory space;
determining from the encoded bit stream an indication of motion of the camera;
using the motion indication to determine a displacement of the camera between the successive frames; and
using the determined displacement to compensate for motion in the decoded video signal between the successive frames caused by the motion of the camera to stabilize the decoded video signal by:
shifting the image to generate a shifted version of the image by creating an adjusted pointer of the image and an adjusted width value of the image according to the displacement from a region of the image to a corresponding region of the shifted version of the image;
storing the adjusted pointer and adjusted width value; and
retrieving the shifted version of the image from the memory space by utilizing the adjusted pointer and the adjusted width value without changing the memory space of the image and without copying data of the image.

14. The memory of claim 13 , wherein said determining an indication of motion comprises receiving samples from a motion sensor associated with the camera and provided in the bit stream.

15. The memory of claim 13 , wherein the image comprises a plurality of image planes, each of the plurality of image planes is represented by a respective pointer and a respective width value, and said shifting the image comprises shifting the plurality of image planes together by adjusting the pointer for each respective image plane by the same amount and by adjusting the width value for each respective image plane by the same amount.

16. The method of claim 1 , wherein:
storing an image of the successive frame in a memory space comprises storing a plurality of pixel values of the image in the memory space and s

Assignees

• Carlsson Pontus
• Jefremov Andrei
• Sablin Sergey
• Zhao David
• Skype

Inventors

• Carlsson Pontus
• Jefremov Andrei
• Sablin Sergey
• Zhao David

Classifications

• H04N23/6812 Primary

  based on additional sensors, e.g. acceleration sensors · CPC title

• H04N23/683 Primary

  performed by a processor, e.g. controlling the readout of an image memory · CPC title

• H04N23/68 Primary

  for stable pick-up of the scene, e.g. compensating for camera body vibrations · CPC title

• H04N5/23248 Primary

  Electricity · mapped topic

• H04N5/23267

  Electricity · mapped topic