Systems and methods for stabilizing videos

Patent metadata

Field	Value
Publication number	US-10432864-B1
Application number	US-201916392501-A
Country	US
Kind code	B1
Filing date	Apr 23, 2019
Priority date	Sep 19, 2018
Publication date	Oct 1, 2019
Grant date	Oct 1, 2019

Abstract

Visual content is captured by an image capture device during a capture duration. The image capture devices experiences change in position during the capture duration. The trajectory of the image capture device is smoothed based on a look ahead of the trajectory. A punchout of the visual content is determined based on the smoothed trajectory. The punchout of the visual content is used to generate stabilized visual content.

First claim

What is claimed is:

1. An image capture system that stabilizes videos, the image capture system comprising:
a housing;
an optical element carried by the housing and configured to guide light within a field of view to an image sensor;
the image sensor carried by the housing and configured to generate a visual output signal conveying visual information based on light that becomes incident thereon during a capture duration, the visual information defining visual content having the field of view;
a position sensor carried by the housing and configured to generate a position output signal based on positions of the housing during the capture duration, the position output signal conveying position information that characterizes rotational positions of the housing at different moments within the capture duration; and
one or more physical processors configured by machine-readable instructions to:
determine a trajectory of the housing during the capture duration based on the position information, the trajectory reflecting the rotational positions of the housing at the different moments within the capture duration, the trajectory including a first portion corresponding to a first moment within the capture duration and a second portion corresponding to a second moment subsequent to the first moment within the capture duration;
determine a smoothed trajectory of the housing based on a subsequent portion of the trajectory such that a portion of the smoothed trajectory corresponding to the first portion of the trajectory is determined based on the second portion of the trajectory, the smoothed trajectory having smoother changes in the rotational positions of the housing than the trajectory;
determine a viewing window for the visual content based on the smoothed trajectory of the housing, the viewing window defining one or more extents of the visual content; and
generate stabilized visual content of a video based on the viewing window, the stabilized visual content including a punchout of the one or more extents of the visual content within the viewing window.

2. The image capture system of claim 1 , wherein the position information further characterizes translational positions of the housing at the different moments within the capture duration and the trajectory further reflects the translational positions of the housing at the different moments within the capture duration.

3. The image capture system of claim 1 , wherein at least one of the one or more processors is a remote processor located remotely from the housing, and generation of the stabilized visual content is performed by the remote processor post capture of the visual content.

4. The image capture system of claim 1 , wherein the smoothed trajectory having smoother changes in the rotational positions of the housing than the trajectory is characterized by the smoothed trajectory having less jitters in the rotational positions of the housing than the trajectory.

5. The image capture system of claim 4 , wherein the smoothed trajectory having smoother changes in the rotational positions of the housing than the trajectory includes high frequency changes in the rotational positions of the housing in the trajectory being removed from the smoothed trajectory.

6. The image capture system of claim 1 , wherein the determination of the smoothed trajectory of the housing based on the subsequent portion of the trajectory includes:
obtaining a rotational position of the housing at the first moment within the capture duration, the first moment being a point in time and corresponding to a video frame of the visual content captured at the point in time within the capture duration;
obtaining rotational positions of the housing at the second moment within the capture duration, the second moment being a duration of time subsequent to the point in time within the capture duration; and
determining a corresponding rotational position of the housing within the smoothed trajectory at the first moment based on the rotational position of the housing at the point in time and the rotational positions of the housing during the duration of time subsequent to the point in time;
wherein placement of the viewing window for the visual content with respect to the field of view of the visual content captured at the first moment is determined based on the corresponding rotational position of the housing within the smoothed trajectory at the first moment.

7. The image capture system of claim 6 , wherein the determination of the smoothed trajectory of the housing based on the subsequent portion of the trajectory further includes:
determining whether the placement of the viewing window for the visual content at the first moment results in one or more portions of the viewing window exceeding the field of view of the visual content;
responsive to a determination that the one or more portions of the viewing window for the visual content at the first moment exceeds the field of view of the visual content, adjusting the corresponding rotational position of the housing within the smoothed trajectory at the first moment such that the viewing window for the visual content at the first moment does not exceed the field of view of the visual content.

8. The image capture system of claim 6 , wherein the corresponding rotational position of the housing within the smoothed trajectory at the first moment is initially determined based on a combination of the rotational position of the housing at the first moment and an estimate of the corresponding rotational position of the housing.

9. The image capture system of claim 8 , wherein the estimate of the corresponding rotational position of the housing is determined based on minimization of a combination of a rotational velocity of the housing and a rotational acceleration of the housing.

10. The image capture system of claim 1 , wherein an extent to which the smoothed trajectory of the housing deviates from the trajectory of the housing depends on an amount of rotational motion experienced by the housing during the capture duration and an exposure time with which the visual content is captured.

11. A method for stabling videos, the method performed by an image capture system including one or more processors, an optical element, an image sensor, and a position sensor, the optical element carried by an image capture device and configured to guide light within a field of view to the image sensor, the method comprising:
generating, by the image sensor, a visual output signal conveying visual information based on light that becomes incident thereon during a capture duration, the visual information defining visual content having the field of view;
generating, by the position sensor, a position output signal based on positions of the image capture device during the capture duration, the position output signal conveying position information that characterizes rotational positions of the image capture device at different moments within the capture duration;
determining, by the one or more processors, a trajectory of the image capture device during the capture duration based on the position information, the trajectory reflecting the rotational positions of the image capture device at the different moments within the capture duration, the trajectory including a first portion corresponding to a first moment within the capture duration and a second portion corresponding to a second moment subsequent to the first moment within the capture duration;
determining, by the one or more processors, a smoothed trajectory of the image capture device based on a subsequent portion of the trajectory such that a portion of the smoothed trajectory corresponding to the first porti

Assignees

• Gopro Inc

Inventors

• Douady Cesar
• Derbanne Thomas
• Karpushin Maxim

Classifications

• H04N23/55

  Optical parts specially adapted for electronic image sensors; Mounting thereof · CPC title

• H04N23/631

  Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters · CPC title

• H04N23/51

  Housings · CPC title

• H04N23/683 Primary

  performed by a processor, e.g. controlling the readout of an image memory · CPC title

• H04N23/6812

  based on additional sensors, e.g. acceleration sensors · CPC title