Systems and methods for generating time-lapse videos

Patent metadata

Field	Value
Publication number	US-10916272-B2
Application number	US-202016786947-A
Country	US
Kind code	B2
Filing date	Feb 10, 2020
Priority date	May 14, 2018
Publication date	Feb 9, 2021
Grant date	Feb 9, 2021

Abstract

Video content may be captured by an image capture device during a capture duration. The video content may include video frames that define visual content viewable as a function of progress through a progress length of the video content. Rotational position information may characterize rotational positions of the image capture device during the capture duration. Time-lapse video frames may be determined from the video frames of the video content based on a spatiotemporal metric. The spatiotemporal metric may characterize spatial smoothness and temporal regularity of the time-lapse video frames. The spatial smoothness may be determined based on the rotational positions of the image capture device corresponding to the time-lapse video frames, and the temporal regularity may be determined based on moments corresponding to the time-lapse video frames. Time-lapse video content may be generated based on the time-lapse video frames.

First claim

What is claimed is:

1. A system that generates time-lapse videos, the system comprising:
one or more physical processors configured by machine-readable instructions to:
obtain video information defining video content, the video content captured by an image capture device during a capture duration, the video content having a progress length, the video content including video frames that define visual content viewable as a function of progress through the progress length, the video frames corresponding to moments within the progress length;
obtain rotational position information of the image capture device, the rotational position information characterizing rotational positions of the image capture device during the capture duration;
determine time-lapse video frames from the video frames of the video content based on a spatiotemporal metric including a temporal velocity component, a temporal acceleration component, a spatial velocity component, and a spatial acceleration component, the temporal velocity component determined based on a comparison of a speed pattern of the time-lapse video frames and a desired speed pattern for the time-lapse video content, the temporal acceleration component determined based on differences between moments corresponding to the time-lapse video frames, the spatial velocity component determined based on angular velocity of the image capture device during capture of the time-lapse video frames, the spatial acceleration component determined based on angular acceleration of the image capture device during capture of the time-lapse video frames; and
generate time-lapse video content based on the time-lapse video frames, the time-lapse video content including a fewer number of video frames than the video content.

2. The system of claim 1 , wherein the spatiotemporal metric characterizes spatial smoothness and temporal regularity of the time-lapse video frames, the spatial smoothness determined based on the rotational positions of the image capture device corresponding to the time-lapse video frames and the temporal regularity determined based on the moments corresponding to the time-lapse video frames.

3. The system of claim 2 , wherein the spatial smoothness is determined further based on one or more transformations applied to the time-lapse video frames.

4. The system of claim 1 , wherein the spatiotemporal metric further includes a content component, the content component determined based on the visual content of the time-lapse video frames, and the spatiotemporal metric further characterizes content characteristics of the time-lapse video frames.

5. The system of claim 4 , wherein the speed pattern of the time-lapse video frames is determined based on the content characteristics of the time-lapse video frames.

6. The system of claim 5 , wherein the speed pattern of the time-lapse video frames includes a speed-up for a sub-set of the time-lapse video frames based on the content characteristics of the sub-set of the time-lapse video frames indicating no highlight event within the sub-set of the time-lapse video frames.

7. The system of claim 1 , wherein the one or more physical processors are, to determine the time-lapse video frames, further configured to:
select a set of video frames of the video content based on the spatiotemporal metric; and
stabilize at least some of the set of video frames.

8. The system of claim 7 , wherein the time-lapse video content is generated based on storage of the set of video frames in a frame selection buffer and a stabilization buffer.

9. The system of claim 1 , wherein the rotational position information is generated by a gyroscope, an accelerometer, or an inertial measurement unit.

10. The system of claim 1 , wherein the time-lapse video frames are determined further based on a skipping bound.

11. A method for generating time-lapse videos, the method performed by a computing system including one or more processors, the method comprising:
obtaining, by the computing system, video information defining video content, the video content having a progress length, the video content including video frames that define visual content viewable as a function of progress through the progress length, the video frames corresponding to moments within the progress length;
obtaining, by the computing system, rotational position information of the image capture device, the rotational position information characterizing rotational positions of the image capture device during the capture period;
determining, by the computing system, time-lapse video frames from the video frames of the video content based on a spatiotemporal metric including a temporal velocity component, a temporal acceleration component, a spatial velocity component, and a spatial acceleration component, the temporal velocity component determined based on a comparison of a speed pattern of the time-lapse video frames and a desired speed pattern for the time-lapse video content, the temporal acceleration component determined based on differences between moments corresponding to the time-lapse video frames, the spatial velocity component determined based on angular velocity of the image capture device during capture of the time-lapse video frames, the spatial acceleration component determined based on angular acceleration of the image capture device during capture of the time-lapse video frames; and
generating, by the computing system, time-lapse video content based on the time-lapse video frames, the time-lapse video content including a fewer number of video frames than the video content.

12. The method of claim 11 , wherein the spatiotemporal metric characterizes spatial smoothness and temporal regularity of the time-lapse video frames, the spatial smoothness determined based on the rotational positions of the image capture device corresponding to the time-lapse video frames and the temporal regularity determined based on the moments corresponding to the time-lapse video frames.

13. The method of claim 12 , wherein the spatial smoothness is determined further based on one or more transformations applied to the time-lapse video frames.

14. The method of claim 11 , wherein the spatiotemporal metric further includes a content component, the content component determined based on the visual content of the time-lapse video frames, and the spatiotemporal metric further characterizes content characteristics of the time-lapse video frames.

15. The method of claim 14 , wherein the speed pattern of the time-lapse video frames is determined based on the content characteristics of the time-lapse video frames.

16. The method of claim 15 , wherein the speed pattern of the time-lapse video frames includes a speed-up for a sub-set of the time-lapse video frames based on the content characteristics of the sub-set of the time-lapse video frames indicating no highlight event within the sub-set of the time-lapse video frames.

17. The method of claim 11 , wherein determining the time-lapse video frames includes:
selecting a set of video frames of the video content based on the spatiotemporal metric; and
stabilizing at least some of the set of video frames.

18. The method of claim 17 , wherein the time-lapse video content is generated based on storage of the set of video frames in a frame selection buffer and a stabilization buffer.

19. The method of claim 11 , wherein the rotational position information is generated by a gyroscope, an accelerometer, or an inertial measurement unit.

20. The method of claim 11 , wherein the time

Assignees

• Gopro Inc

Inventors

• Derbanne Thomas
• Douady César
• Karpushin Maxim

Classifications

• H04N5/2621 Primary

  Cameras specially adapted for the electronic generation of special effects during image pickup, e.g. digital cameras, camcorders, video cameras having integrated special effects capability · CPC title

• H04N23/683

  performed by a processor, e.g. controlling the readout of an image memory · CPC title

• H04N23/6812

  based on additional sensors, e.g. acceleration sensors · CPC title

• G11B27/031

  Electronic editing of digitised analogue information signals, e.g. audio or video signals · CPC title

• G11B27/005 Primary

  Reproducing at a different information rate from the information rate of recording (for television signals H04N5/783) · CPC title