Scene and activity identification in video summary generation based on motion detected in a video

Patent metadata

Field	Value
Publication number	US-9646652-B2
Application number	US-201514705864-A
Country	US
Kind code	B2
Filing date	May 6, 2015
Priority date	Aug 20, 2014
Publication date	May 9, 2017
Grant date	May 9, 2017

Abstract

Video and corresponding metadata is accessed. Events of interest within the video are identified based on the corresponding metadata, and best scenes are identified based on the identified events of interest. In one example, best scenes are identified based on the motion values associated with frames or portions of a frame of a video. Motion values are determined for each frame and portions of the video including frames with the most motion are identified as best scenes. Best scenes may also be identified based on the motion profile of a video. The motion profile of a video is a measure of global or local motion within frames throughout the video. For example, best scenes are identified from portion of the video including steady global motion. A video summary can be generated including one or more of the identified best scenes.

First claim

What is claimed is:

1. A method for identifying scenes in captured video for inclusion in a video summary, the method comprising:
accessing a video, the video including a plurality of frames;
determining a velocity of content of individual frames of the video;
determining an acceleration of the content of the individual frames of the video based on at least two of the velocities of the content of the individual frames;
determining a score for the individual frames of the video based on the velocity and the acceleration of the content of the individual frames;
selecting one or more of the individual frames of the video based on the determined scores;
identifying, for individual selected frames, a corresponding video scene, the corresponding video scene comprising a first amount of the video occurring before the selected frame and a second amount of the video occurring after the selected frame; and
selecting one or more of the identified video scenes for inclusion in the video summary.

2. The method of claim 1 , further comprising:
receiving a request for the video summary from a user; and
generating the video summary including the selected video scenes.

3. The method of claim 2 , wherein generating the video summary comprises concatenating the selected video scenes.

4. The method of claim 1 , wherein determining the velocity of the content of the individual frames of the video comprises:
determining a magnitude of one or more motion vectors in the individual frames; and
determining the velocity of the content of the individual frames based on the magnitude of the one or more motion vectors in the individual frames.

5. The method of claim 1 , wherein determining the acceleration of the content of the individual frames of the video comprises:
determining the velocities of the content of two adjacent individual frames; and
determining the acceleration of the content of the individual frames based on the difference between the velocities of the content of the two adjacent individual frames.

6. The method of claim 5 , further comprising:
aggregating a threshold number of the accelerations of the content of the individual frames in a sequence of frames; and
determining a single acceleration for the individual frames in the sequence of frames based on the aggregated accelerations.

7. The method of claim 1 , further comprising:
ranking the identified video scenes based on a likelihood that the identified video scenes will be of interest to a user;
wherein the one or more of the identified video scenes are selected for inclusion in the video summary based on the ranking.

8. A system for identifying scenes in captured video for inclusion in a video summary, the system comprising:
a non-transitory computer-readable storage medium comprising executable computer instructions configured to, when executed, cause a hardware processor to perform steps comprising:
accessing a video, the video including a plurality of frames;
determining a velocity of content of individual frames of the video;
determining an acceleration of the content of the individual frames of the video based on at least two of the velocities of the content of the individual frames;
determining a score for the individual frames of the video based on the velocity and the acceleration of the content of the individual frames;
selecting one or more of the individual frames of the video based on the determined scores;
identifying, for individual selected frames, a corresponding video scene, the corresponding video scene comprising a first amount of the video occurring before the selected frame and a second amount of the video occurring after the selected frame; and
selecting one or more of the identified video scenes for inclusion in the video summary; and

the hardware processor.

9. The system of claim 8 , wherein the computer instructions are further configured to, when executed, cause the hardware processor to perform further steps comprising:
receiving a request for the video summary from a user; and
generating the video summary including the selected video scenes.

10. The system of claim 9 , wherein generating the video summary comprises concatenating the selected video scenes.

11. The system of claim 8 , wherein determining the velocity of the content of the individual frames of the video comprises:
determining a magnitude of one or more motion vectors in the individual frames; and
determining the velocity of the content of the individual frames based on the magnitude of the one or more motion vectors in the individual frames.

12. The system of claim 8 , wherein determining the acceleration of the content of the individual frames of the video comprises:
determining the velocities of the content of two adjacent individual frames; and
determining the acceleration of the content of the individual frames based on the difference between the velocities of the content of the two adjacent individual frames.

13. The system of claim 12 , wherein the computer instructions are further configured to, when executed, cause the hardware processor to perform further steps comprising:
aggregating a threshold number of the accelerations of the content of the individual frames in a sequence of frames; and
determining a single acceleration for the individual frames in the sequence of frames based on the aggregated accelerations.

14. The system of claim 8 , wherein the computer instructions are further configured to, when executed, cause the hardware processor to perform further steps comprising:
ranking the identified video scenes based on a likelihood that the identified video scenes will be of interest to a user;
wherein the one or more of the identified video scenes are selected for inclusion in the video summary based on the ranking.

15. A non-transitory computer-readable storage medium storing executable computer instructions configured to, when executed by a processor, cause the processor to perform steps comprising:
accessing a video, the video including a plurality of frames;
determining a velocity of content of individual frames of the video;
determining an acceleration of the content of the individual frames of the video based on at least two of the velocities of the content of the individual frames;
determining a score for the individual frames of the video based on the velocity and the acceleration of the content of the individual frames;
selecting one or more of the individual frames of the video based on the determined scores;
identifying, for individual selected frames, a corresponding video scene, the corresponding video scene comprising a first amount of the video occurring before the selected frame and a second amount of the video occurring after the selected frame; and
selecting one or more of the identified video scenes for inclusion in a video summary.

16. The computer-readable storage medium of claim 15 , wherein the computer instructions are further configured to, when executed, cause the processor to perform further steps comprising:
receiving a request for the video summary from a user; and
generating the video summary including the selected video scenes.

17. The computer-readable storage medium of claim 16 , wherein generating the video summary comprises concatenating the selected video scenes.

18. The computer-readable storage medium of claim 15 , wherein determining the velocity of the content of the individual frames of the video comprises:
determining a magnitude of one or more motion vectors in the individual frames; and

Assignees

• Gopro Inc

Inventors

• Adsumilli Balineedu Chowdary
• Lustig Ryan
• Newman David A

Classifications

• H04N9/8205

  involving the multiplexing of an additional signal and the colour video signal · CPC title

• G11B27/28

  by using information signals recorded by the same method as the main recording {(G11B27/22 takes precedence)} · CPC title

• G10L25/54

  for retrieval · CPC title

• H04N5/91

  Television signal processing therefor · CPC title

• G11B27/10 Primary

  Indexing; Addressing; Timing or synchronising; Measuring tape travel · CPC title