Scene and activity identification in video summary generation based on motion detected in a video

Patent metadata

Field	Value
Publication number	US-10192585-B1
Application number	US-201715467804-A
Country	US
Kind code	B1
Filing date	Mar 23, 2017
Priority date	Aug 20, 2014
Publication date	Jan 29, 2019
Grant date	Jan 29, 2019

Abstract

Video and corresponding metadata is accessed. Events of interest within the video are identified based on the corresponding metadata, and best scenes are identified based on the identified events of interest. In one example, best scenes are identified based on the motion values associated with frames or portions of a frame of a video. Motion values are determined for each frame and portions of the video including frames with the most motion are identified as best scenes. Best scenes may also be identified based on the motion profile of a video. The motion profile of a video is a measure of global or local motion within frames throughout the video. For example, best scenes are identified from portion of the video including steady global motion. A video summary can be generated including one or more of the identified best scenes.

First claim

What is claimed is:

1. A method for identifying video clips for inclusion in a video summary, the method comprising:
accessing a video, the video including video frames;
dividing individual video frames into tiles;
determining motion values of individual tiles of the individual video frames, the motion values characterizing movement of visual content within a corresponding tile between proximate video frames;
identifying tiles in the video frames having motion values that satisfy one or more motion criteria, wherein the one or more motion criteria includes a first motion criterion defining a first threshold velocity value, the individual tiles satisfying the first motion criterion based on the motion values of the individual tiles being greater than the first threshold velocity value;
generating a motion profile for the video based on the motion values determined for the individual tiles of the individual video frames, the motion profile characterizing, on a per video frame basis, an amount of the tiles within the individual video frames that satisfy the one or more motion criteria;
identifying one or more of the video frames based on the motion profile; and
generating the video summary, the video summary including the identified video frames.

2. The method of claim 1 , wherein the tiles are laid out in an array, individual ones of the tiles including multiple pixels.

3. The method of claim 1 , wherein the motion values includes a velocity value and an acceleration value.

4. The method of claim 1 , wherein the one or more motion criteria includes a second motion criterion defining a second threshold velocity value, the second threshold velocity value being less than the first threshold velocity value, the individual tiles satisfying the second motion criterion based on the motion values of the individual tiles being less than the second threshold velocity value.

5. The method of claim 1 , wherein the one or more motion criteria includes a second motion criterion defining a threshold acceleration value, the individual tiles satisfying the second motion criterion based on a comparison of the motion values of the individual tiles to the threshold acceleration value.

6. The method of claim 1 , wherein determining the motion values of the individual tiles comprises:
determining one or more motion vectors in the individual tiles; and
determining the motion values of the individual tiles based on the one or more motion vectors in the individual tiles.

7. The method of claim 1 , wherein determining the motion values of the individual tiles comprises:
determining one or more motion vectors in two corresponding tiles in two adjacent video frames; and
determining the motion values of the individual tiles based on the difference between the one or more motion vectors in the two corresponding tiles in the two adjacent video frames.

8. A method for identifying video clips for inclusion in a video summary, the method comprising:
accessing a video, the video including video frames;
dividing individual video frames into tiles;
determining motion values of individual tiles of the individual video frames, the motion values characterizing movement of visual content within a corresponding tile between proximate video frames;
identifying tiles in the video frames having motion values that satisfy one or more motion criteria, wherein the one or more motion criteria includes a band motion criterion defining a range of threshold velocity values or a range of threshold acceleration values, the individual tiles satisfying the band motion criterion based on the motion values of the individual tiles falling within the range of threshold velocity values or the range of threshold acceleration values;
generating a motion profile for the video based on the motion values determined for the individual tiles of the individual video frames, the motion profile characterizing, on a per video frame basis, an amount of the tiles within the individual video frames that satisfy the one or more motion criteria;
identifying one or more of the video frames based on the motion profile; and
generating the video summary, the video summary including the identified video frames.

9. The method of claim 8 , wherein determining the motion values of the individual tiles comprises:
determining one or more motion vectors in the individual tiles; and
determining the motion values of the individual tiles based on the one or more motion vectors in the individual tiles.

10. The method of claim 8 , wherein determining the motion values of the individual tiles comprises:
determining one or more motion vectors in two corresponding tiles in two adjacent video frames; and
determining the motion values of the individual tiles based on the difference between the one or more motion vectors in the two corresponding tiles in the two adjacent video frames.

11. A system for identifying video clips for inclusion in a video summary, the system comprising:
one or more physical processors configured by machine readable instructions to:
access a video, the video including video frames;
divide individual video frames into tiles;
determine motion values of individual tiles of the individual video frames, the motion values characterizing movement of visual content within a corresponding tile between proximate video frames;
identify tiles in the video frames having motion values that satisfy one or more motion criteria, wherein the one or more motion criteria includes a first motion criterion defining a first threshold velocity value, the individual tiles satisfying the first motion criterion based on the motion values of the individual tiles being greater than the first threshold velocity value;
generate a motion profile for the video based on the motion values determined for the individual tiles of the individual video frames, the motion profile characterizing, on a per video frame basis, an amount of the tiles within the individual video frames that satisfy the one or more motion criteria;
identify one or more of the video frames based on the motion profile; and
generate the video summary, the video summary including the identified video frames.

12. The system of claim 11 , wherein the tiles are laid out in an array, individual ones of the tiles including multiple pixels.

13. The system of claim 11 , wherein the motion values includes a velocity value and an acceleration value.

14. The system of claim 11 , wherein the one or more motion criteria includes a second motion criterion defining a second threshold velocity value, the second threshold velocity value being less than the first threshold velocity value, the individual tiles satisfying the second motion criterion based on the motion values of the individual tiles being less than the second threshold velocity value.

15. The system of claim 11 , wherein the one or more motion criteria includes a second motion criterion defining a threshold acceleration value, the individual tiles satisfying the second motion criterion based on a comparison of the motion values of the individual tiles to the threshold acceleration value.

16. The system of claim 11 , wherein determining the motion values of the individual tiles comprises:
determining one or more motion vectors in the individual tiles; and
determining the motion values of the individual tiles based on the one or more motion vectors in the individual tiles.

17. The system of claim 11 , wherein determining the motion values of the individual tiles comprises:
determining one or more motion vectors in two correspond

Assignees

• Gopro Inc

Inventors

• Adsumilli Balineedu Chowdary
• Lustig Ryan
• Newman David A

Classifications

• H04N9/8205

  involving the multiplexing of an additional signal and the colour video signal · CPC title

• G11B27/28

  by using information signals recorded by the same method as the main recording {(G11B27/22 takes precedence)} · CPC title

• G10L25/54

  for retrieval · CPC title

• H04N5/91

  Television signal processing therefor · CPC title

• G11B27/10 Primary

  Indexing; Addressing; Timing or synchronising; Measuring tape travel · CPC title