Scene and activity identification in video summary generation

Patent metadata

Field	Value
Publication number	US-10776629-B2
Application number	US-201816124607-A
Country	US
Kind code	B2
Filing date	Sep 7, 2018
Priority date	Jul 23, 2014
Publication date	Sep 15, 2020
Grant date	Sep 15, 2020

Abstract

Video and corresponding metadata is accessed. Events of interest within the video are identified based on the corresponding metadata, and best scenes are identified based on the identified events of interest. A video summary can be generated including one or more of the identified best scenes. The video summary can be generated using a video summary template with slots corresponding to video clips selected from among sets of candidate video clips. Best scenes can also be identified by receiving an indication of an event of interest within video from a user during the capture of the video. Metadata patterns representing activities identified within video clips can be identified within other videos, which can subsequently be associated with the identified activities.

First claim

What is claimed is:

1. A method for identifying video scenes, the method comprising:
accessing a video of an activity, the activity including an event at a moment within the video;
accessing metadata associated with the video, the metadata associated with the video enabling identification of a type of the activity and a type of the event;
identifying the type of the activity based on the metadata associated with the video including patterns associated with the type of the activity;
identifying the type of the event based on the metadata associated with the video including patterns associated with the type of the event;
identifying a scene of the video for the event, the scene including a length of the video that encompasses the moment and extends both before and after the moment within the video, wherein the length of the video is determined based on the type of the activity and the type of the event such that:
the length is a first length based on the type of the activity being of a first activity type and the type of the event being of a first event type;
the length is a second length based on the type of the activity being of the first activity type and the type of the event being of a second event type;
the length is a third length based on the type of the activity being of a second activity type and the type of the event being of the first event type; and
the length is a fourth length based on the type of the activity being of the second activity type and the type of the event being of the second event type,
wherein:
the first length is different from the second length, the third length, and the fourth length;
the second length is different from the third length and the fourth length;

and
the third length is different from the fourth length; and

outputting the scene of the video for playback.

2. The method of claim 1 , wherein outputting the scene of the video for playback includes generating a video summary comprising the scene.

3. The method of claim 1 , wherein the metadata is generated by a camera during the capture of the video.

4. The method of claim 3 , wherein the metadata comprises telemetry data describing a motion of the camera during the capture of the video.

5. The method of claim 3 , wherein the metadata comprises location data describing a location of the camera during the capture of the video.

6. The method of claim 3 , wherein the metadata comprises biometric data describing characteristics of a user of the camera during the capture of the video.

7. The method of claim 1 , wherein the metadata is accessed from an external entity after the capture of the video.

8. The method of claim 7 , wherein the metadata comprises environment data describing characteristics of an environment in which the video was captured.

9. The method of claim 1 , wherein position of the moment within the length is determined based on the type of the activity and the type of the event.

10. A system that identifies video scenes, the system comprising:
a non-transitory computer-readable storage medium storing instructions configured, when executed, to cause the system to perform:
accessing a video of an activity, the activity including an event at a moment within the video;
accessing metadata associated with the video, the metadata associated with the video enabling identification of a type of the activity and a type of the event;
identifying the type of the activity based on the metadata associated with the video including patterns associated with the type of the activity;
identifying the type of the event based on the metadata associated with the video including patterns associated with the type of the event;
identifying a scene of the video for the event, the scene including a length of the video that encompasses the moment and extends both before and after the moment within the video, wherein the length of the video is determined based on the type of the activity and the type of the event such that
the length is a first length based on the type of the activity being of a first activity type and the type of the event being of a first event type;
the length is a second length based on the type of the activity being of the first activity type and the type of the event being of a second event type;
the length is a third length based on the type of the activity being of a second activity type and the type of the event being of the first event type; and
the length is a fourth length based on the type of the activity being of the second activity type and the type of the event being of the second event type,
wherein:
the first length is different from the second length, the third length, and the fourth length;
the second length is different from the third length and the fourth length; and
the third length is different from the fourth length; and

outputting the scene of the video for playback; and

a processor configured to execute the instructions.

11. The system of claim 10 , wherein outputting the scene of the video for playback includes generating a video summary comprising the scene.

12. The system of claim 10 , wherein the metadata is generated by a camera during the capture of the video.

13. The system of claim 12 , wherein the metadata comprises telemetry data describing a motion of the camera during the capture of the video.

14. The system of claim 12 , wherein the metadata comprises location data describing a location of the camera during the capture of the video.

15. The system of claim 12 , wherein the metadata comprises biometric data describing characteristics of a user of the camera during the capture of the video.

16. The system of claim 10 , wherein the metadata is accessed from an external entity after the capture of the video.

17. The system of claim 16 , wherein the metadata comprises environment data describing characteristics of an environment in which the video was captured.

18. The system of claim 10 , wherein position of the moment within the length is determined based on the type of the activity and the type of the event.

Assignees

• Gopro Inc

Inventors

• Newman David A
• Silver Adam

Classifications

• H04N23/62

  Control of parameters via user interfaces · CPC title

• H04N23/60

  Control of cameras or camera modules · CPC title

• H04N5/77 Primary

  between a recording apparatus and a television camera · CPC title

• G06V20/47 Primary

  Detecting features for summarising video content · CPC title

• G06V40/179

  metadata assisted face recognition · CPC title