Hierarchical segmentation and quality measurement for video editing

Patent metadata

Field	Value
Publication number	US-2016358628-A1
Application number	US-201615173465-A
Country	US
Kind code	A1
Filing date	Jun 3, 2016
Priority date	Jun 5, 2015
Publication date	Dec 8, 2016
Grant date	—

Abstract

Methods for organizing media data by automatically segmenting media data into hierarchical layers of scenes are described. The media data may include metadata and content having still image, video or audio data. The metadata may be content-based (e.g., differences between neighboring frames, exposure data, key frame identification data, motion data, or face detection data) or non-content-based (e.g., exposure, focus, location, time) and used to prioritize and/or classify portions of video. The metadata may be generated at the time of image capture or during post-processing. Prioritization information, such as a score for various portions of the image data may be based on the metadata and/or image data. Classification information such as the type or quality of a scene may be determined based on the metadata and/or image data. The classification and prioritization information may be metadata and may be used to organize the media data.

First claim

We claim:

1 . A method for organizing media data, comprising:
parsing video data into media items according to scene detection; clustering the media items based on content analysis of the media items; opening a media editing application; and displaying a workspace of the media editing application in which the media items are grouped according to the clustering.

2 . The method of claim 1 , further comprising:
recommending the media items based on content analysis of the media items.

3 . The method of claim 1 , further comprising: clustering scenes hierarchically into layers where a plurality of clusters have one or more similar characteristics.

4 . The method of claim 1 , further comprising:
assigning quality scores to the media items; and providing an indicator of the assigned quality scores in the media editing application.

5 . The method of claim 1 , wherein a quality score is based on operational parameters of a camera at a time a respective media item was captured.

6 . The method of claim 1 , wherein a quality score is based on operational parameters of the media editing application, wherein the media editing application was used to edit the media data.

7 . The method of claim 1 , wherein a quality score is based on a location at which a respective media item was captured.

8 . The method of claim 1 , wherein a quality score is based on a similarity of content of a respective media item to content of a reference media item.

9 . The method of claim 1 , wherein a quality score is based on a level of activity in content of a respective media item.

10 . A non-transitory computer readable medium comprising instructions that, when executed on the computer, cause the computer to at least:
parse video data into media items according to scene detection; cluster the media items based on content analysis of the media items; open a media editing application; and display a workspace of the media editing application in which the media items are grouped according to the clustering.

11 . The non-transitory computer readable medium of claim 10 , wherein the instructions further cause the computer to:
recommend the media items based on content analysis of the media items.

12 . The non-transitory computer readable medium of claim 10 , wherein the instructions further cause the computer to:
cluster scenes hierarchically into layers where a plurality of clusters have one or more similar characteristics.

13 . The non-transitory computer readable medium of claim 10 , wherein the instructions further cause the computer to:
assign quality scores to the media items; and provide an indicator of the assigned quality scores in the media editing application.

14 . The non-transitory computer readable medium of claim 10 , wherein a quality score is based on operational parameters of a camera at a time a respective media item was captured.

15 . The non-transitory computer readable medium of claim 10 , wherein a quality score is based on operational parameters of the media editing application, wherein the media editing application was used to edit the media data.

16 . The non-transitory computer readable medium of claim 10 , wherein a quality score is based on a location at which a respective media item was captured.

17 . The non-transitory computer readable medium of claim 10 , wherein a quality score is based on a similarity of content of a respective media item to content of a reference media item.

18 . The non-transitory computer readable medium of claim 10 , wherein a quality score is based on a level of activity in content of a respective media item.

19 . A system for organizing media data, comprising a processor and a memory, the memory including instructions that, when executed on the processor, caused the system to at least:
parse video data into media items according to scene detection; cluster the media items based on content analysis of the media items; open a media editing application; and display a workspace of the media editing application in which the media items are grouped according to the clustering.

20 . The system of claim 19 , wherein the memory further comprises instructions to:
recommend the media items based on content analysis of the media items.

21 . The system of claim 19 , wherein the memory further comprises instructions to:
cluster scenes hierarchically into layers where a plurality of clusters have one or more similar characteristics.

22 . The system of claim 19 , wherein the memory further comprises instructions to:
assign quality scores to the media items; and provide an indicator of the assigned quality scores in the media editing application.

23 . The system of claim 19 , wherein a quality score is based on operational parameters of a camera at a time a respective media item was captured.

24 . The system of claim 19 , wherein a quality score is based on operational parameters of the media editing application, wherein the media editing application was used to edit the media data.

25 . The system of claim 19 , wherein a quality score is based on a location at which a respective media item was captured.

26 . The system of claim 19 , wherein a quality score is based on a similarity of content of a respective media item to content of a reference media item.

27 . The system of claim 19 , wherein a quality score is based on a level of activity in content of a respective media item.

Assignees

• Apple Inc

Inventors

• Liu Shujie
• Zheng Yunfei
• Zhou Xiaosong
• Wu Hsi-Jung
• Zhang Ke
• Keinath Albert E
• Chung Chris Y

Classifications

• G11B27/031 Primary

  Electronic editing of digitised analogue information signals, e.g. audio or video signals · CPC title

• G06K9/00765

  Physics · mapped topic

• G06K9/00718

  Physics · mapped topic

• G06K9/00751

  Physics · mapped topic

• G06V20/41

  Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items (segmenting video sequences G06V20/49) · CPC title