Automatically generating theme-based folders by clustering media items in a semantic space

Patent metadata

Field	Value
Publication number	US-11030257-B2
Application number	US-201916417232-A
Country	US
Kind code	B2
Filing date	May 20, 2019
Priority date	May 20, 2019
Publication date	Jun 8, 2021
Grant date	Jun 8, 2021

Abstract

The present disclosure relates to systems, methods, and non-transitory computer readable media for clustering media items in a semantic space to generate theme-based folders that organize media items by content theme. In particular, the disclosed systems can access media items that are stored in an original folder structure. The disclosed systems can generate content-based tags for each media item in a collection of media items. Based on the generated tags, the disclosed systems can map the collection of media items to a semantic space and cluster the collection of media items. The disclosed systems determine themes for the clusters based on the generated tags. The disclosed systems can present a media item navigation graphical user interface comprising the collection of media items organized by themes. The disclosed system can present the media item navigation graphical user interface without altering the original folder structure.

First claim

What is claimed is:

1. A non-transitory computer readable medium for generating a customized user interface for navigating files, the non-transitory computer readable medium comprising instructions that, when executed by at least one processor, cause a computing device to:
generate tags and confidence scores for media items in a collection of media items, wherein the confidence scores indicate how strongly a given tag corresponds to a given media item;
map the collection of media items to a semantic space based on the tags;
cluster the collection of media items in the semantic space to create clusters;
determine themes for the clusters based on the tags of the media items in the clusters;
generate a reverse index that maps the tags to associated media items with associated confidence scores;
generate a media item navigation graphical user interface comprising the collection of media items organized by the themes and a search query element;
receive a search query via the search query element;
based on determining that the search query is not in the reverse index, determine a nearest term to the search query; and
provide media items associated with the nearest term based on corresponding confidence scores.

2. The non-transitory computer readable medium of claim 1 , wherein the instructions, when executed by the at least one processor, cause the computing device to generate the tags for the media items in the collection of media items by:
generating, utilizing a neural network, a feature map for a media item; and
identifying, utilizing the neural network, one or more tags for the media item based on the feature map.

3. The non-transitory computer readable medium of claim 1 , further comprising instructions, that when executed by the at least one processor, cause the computing device to generate confidence scores for each tag associated with a media item.

4. The non-transitory computer readable medium of claim 3 , wherein the instructions, when executed by the at least one processor, cause the computing device to map the collection of media items to the semantic space by generating semantic feature vectors for the media items in the collection of media items.

5. The non-transitory computer readable medium of claim 4 , wherein generating the semantic feature vectors comprises, for a given media item, converting one or more tags associated with the given media item into a semantic feature vector using a word to vector algorithm.

6. The non-transitory computer readable medium of claim 5 , wherein the instructions, when executed by the at least one processor, cause the computing device to cluster the collection of media items in the semantic space by using K-means clustering on the semantic feature vectors.

7. The non-transitory computer readable medium of claim 5 , wherein the instructions, when executed by the at least one processor, cause the computing device to cluster the collection of media items in the semantic space by processing the collection of media items using Latent Dirichlet Allocation.

8. The non-transitory computer readable medium of claim 5 , wherein the instructions, when executed by the at least one processor, cause the computing device to cluster the collection of media items in the semantic space by:
identifying cluster centers for the clusters in the semantic space;
determining distances between the cluster centers and the semantic feature vectors;
determining that a distance between a cluster center and a set of semantic feature vectors falls below a threshold; and
assigning media items associated with set of semantic feature vectors to the cluster.

9. The non-transitory computer readable medium of claim 8 , further comprising instructions that, when executed by the at least one processor, cause the computing device to determine themes for the clusters by:
identifying cluster tags for media items associated with the set of semantic feature vectors in the cluster;
determining a number of times each tag of the cluster tags is in the cluster;
identifying a most frequently occurring tag in the cluster; and
associating a human-understandable name associated with the most frequently occurring tag as the theme.

10. A system comprising:
at least one processor; and
a computer readable storage medium storing instructions that, when executed by the at least one processor, cause the system to:
generate tags and confidence scores for media items in a collection of media items utilizing a neural network, wherein the confidence scores indicate how strongly a given tag corresponds to a given media item;
map the collection of media items to a semantic space based on the tags by generating semantic feature vectors for the media items of the collection of media items from the tags using a word to vector algorithm;
cluster the collection of media items in the semantic space to create clusters by grouping similar semantic feature vectors;
assign the media items of the collection of media items to the clusters;
determine themes for the clusters based on the tags of the media items assigned to the clusters;
generate a reverse index that maps the tags to associated media items with associated confidence scores;
generate a media item navigation graphical user interface comprising the collection of media items organized into folders corresponding to the clusters and titled with the themes and a search query element;
based on receiving a search query via the search query element, determine media items associated with the search query in the reverse index by:
determining if the search query is in the reverse index;
based on determining that the search query is not in the reverse index, determining a nearest term to the search query; and
providing the media items associated with the nearest term based on the associated confidence scores; and

provide, for display via the media item navigation graphical user interface, the media items associated with the search query.

11. The system of claim 10 , wherein the instructions, when executed by the at least one processor, cause the system to assign the media items of the collection of media items to the clusters by:
identifying cluster centroids in the semantic space for the clusters;
determining distances between the cluster centroids and the semantic feature vectors;
determining that a distance between a cluster centroid for a cluster and a set of semantic feature vectors falls below a distance threshold; and
assigning media items associated with the set of semantic feature vectors to the cluster.

12. The system of claim 10 , further comprising instructions that, when executed by the at least one processor, cause the system to determine themes for the clusters by:
identifying cluster tags for media items associated with a set of semantic feature vectors in the cluster;
determining a number of times each tag of the cluster tags is in the cluster;
identifying a most frequently occurring tag in the cluster; and
associating a human-understandable name associated with the most frequently occurring tag as the theme.

13. The system of claim 10 , further comprising instructions that, when executed by the at least one processor, cause the system to:
receive, via the search query element, a set of model media items, wherein the search query comprises the set of model media items; and
determine the media items associated with the search query by:
generating tags for the set of model media items; and
determining media items associated with the tags for the set of model media items.

14. The system of claim 10 , further comprising instructions, that w

Assignees

• Adobe Inc

Inventors

• Dahl Jonas
• Tumu Sudheer
• Kota Nithyanand
• Naware Mihir
• Dewan Maneesh
• Chhugani Jatin
• Mallya Ganesh Satish

Classifications

• G06F3/0483 Primary

  Interaction with page-structured environments, e.g. book metaphor · CPC title

• G06V20/35

  Categorising the entire scene, e.g. birthday party or wedding scene · CPC title

• G06V20/70

  Labelling scene content, e.g. deriving syntactic or semantic representations · CPC title

• G06V10/945

  User interactive design; Environments; Toolboxes · CPC title

• G06V10/82

  using neural networks · CPC title