Annotating a collection of media content items

Patent metadata

Field	Value
Publication number	US-12056441-B2
Application number	US-202117479383-A
Country	US
Kind code	B2
Filing date	Sep 20, 2021
Priority date	Mar 30, 2018
Publication date	Aug 6, 2024
Grant date	Aug 6, 2024

Abstract

Various embodiments provide for systems, methods, and computer-readable storage media for annotating a collection of media items, such as digital images. According to some embodiments, an annotation system automatically determines one or more annotations for a plurality of media content items, and generates a collection of media content items that associates the determined annotations with the plurality of media content items. Depending on the embodiment, annotations that may be determined for the plurality of media content (and associated with the collection for the media content items) can include, without limitation, a caption, a geographic location, a category, a novelty measurement, an event, and a highlight media content item representing the collection.

First claim

What is claimed is:

1. A method comprising:
determining, by one or more processors, a set of scores for a plurality of media content items by determining an individual score for an individual media content item in the plurality of media content items;
selecting, by the one or more processors, a set of highlight media content items from the plurality of media content items based on the set of scores;
determining, by the one or more processors, an individual caption for the plurality of media content items, the determining of the individual caption comprising:
extracting, from the plurality of media content items, a set of captions associated with one or more individual media content items in the plurality of media content items;
determining a set of caption scores for the set of captions by assigning a select caption score for each select caption in the set of captions, at least one caption in the set of captions being assigned an individual caption score based on an independence of the at least one caption within the plurality of media content items by:
determining a distribution of coverage scores, the determining of the distribution of coverage scores comprising assigning, for each individual media content item in the plurality of media content items associated with a given caption that includes the at least one caption, a coverage score that measures how much of the given caption is covered by the at least one caption; and
determining the independence of the at least one caption based on the distribution of coverage scores, the independence indicating how sufficient the at least one caption is in describing a majority of the media content items in the plurality of media content items, the at least one caption comprising a phrase;

determining a ranking for the set of captions based on the set of caption scores; and
selecting, from the set of captions, the individual caption based on the ranking;

generating, by the one or more processors, a collection of media content items that comprises the plurality of media content items, the individual caption, and collection annotation data, the collection annotation data associating the collection of media content items with the set of highlight media content items; and
providing, by the one or more processors, the collection of media content items to a client device for access by a user at the client device, one or more media content items from the set of highlight media content items being used to generate a graphical tile on the client device, the graphical tile being used to graphically represent the collection of media content items on the client device.

2. The method of claim 1 , wherein the individual caption score is a first caption score, and wherein at least another caption in the set of captions is assigned a second caption score based on at least one of:
a popularity of the select caption within the plurality of media content items; or
a uniqueness of the select caption within the plurality of media content items.

3. The method of claim 1 , comprising:
identifying, by the one or more processors, the plurality of media content items by grouping specific media content items based on at least one of proximity of geographic locations associated with the specific media content items, proximity of times associated with the specific media content items, topics associated with the specific media content items, media sources associated with the specific media content items, or media types associated with the specific media content items.

4. The method of claim 1 , comprising:
determining, by the one or more processors, whether the plurality of media content items is associated with a periodic event based on at least one of:
a similarity of at least one caption, in a set of captions extracted from the plurality of media content items, to a given caption of one or more other media content items associated with the periodic event; or
a similarity of at least one visual label, in a set of visual labels identified for the plurality of media content items, to a given visual label of the one or more other media content items associated with the periodic event;

the collection annotation data at least associating the collection of media content items with the periodic event in response to determining that the plurality of media content items is associated with the periodic event.

5. The method of claim 1 , comprising:
determining, by the one or more processors, whether the plurality of media content items is associated with an ongoing event based on a trend of media content items being added to the plurality of media content items over a period of time, the collection annotation data at least associating the collection of media content items with the ongoing event in response to determining that the plurality of media content items is associated with the ongoing event.

6. The method of claim 1 , comprising:
determining, by the one or more processors, whether the collection of media content items is associated with a concluded event, the collection annotation data at least associating the collection of media content items with the concluded event in response to determining that the plurality of media content items is associated with the concluded event.

7. The method of claim 1 , wherein the assigning of the select caption score for the select caption is based on a popularity of the select caption within the plurality of media content items, the popularity being determined based on at least one of:
a number of media content items in the plurality of media content items that are associated with the select caption; or
a number of users providing one or more media content items in the plurality of media content items who have used the select caption with respect to media content items not in the plurality of media content items.

8. A system comprising:
one or more processors; and
one or more machine-readable mediums storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
determining a set of scores for a plurality of media content items by determining an individual score for an individual media content item in the plurality of media content items;
selecting a set of highlight media content items from the plurality of media content items based on the set of scores;
determining an individual caption for the plurality of media content items, the determining of the individual caption comprising:
extracting, from the plurality of media content items, a set of captions associated with one or more individual media content items in the plurality of media content items;
determining a set of caption scores for the set of captions by assigning a select caption score for each select caption in the set of captions, at least one caption in the set of captions being assigned an individual caption score based on an independence of the at least one caption within the plurality of media content items, the independence indicating how sufficient the at least one caption is in describing a majority of the media content items in the plurality of media content items by:
determining a distribution of coverage scores, the determining of the distribution of coverage scores comprising assigning, for each individual media content item in the plurality of media content items associated with a given caption that includes the at least one caption, a coverage score that measures how much of the given caption is covered by the at least one caption; and
determining the independence of the at least one caption based on the distribution of coverage scores, the at least one caption comprising a phrase;

determining a ranking for the set of capti

Assignees

• Snap Inc

Inventors

• Al Majid Newar Husam
• Dakka Wisam
• Giovannini Donald
• Madeira Andre
• Mir Ghaderi Seyed Reza
• Lin Yaming
• Wu Yan
• Kunal Ranveer
• Damien Aymeric
• Daneshi Maryam
• Liu Yi

Classifications

• G06F16/24578

  using ranking · CPC title

• G06F16/489

  using time information · CPC title

• G06F16/487

  using geographical or spatial information, e.g. location · CPC title

• G06F16/41 Primary

  Indexing; Data structures therefor; Storage structures · CPC title

• G06F16/48

  Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually · CPC title