Automatic generation of people groups and image-based creations

Patent metadata

Field	Value
Publication number	US-11475616-B2
Application number	US-202017252707-A
Country	US
Kind code	B2
Filing date	Jun 25, 2020
Priority date	Jun 25, 2020
Publication date	Oct 18, 2022
Grant date	Oct 18, 2022

Abstract

Implementations described herein relate to methods, devices, and computer-readable media to generate and provide image-based creations. A computer-implemented method includes obtaining a plurality of episodes, each episode associated with a corresponding time period and including a respective set of images and person identifiers for each image. The method further includes forming a respective cluster for each episode that includes at least two person identifiers. The method further includes determining whether one or more person identifiers are included in less than a threshold number of clusters, and in response, removing the one or more person identifiers from the clusters that the one or more person identifiers that are included in. The method further includes merging identical clusters to obtain a plurality of people groups that each include two or more person identifiers and providing a user interface that includes an image-based creation based on a particular people group.

First claim

The invention claimed is:

1. A computer-implemented method, the method comprising:
obtaining a plurality of episodes from an image library, wherein each episode is associated with a corresponding time period, and includes a respective set of images and person identifiers for each image in the respective set of images;
forming a plurality of clusters that include a respective cluster for each episode, wherein each cluster includes at least two person identifiers;
determining a threshold cluster number based, at least in part, on a size of the image library;
determining whether one or more person identifiers are only included in a number of clusters of the plurality of clusters, wherein the number of clusters is less than the threshold cluster number;
if it is determined that the one or more person identifiers are only included in the number of clusters that is less than the threshold cluster number, removing the one or more person identifiers from the clusters included in the number of clusters;
after the removing, merging identical clusters to obtain a plurality of people groups, wherein each people group includes two or more person identifiers; and
providing a user interface that includes an image-based creation based on a particular people group of the plurality of people groups.

2. The computer-implemented method of claim 1 , wherein forming the plurality of clusters comprises, for each cluster:
mapping all person identifiers that appear in at least one image in the set of images of the episode to the cluster;
determining a most frequent person identifier for the episode, wherein the most frequent person identifier appears in more images of the set of images of the episode than all other identifiers of the person identifiers;
determining a number of images from the set of images of the episode that include the most frequent person identifier for the episode;
determining a threshold image number based on the number of images that include the most frequent person identifier for the episode; and
removing, from the cluster, person identifiers that only appear in a number of images of the cluster that is less than the threshold image number.

3. The computer-implemented method of claim 1 , further comprising:
determining a threshold episode number;
after the merging, determining that at least one people group only appears in a number of episodes of the plurality of episodes, wherein the number of episodes is less than the threshold episode number; and
in response to determination that at least one people group only appears in the number of episodes that is less than the threshold episode number, combining the at least one people group with one or more other people groups, wherein the one or more other people groups each include a subset of the person identifiers included in the at least one people group,
and wherein providing the user interface is performed after combining the at least one people group.

4. The computer-implemented method of claim 1 ,
wherein merging the identical clusters comprises associating, based on the clusters, a respective set of episodes to each of the plurality of people groups, and
wherein providing the image-based creation comprises selecting a subset of images for the image-based creation from the respective sets of images included in the episodes associated with the particular people group, wherein each image in the subset of images depicts persons that correspond to at least two of the two or more person identifiers included in the particular people group.

5. The computer-implemented method of claim 1 , wherein providing the user interface includes displaying at least one actionable user interface element in response to at least one selected from the group of:
detecting that an image that matches the particular people group was captured after a previous time that the at least one actionable user interface element was displayed; and
detecting that a current date at a time of providing the user interface matches a date associated with the particular people group,
and combinations thereof.

6. The computer-implemented method of claim 1 , wherein providing the image-based creation comprises:
selecting a subset of images for the image-based creation based on the particular people group, wherein each image in the subset of images depicts persons that correspond to at least two of the two or more person identifiers included in the particular people group; and
generating the image-based creation based on the subset of images.

7. The computer-implemented method of claim 6 , wherein each image in the subset of images depicts persons that correspond to each of the two or more person identifiers included in the particular people group.

8. The computer-implemented method of claim 6 , wherein each image in the subset of images depicts persons that correspond to each of the two or more person identifiers included in the particular people group and each image depicts one or more additional persons that are not included in any other people group.

9. The computer-implemented method of claim 6 , wherein the subset of images includes images from at least two episodes of the plurality of episodes.

10. The computer-implemented method of claim 9 , wherein the subset of images is selected to cause the subset to provide one or more selected from the group of:
location diversity in which images captured at a plurality of locations are included in the subset, and
visual diversity in which multiple different types of objects are depicted in the images, and combinations thereof.

11. A non-transitory computer-readable medium with instructions stored thereon that, when executed by a processor, cause the processor to perform operations comprising:
obtaining a plurality of episodes from an image library, wherein each episode is associated with a corresponding time period, and includes a respective set of images and person identifiers for each image in the respective set of images;
forming a plurality of clusters that include a respective cluster for each episode, wherein each cluster includes at least two person identifiers;
determining a threshold cluster number based, at least in part, on a size of the image library;
determining whether one or more person identifiers are only included in a number of clusters of the plurality of clusters, wherein the number of clusters is less than the threshold cluster number;
if it is determined that the one or more person identifiers are only included in the number of clusters that is less than the threshold cluster number, removing the one or more person identifiers from the clusters included in the number of clusters;
after the removing, merging identical clusters to obtain a plurality of people groups, wherein each people group includes two or more person identifiers; and
providing a user interface that includes an image-based creation based on a particular people group of the plurality of people groups.

12. The non-transitory computer-readable medium of claim 11 , wherein forming the respective cluster for each episode comprises:
mapping all person identifiers that appear in at least one image in the set of images of the episode to the respective cluster;
determining a most frequent person identifier for the episode, wherein the most frequent person identifier appears in more images of the set of images of the episode than all other identifiers of the person identifiers;
determining a number of images from the set of images of the episode that include the most frequent person identifier for the episode;
determining a threshold image number based on the number of images that include the most frequ

Assignees

• Google Llc

Inventors

• Lim Dominick
• Bohl Kristi
• Chang Jason
• Valmikinathan Vidya
• Lee Taehee
• Zhu Jeremy

Classifications

• G06F18/23211

  with adaptive number of clusters · CPC title

• G06F18/23

  Clustering techniques · CPC title

• G06V40/173 Primary

  face re-identification, e.g. recognising unknown faces across different face tracks · CPC title

• G06V20/30

  in albums, collections or shared content, e.g. social network photos or video · CPC title

• G06F16/7867

  using information manually generated, e.g. tags, keywords, comments, title and artist information, manually generated time, location and usage information, user ratings · CPC title