Event-based image classification and scoring

Patent metadata

Field	Value
Publication number	US-10628680-B2
Application number	US-201715690104-A
Country	US
Kind code	B2
Filing date	Aug 29, 2017
Priority date	Aug 6, 2014
Publication date	Apr 21, 2020
Grant date	Apr 21, 2020

Abstract

Embodiments of the present invention analyze and score each image associated with a group to determine representative image or images for the group. Such analysis can include detecting objects shown in the images, determining the quality of the images, and/or contextually analyzing the images as a group. In some embodiments, each image in a group (e.g., an event) of images can be analyzed by one or more image analysis modules that calculate a score for the image based on a different image characteristic. A composite image score can then be calculated based on the various image characteristic scores to identify the image or images to be used as to represent the group.

First claim

What is claimed is:

1. A method comprising:
identifying a default scoring model trained to score images based on coefficients reflecting user preferences of multiple users;
receiving, from a client device, indications of user selections by a particular user indicating user preferences of images having a first visual image characteristic corresponding to a first scene type and images having a second visual image characteristic corresponding to a second scene type, the second scene type differing from the first scene type;
based on the indications of user selections, modifying the default scoring model to generate a personalized scoring model comprising adjusted coefficients for the particular user reflecting the user preferences of images having the first visual image characteristic and images having the second visual characteristic;
identifying, from among a plurality of images corresponding to an event, a first image having the first visual image characteristic and a second image having the second visual image characteristic;
determining a first composite image score for the first image based on the first visual image characteristic and a second composite image score for the second image based on the second visual characteristic according to the adjusted coefficients of the personalized scoring model; and
based on the personalized scoring model, the first composite image score for the first image score, and the second composite image score for the second image score, selecting, for display within a user interface of the client device, the first image as a first representative image corresponding to the first scene type for the event and the second image as a second representative image corresponding to the second scene type for the event.

2. The method of claim 1 , wherein:
receiving the indications of user selections indicating the user preferences of images having the first visual image characteristic corresponding to the first scene type comprises receiving indications of user preferences of images corresponding to a portrait scene type; and
receiving the indications of user selections indicating the user preferences of images having the second visual image characteristic corresponding to the second scene type comprises receiving indications of user preferences of images corresponding to a landscape scene type.

3. The method of claim 1 , wherein determining the first composite image score for the first image based on the first visual image characteristic according to the adjusted coefficients of the personalized scoring model comprises weighting the first composite image score for the first image according to the adjusted coefficients of the personalized scoring model.

4. The method of claim 1 , further comprising:
classifying the first image as corresponding to the first scene type based on visual image characteristics of the first image;
classifying the second image as corresponding to the second scene type based on visual image characteristics of the second image;
comparing the first composite image score for the first image to composite image scores for images corresponding to the first scene type from the plurality of images;
comparing the second composite image score for the second image to composite image scores for images corresponding to the second scene type from the plurality of images; and
based on comparing the first composite image score and the second composite image score to composite image scores for images corresponding to the first scene type and the second scene type from the plurality of images, selecting the first image as the first representative image corresponding to the first scene type and the second image as the second representative image corresponding to the second scene type.

5. The method of claim 4 , wherein:
classifying the first image comprises classifying the first image as corresponding to a portrait scene type based on detecting a person depicted in the first image; and
classifying the second image comprises classifying the second image as corresponding to a landscape scene type based on detecting geographic characteristics in the second image.

6. The method of claim 1 , wherein receiving the indications of user selections by the particular user indicating the user preferences of images comprises receiving a reordering by the particular user of an additional plurality of images corresponding to an additional event.

7. The method of claim 1 , wherein receiving the indications of user selections by the particular user indicating the user preferences of images comprises:
receiving a first indication via a social media service that the particular user likes at least one image having the first visual image characteristic; and
receiving a second indication via the social media service that the particular user likes at least one image having the second visual image characteristic.

8. The method of claim 1 , wherein selecting, for display within the user interface of the client device, the first image as the first representative image corresponding to the first scene type for the event and the second image as the second representative image corresponding to the second scene type for the event comprises causing the client device to present within the user interface:
the first representative image as a first thumbnail;
the second representative image as a second thumbnail; and
a selectable option associated with the first thumbnail or the second thumbnail that, upon selection, causes display of one or more of the plurality of images corresponding to the event.

9. A system comprising:
at least one processor; and
at least one non-transitory computer-readable storage medium comprising instructions that, when executed by the at least one processor, cause the system to:
identify a default scoring model trained to score images based on coefficients reflecting user preferences of multiple users;
receive, from a client device, indications of user selections by a particular user indicating user preferences of images having a first visual image characteristic corresponding to a first scene type and images having a second visual image characteristic corresponding to a second scene type, the second scene type differing from the first scene type;
based on the indications of user selections, modify the default scoring model to generate a personalized scoring model comprising adjusted coefficients for the particular user reflecting the user preferences of images having the first visual image characteristic and images having the second visual characteristic;
identify, from among a plurality of images corresponding to an event, a first image having the first visual image characteristic and a second image having the second visual image characteristic;
determine a first composite image score for the first image based on the first visual image characteristic and a second composite image score for the second image based on the second visual characteristic according to the adjusted coefficients of the personalized scoring model; and
based on the personalized scoring model, the first composite image score for the first image score, and the second composite image score for the second image score, select, for display within a user interface of the client device, the first image as a first representative image corresponding to the first scene type for the event and the second image as a second representative image corresponding to the second scene type for the event.

10. The system of claim 9 , further comprising instructions that, when executed by the at least one processor, cause the system to determine the first composite image score for the first image based on the first visual

Assignees

• Dropbox Inc

Inventors

• Babenko Boris
• Welinder Nils Peter

Classifications

• G06K9/00744 Primary

  Physics · mapped topic

• G06V20/46 Primary

  Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames · CPC title