Associating classifications with images

Patent metadata

Field	Value
Publication number	US-9760803-B2
Application number	US-201314075424-A
Country	US
Kind code	B2
Filing date	Nov 8, 2013
Priority date	May 15, 2013
Publication date	Sep 12, 2017
Grant date	Sep 12, 2017

Abstract

One or more devices may receive multiple images; determine one or more classifications for each of the multiple images; generate a confidence score for each classification of the one or more classifications for each of the multiple images. The confidence score may include a value between two numbers and may identify the likelihood that that a particular classification, of the one or more classifications, correctly corresponds to a particular image, of the multiple images. The confidence score may be based on information, associated with a user, and pixel-based classification information. The one or more devices may sort the multiple images by the one or more classifications and by corresponding confidence scores to form sorted multiple images; and provide the sorted multiple images.

First claim

What is claimed is:

1. A method comprising:
receiving, by one or more devices, a plurality of images;
determining, by the one or more devices, one or more classifications for each of the plurality of images wherein the one or more classifications are structured and linked with at least one other classification, wherein the one or more classifications for each image of the plurality of images are based on data inputs including one or more of pixel-based information determined based on pixels of the image, metadata information determined based on metadata associated with the image, and user information of a user associated with the image;
generating, by the one or more devices, a confidence score for each classification of the one or more classifications for each of the plurality of images, the confidence score identifying a likelihood that a particular classification, of the one or more classifications, correctly corresponds to a particular image, of the plurality of images, the confidence score being based on the data inputs, wherein each data input is assigned a respective weight and wherein the confidence score is generated by combining weighted data values;
sorting, by the one or more devices, the plurality of images by the one or more classifications and by corresponding confidence scores to form a sorted plurality of images; and
providing, by the one or more devices, the sorted plurality of images.

2. The method of claim 1 , further comprising:
receiving a search query;
identifying images, of the plurality of images, based on the one or more classifications and the search query;
generating a ranking score for each of the identified images based on the confidence scores for the images; and
generating a ranked list of the identified images, the identified images being sorted in the ranked list based on the ranking scores,
wherein providing the sorted plurality of images includes providing the ranked list.

3. The method of claim 2 , where the ranking score is further based on at least one of a relevancy score and a quality score value,
the relevancy score being based on the search query,
the quality score value being based on a size, a resolution, or a measure of popularity of the images.

4. The method of claim 1 , where determining the one or more classifications is based on an object in respective image, of the plurality of images, that identifies a particular event and based on event-related user information that identifies the particular event.

5. The method of claim 1 , is based on wherein the metadata identifies a particular time and the user information identifies an event occurring at the particular time.

6. The method of claim 1 , wherein the metadata identifies a particular time, the user information identifies a particular event occurring at the particular time, and an object in a respective image, of the plurality of images, that identifies the particular event.

7. The method of claim 1 , where each classification of the one or more classifications is indicative of an event, an object, a time of day, an activity, a geographic location, a person, or a facial expression associated with a particular image of the plurality of images.

8. A system comprising:
a storage device; and
a hardware processor coupled to the storage device, the processor configured to:
store a data structure on the storage device, the data structure associating a plurality of images with one or more classifications for each image of the plurality of images, wherein the one or more classifications are structured and linked with at least one other classification, and wherein the one or more classifications for each image of the plurality of images are based on data inputs that include one or more of pixel-based information determined based on pixels of the image, metadata information determined based on metadata associated with the image, and user information of a user associated with the image, the data structure associating a confidence score for each classification of the one or more classifications for each image of the plurality of images, the confidence score being based on the data inputs, wherein each data input is assigned a respective weight and wherein the confidence score is generated by combining weighted data values, the data structure sorting the plurality of images by the one or more classifications and by corresponding confidence scores to form a sorted plurality of images;
receive a search query;
identify images, of the plurality of images, based on the one or more classifications stored in the data structure on the storage device, and the search query;
generate a ranking score for each image of the identified images based on the confidence scores for the images;
generate a ranked list of the identified images, the identified images being sorted in the ranked list based on the ranking scores; and
provide the ranked list to a user device to cause the user device to display the ranked list.

9. The system of claim 8 , wherein the one or more classifications are based on an object in a particular image, of the plurality of images, that identifies a particular event and wherein the user information identifies the particular event.

10. The system of claim 8 , the metadata identifies a particular time and the user information identifies an event occurring at the particular time.

11. The system of claim 8 , where each classification of the one or more classifications is indicative of an event, an object, a time of day, an activity, a geographic location, a person, or a facial expression associated with a particular image of the plurality of images.

12. A non-transitory computer-readable medium for storing instructions, the instructions comprising:
a plurality of instructions that, when executed by one or more processors associated with a device, causes the one or more processors to:
store a data structure, the data structure associating a plurality of images with one or more classifications-for each image of the plurality of images, wherein the one or more classifications are structured and linked with at least one other classification, and wherein the one or more classifications for each image of the plurality of images are based on data inputs including one or more of pixel-based information determined based on pixels of the image, metadata information determined based on metadata associated with the image, and user information of a user associated with the image, the data structure associating a confidence score for each classification of the one or more classifications for each image of the plurality of images, the confidence score identifying a likelihood that that a particular classification, of the one or more classifications, correctly corresponds to a particular image, of the plurality of images, the confidence score being based on the data inputs, wherein each data input is assigned a respective weight and wherein the confidence score is generated by combining weighted data values;
receive a search query;
identify images, of the plurality of images, based on the one or more classifications stored in the data structure, and the search query;
generate a ranking score for each image of the identified images based on the confidence scores for the images;
generate a ranked list of the identified images, the identified images being sorted in the ranked list based on the ranking scores; and
provide the ranked list to a user device to cause the user device to display the ranked list.

13. The non-transitory computer-readable medium of claim 12 , wherein the user information includes at least one of an e-mail account associated with the use

Assignees

• Google Inc

Inventors

• Kulick Matthew
• Adam Hartwig
• Lee Taehee
• Potetz Brian

Classifications

• G06F18/24

  Classification techniques · CPC title

• G06F16/51 Primary

  Indexing; Data structures therefor; Storage structures · CPC title

• G06F17/30247

  Physics · mapped topic

• G06K9/6267 Primary

  Physics · mapped topic

• G06F17/3028

  Physics · mapped topic