Automatic discovery of popular landmarks

Patent metadata

Field	Value
Publication number	US-9483500-B2
Application number	US-201514680000-A
Country	US
Kind code	B2
Filing date	Apr 6, 2015
Priority date	May 12, 2008
Publication date	Nov 1, 2016
Grant date	Nov 1, 2016

Abstract

In one embodiment the present invention is a method for populating and updating a database of images of landmarks including geo-clustering geo-tagged images according to geographic proximity to generate one or more geo-clusters, and visual-clustering the one or more geo-clusters according to image similarity to generate one or more visual clusters. In another embodiment, the present invention is a system for identifying landmarks from digital images, including the following components: a database of geo-tagged images; a landmark database; a geo-clustering module; and a visual clustering module. In other embodiments the present invention may be a method of enhancing user queries to retrieve images of landmarks, or a method of automatically tagging a new digital image with text labels.

First claim

What is claimed is:

1. A non-transitory computer-readable medium having instructions stored thereon to display landmark information, the instructions executable by one or more processors to cause the one or more processors to:
receive a user query;
identify one or more trigger words in the user query;
select one or more tags from a landmark database, the tags corresponding to the one or more trigger words;
supplement the user query with the one or more tags to generate a supplemented user query;
retrieve one or more visual clusters of landmark images for a landmark based on the supplemented user query; and
generate a user interface including information about the landmark and one or more summary lists of visual clusters, wherein each of the one or more summary lists includes, for each respective visual cluster of landmark images in the one or more summary lists, an identification of the respective visual cluster, and one or more of: a number of images in the respective visual cluster, a number of unique user identifiers associated with the respective visual cluster, a number of authors of images associated with the respective visual cluster, and a link to access images in the respective visual cluster.

2. A method to display landmark information
comprising:
receiving, by one or more computers, a user query;
identifying one or more trigger words in the user query;
selecting one or more tags from a landmark database, the tags corresponding to the one or more trigger words;
supplementing the user query with the one or more tags to generate a supplemented user query;
retrieving, based on the supplemented user query, one or more visual clusters of landmark images;
generating a user interface including the one or more visual clusters of retrieved landmark images; and
causing one or more summary lists of visual clusters of landmark images to be displayed on the user interface, wherein each of the one or more summary lists includes descriptive information and access information for each visual cluster in the one or more summary lists displayed.

3. The method of claim 2 , wherein the descriptive information comprises a number of images.

4. The method of claim 2 , wherein the descriptive information comprises a number of unique user identifiers.

5. The method of claim 2 , wherein the descriptive information comprises popularity information.

6. The method of claim 2 , wherein the descriptive information comprises an unique identifier for each visual cluster in the one or more summary lists displayed.

7. The method of claim 2 , wherein the access information for each respective visual cluster in the one or more summary lists displayed comprises a respective link to access images associated with the respective visual cluster.

8. The method of claim 2 , further comprising ordering the visual clusters in the summary lists according to popularity criteria.

9. The method of claim 2 , further comprising:
associating a landmark with the one or more visual clusters of retrieved landmark images;
determining whether there are additional visual clusters corresponding to the associated landmark to display; and
based upon a determination that there are no additional visual clusters to display, displaying information about the associated landmark.

10. The method of claim 9 , wherein displaying information about the associated landmark includes displaying one or more of: name of the associated landmark, location of the associated landmark, popularity, and number of images.

11. The method of claim 2 , further comprising providing a user interface element to receive user input corresponding to each visual cluster.

12. A system to display landmark information, comprising:
one or more processors; and
a non-transitory computer-readable medium having instructions stored thereon, the instructions executable by the one or more processors to:
receive a user query comprising one or more keywords;
select one or more corresponding tags from a landmark database corresponding to the one or more keywords;
supplement the user query with the one or more corresponding tags;
retrieve one or more landmark images based on the supplemented user query;
create visual clusters from the one or more retrieved landmark images; and
generate a user interface including one or more summary lists of the created visual clusters for the one or more retrieved landmark images, wherein each summary list corresponds to one of the one or more retrieved landmarks and includes descriptive information and access information for each visual cluster included in the summary list.

13. The system of claim 12 , wherein the descriptive information comprises one of a number of images and a number of unique user identifiers.

14. The system of claim 12 , wherein the descriptive information comprises popularity information.

15. The system of claim 12 , wherein the descriptive information comprises an identifier for one of the visual clusters included in the summary list.

16. The system of claim 12 , wherein the access information comprises, for each respective visual cluster included in the summary list, a respective link to access images in the respective visual cluster.

17. The system of claim 12 , wherein the instructions are further executable by the one or more processors to evaluate each cluster based on popularity criteria and to order each cluster based on the evaluation.

18. The system of claim 12 , wherein the instructions are further executable by the one or more processors to determine whether there are more visual clusters to display, and upon determination that there are no more visual clusters to display corresponding to the one of the one or more retrieved landmarks, display information about the one of the one or more retrieved landmarks.

19. The system of claim 18 , wherein the information about the one of the one or more retrieved landmarks includes one or more of: name of the one landmark, location of the one landmark, popularity of the one landmark, and number of images associated with the one of the one or more retrieved landmarks.

20. The system of claim 12 , wherein the instructions are further executable by the one or more processors to provide a checkbox to receive user input corresponding to each visual cluster.

Assignees

• Google Inc

Inventors

• Brucher Fernando A
• Buddemeier Ulrich
• Adam Hartwig
• Neven Hartmut

Classifications

• G06F16/5838 Primary

  using colour · CPC title

• G06F17/30241

  Physics · mapped topic

• G06K9/0063

  Physics · mapped topic

• G06F3/04842

  Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence) · CPC title

• G06T7/0022

  Physics · mapped topic