Scalable visual search system simplifying access to network and device functionality

Patent metadata

Field	Value
Publication number	US-9600584-B2
Application number	US-201414465312-A
Country	US
Kind code	B2
Filing date	Aug 21, 2014
Priority date	Nov 4, 2005
Publication date	Mar 21, 2017
Grant date	Mar 21, 2017

Abstract

In one embodiment, an indication of information desired by a user is received, and a list of candidates for the desired information is provided for presentation on a mobile device of the user.

First claim

What is claimed is:

1. A method comprising:
receiving in real time at a mobile device an indication of information desired by a user once the mobile device is pointed to one or more objects in a real world by the user, wherein the indication of the desired information comprises live visual media content presenting the one or more objects therein, meta-information providing one or more properties of the media content, and one or more characteristics associated with at least one of the mobile device and the user;
causing, at least in part, providing candidates for the desired information for presentation on the mobile device based, at least in part, on at least one of the live visual media content and the meta-information;
determining a set of properties for the candidates;
adjusting, based on the determined set of properties for the candidates, an access to the candidates for the information desired by the user; and
creating one or more first links automatically based on the live visual media content, if no matching data is found in a database,
wherein the candidates are concurrently presented with the live visual media content on one screen to the user for selection in real time without requiring the user to make any clicks from the time the user starts pointing the mobile device to the objects.

2. The method of claim 1 , further comprising:
searching the database for data matching the indication of the desired information for the candidates; and
constraining a scope of a search based on at least one of the media content and the meta-information.

3. The method of claim 2 , further comprising:
identifying the one or more objects based upon one or more recognition parameters,
wherein the search in the database for the candidates is further based upon one or more identified objects,
if matching data is found in the database, each candidate is a second link that maps the one or more objects, the one or more characteristics, or a combination thereof to a desired information item, and
the desired information item is a web site, a phone number, a message template, an application, an application state, a web page entry, a phone functionality, a keyword, a computer functionality, a document, a television program, a radio program, a music recording, or a combination thereof.

4. The method of claim 2 , further comprising:
associating each of the one or more first links with the live visual media content, the meta-information, or a combination thereof; and
storing the one or more first links in the database.

5. The method of claim 2 , further comprising:
creating an intermediate representation of the media content, the intermediate representation being used to search the database for data matching the indication of the desired information,
wherein the scope of the search is constrained by one or more other objects in the real world that are geographically near-by the one or more objects.

6. The method of claim 1 , wherein:
the candidates are overlaid on top of the live visual media content within a viewfinder of the mobile device; and
the candidates are presented to the user without requiring the user to make any clicks to capture an image of the objects, from the time the user starts pointing the mobile device to the objects.

7. The method of claim 6 , further comprising:
in response to a subsequent search query of the user, searching within the stored association and the at least one of the media content and the meta-information to assemble the candidates as search results.

8. The method of claim 6 , wherein:
at least one candidate has a title that includes a preview of the desired information; and
the method further comprises causing display of the title including the preview of the desired information when presenting the candidates to the user,
wherein the displaying provides user access to the preview of the desired information when the user points the mobile device to one or more other objects.

9. The method of claim 1 , further comprising:
determining that the user has not selected a candidate;
presenting a directory of information items to the user;
allowing the user to select an information item from the directory;
creating a link mapping the selected information item to the received indication of the desired information; and
storing the link in the database.

10. The method of claim 1 , further comprising selecting the candidates by searching a link database to identify one or more candidates associated with the indication of the desired information and generating the one or more candidates based upon one or more predefined rules.

11. The method of claim 1 , further comprising:
causing, at least in part, reception of an indication of a selection of the user in response to the providing of the candidates;
creating an association between the selection of the user and the indication of information desired by the user; and
causing, at least in part, storage of the association and at least one of the media content and the meta-information,
wherein the media content is produced by the mobile device and comprises one or more digital photographs, one or more videos, one or more audio recordings, or a combination thereof, and
the meta-information comprises one or more characteristics of the media content, one or more environmental characteristics associated with the mobile device, one or more geographic characteristics associated with the mobile device, one or more personal characteristics associated with the user, one or more online behavior characteristics of the user, or a combination thereof.

12. The method of claim 1 , further comprising:
causing, at least in part, starting of a timer when the mobile device is pointed to one or more objects in a real world by the user; and
causing, at least in part, providing the candidates for the desired information based on regular or irregular intervals based on the timer.

13. An apparatus comprising:
at least one processor; and at least one memory including computer program code,
the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to perform at least the following,
receive in real time at a mobile device an indication of information desired by a user once the mobile device is pointed to one or more objects in a real world by the user, wherein the indication of the desired information comprises live visual media content presenting the one or more objects therein, meta-information providing one or more properties of the media content, and one or more characteristics associated with at least one of the mobile device and the user;
cause, at least in part, providing candidates for the desired information for presentation on the mobile device based, at least in part, on at least one of the live visual media content and the meta-information;
determine a set of properties for the candidates;
adjust, based on the determined set of properties for the candidates, an access to the candidates for the information desired by the user; and
create one or more links automatically based on the live visual media content, if no matching data is found in a database,
wherein the candidates are concurrently presented with the live visual media content on one screen to the user for selection in real time without requiring the user to make any clicks from the time the user starts pointing the mobile device to the objects.

14. The apparatus of claim 13 , wherein the apparatus is further caused to select the candidates by searching a link database to identify one or more candidates associated with the

Assignees

• Nokia Technologies Oy

Inventors

• Schloter Philipp

Classifications

• H04W4/16

  Communication-related supplementary services, e.g. call-transfer or call-hold · CPC title

• G06F17/30867 Primary

  Physics · mapped topic

• G06F17/30864

  Physics · mapped topic

• H04L65/4084

  Electricity · mapped topic

• H04W4/02

  Services making use of location information · CPC title