Data access based on content of image recorded by a mobile device

Patent metadata

Field	Value
Publication number	US-9483499-B2
Application number	US-201313849968-A
Country	US
Kind code	B2
Filing date	Mar 25, 2013
Priority date	May 15, 2008
Publication date	Nov 1, 2016
Grant date	Nov 1, 2016

Abstract

Embodiments of the invention are directed to using image data and contextual data to determine information about a scene, based on one or more previously obtained images. Contextual data, such location of image capture, can be used to determine previously obtained images related to the contextual data and other location-related information, such as billboard locations. With even low resolution devices, such as cell phone, image attributes, such as a histogram or optically recognized characters, can be compared between the previously obtained images and the newly captured image. Attributes matching within a predefined threshold indicate matching images. Information on the content of matching previously obtained images can be provided back to a user who captured the new image. User profile data can refine the content information. The content information can also be used as search terms for additional searching or other processing.

First claim

What is claimed as new and desired to be protected by Letters Patent is:

1. A method comprising:
receiving, by a processor, image data representing an image captured by a mobile device;
receiving, by the processor, a mobile device identifier of the mobile device;
accessing, by the processor and based on the mobile device identifier, profile information;
receiving, by the processor, location data indicating a location at which the image was captured by the mobile device;
receiving, by the processor, a time at which the image was captured by the mobile device;
determining, by the processor, image attribute information from the image data;
determining, by the processor, whether the image attribute information matches previously obtained attribute information within a predefined matching threshold, the previously obtained attribute information obtained from previously obtained image data representing a prior image captured within a predefined distance from the location and the time at which the image was captured by the mobile device, the previously obtained image data obtained with a second mobile device;
accessing, by the processor, image content information based on the location data if the image attribute information matches the previously obtained attribute information within the predefined matching threshold, the content information identifying content of the image that is associated with the previously obtained image data and associated with the location; and
selecting, by the processor, a portion of the image content information based on the profile information.

2. The method of claim 1 , wherein the image attribute information comprises a histogram of the image, and the previously obtained attribute information comprises a previously determined histogram of the prior image.

3. The method of claim 1 , wherein the image attribute information comprises a character determined from the image by optical character recognition, and the previously obtained attribute information comprises a previously determined character of the prior image.

4. The method of claim 1 , further comprising accessing, by the processor, location-related information based on the image content information if the image attribute information matches the previously obtained attribute information within the predefined matching threshold, wherein the location-related information is associated with the previously obtained image data and associated with the location.

5. The method of claim 1 , wherein the image content information comprises an identifier of merchant information.

6. The method of claim 1 , further comprising:
determining, by the processor, a search term associated with the image content information; and
performing, by the processor, a data search based on the search term.

7. The method of claim 1 , wherein the profile information includes user behavior information associated with a user of the mobile device.

8. The method of claim 1 , further comprising:
determining, by the processor, a priority level of each of a plurality of prior images captured within the predefined distance; and
selecting, by the processor, the previously obtained attribute information based on a determined priority level of the prior image represented by the previously obtained image data.

9. The method of claim 1 , further comprising returning, by the processor, the image content information to the mobile device.

10. A server comprising:
a processor;
a communication interface in communication with the processor and with a network; and
a memory in communication with the processor and storing data and instructions that cause the processor to perform a plurality of operations, including:
receiving image data representing an image captured by a mobile device in communication with the network;
receiving a mobile device identifier of the mobile device;
accessing, based on the mobile device identifier, profile information;
receiving location data indicating a location at which the image was captured by the mobile device;
receiving a time at which the image was captured by the mobile device;
determining image attribute information from the image data;
determining whether the image attribute information matches previously obtained attribute information within a predefined matching threshold, the previously obtained attribute information obtained from previously obtained image data representing a prior image captured within a predefined distance from the location and the time at which the image was captured by the mobile device, the previously obtained image data obtained with a second mobile device;
accessing image content information from a data store based on the location data if the image attribute information matches the previously obtained attribute information within the predefined matching threshold, the content information identifying content of the image that is associated with the previously obtained image data and associated with the location; and
selecting a portion of the image content information based on the profile information.

11. The server of claim 10 , wherein the image attribute information comprises at least one of the following:
a histogram of the image, and wherein the previously obtained attribute information comprises a previously determined histogram of the prior image; and
a character determined from the image by optical character recognition, and wherein the previously obtained attribute information comprises a previously determined character of the prior image.

12. The server of claim 10 , wherein the instructions further cause the processor to perform the operations of:
determining a search term associated with the image content information; and
performing a data search based on the search term.

13. A method comprising:
capturing an image with an imaging device;
determining, by a processor, a location of the imaging device when the image was captured;
determining, by the processor, a time at which the image was captured;
creating, by the processor, image data from the image;
receiving, by the processor, a mobile device identifier of the mobile device;
accessing, by the processor and based on the mobile device identifier, profile information;
determining, by the processor, image attribute information from the image data;
transmitting, by the processor, the image attribute information through a network interface to a data server, wherein the image attribute information is transmitted with a request to determine whether the image attribute information matches previously obtained attribute information within a predefined matching threshold, the previously obtained attribute information obtained from previously obtained image data representing a prior image captured within a predefined distance from the location and time at which the image was captured by the mobile device, the previously obtained image data obtained with a second mobile device;
receiving, by the processor, image content information based on the location data if the image attribute information matches the previously obtained attribute information within the predefined matching threshold,
the content information identifying content of the image that is associated with the previously obtained image data and associated with the location; and
selecting, by the processor, a portion of the image content information based on the profile information.

14. The method of claim 13 , further comprising transmitting, by the processor, a search request to a search engine, wherein the search request includes at least a portion of the image content information.

Assignees

• Yahoo Inc

Inventors

• Ramanujapuram Arun
• Slaney Malcolm

Classifications

• G06F16/5846 Primary

  using extracted text · CPC title

• G06F16/583 Primary

  using metadata automatically derived from the content · CPC title

• G06F17/30247

  Physics · mapped topic

• G06F17/30244 Primary

  Physics · mapped topic

• G06V10/462

  Salient features, e.g. scale invariant feature transforms [SIFT] · CPC title