Dynamic presentation of targeted information in a mixed media reality recognition system

Patent metadata

Field	Value
Publication number	US-10007928-B2
Application number	US-201514804241-A
Country	US
Kind code	B2
Filing date	Jul 20, 2015
Priority date	Oct 1, 2004
Publication date	Jun 26, 2018
Grant date	Jun 26, 2018

Abstract

A context-aware targeted information delivery system comprises a mobile device, an MMR matching unit, a plurality of databases for user profiles, user context and advertising information, a plurality of comparison engines and a plurality of weight adjusters. The mobile device is coupled to deliver an image patch to the MMR matching unit which in turn performs recognition to produce recognize text. The recognized text is provided to a first and second comparison engines to produce relevant topics and relevant ads. The relevant topics and relevant ads are adjusted with information from a user context database including information such as location, date, time, and other information from a user profile. The third comparison engine compares the relevant topics and relevant ads to produce a set of final ads that are most related to the topics of interest for the user and delivered for display on to the mobile device.

First claim

What is claimed is:

1. A computer-implemented method comprising:
receiving an input image from a camera of a mobile device;
performing, with one or more processors, a visual search within a database of mixed media objects using the input image to identify and retrieve a mixed media object corresponding to the input image;
producing recognized text for a portion of the retrieved mixed media object corresponding to the input image and a document context including text surrounding the input image, each word of the recognized text having a word relevancy weight;
creating a user context based on user interaction data over time to identify user trends;
receiving targeted information;
generating, with the one or more processors, relevant targeted information based on the word relevancy weight of each word of the recognized text, the document context including the text surrounding the input image, the user context, and the targeted information; and
providing the relevant targeted information to the user, wherein the relevant targeted information includes an element that allows the user to perform an action.

2. The computer-implemented method of claim 1 , wherein receiving the targeted information comprises searching for the targeted information based on a location of the mobile device.

3. The computer-implemented method of claim 1 , wherein the targeted information includes an advertisement, the advertisement having a hyperlink selectable by the user to perform the action.

4. The computer-implemented method of claim 1 , wherein providing the relevant targeted information to the user comprises transmitting the relevant targeted information for presentation on the mobile device.

5. The computer-implemented method of claim 1 , wherein the document context further includes at least one from a group of: text appearing in the input image; previous targets a user hovered over; and hotspots surrounding the input image.

6. The computer-implemented method of claim 1 , further comprising:
determining the word relevancy weight for each word of the recognized text.

7. The computer-implemented method of claim 6 , wherein the word relevancy weight is based on a distance from the center of the input image.

8. A system comprising:
one or more processors; and
a memory storing instructions, which when executed, cause the one or more processors to:
receive an input image from a camera of a mobile device;
perform a visual search within a database of mixed media objects using the input image to identify and retrieve a mixed media object corresponding to the input image;
produce recognized text for a portion of the retrieved mixed media object corresponding to the input image and a document context including text surrounding the input image, each word of the recognized text having a word relevancy weight;
create a user context based on user interaction data over time to identify user trends;
receive targeted information;
generate relevant targeted information based on the word relevancy weight of each word of the recognized text, the document context including the text surrounding the input image, the user context, and the targeted information; and
provide the relevant targeted information to the user, wherein the relevant targeted information includes an element that allows the user to perform an action.

9. The system of claim 8 , wherein to receive the targeted information, the instructions cause the one or more processors to search for the targeted information based on a location of the mobile device.

10. The system of claim 8 , wherein the targeted information includes an advertisement, the advertisement having a hyperlink selectable by the user to perform the action.

11. The system of claim 8 , wherein to provide the relevant targeted information to the user, the instructions cause the one or more processors to transmit the relevant targeted information for presentation on the mobile device.

12. The system of claim 8 , wherein the document context further includes at least one from a group of: text appearing in the input image; previous targets a user hovered over; and hotspots surrounding the input image.

13. The system of claim 8 , wherein the instructions cause the one or more processors to determine the word relevancy weight for each word of the recognized text.

14. The system of claim 13 , wherein the word relevancy weight is based on a distance from the center of the input image.

15. A computer program product comprising a non-transitory computer useable medium including a computer readable program, wherein the computer readable program when executed on a computer causes the computer to:
receive an input image from a camera of a mobile device;
perform a visual search within a database of mixed media objects using the input image to identify and retrieve a mixed media object corresponding to the input image;
produce recognized text for a portion of the retrieved mixed media object corresponding to the input image and a document context including text surrounding the input image, each word of the recognized text having a word relevancy weight;
create a user context based on user interaction data over time to identify user trends;
receive targeted information;
generate relevant targeted information based on the word relevancy weight of each word of the recognized text, the document context including the text surrounding the input image, the user context, and the targeted information; and
provide the relevant targeted information to the user, wherein the relevant targeted information includes an element that allows the user to perform an action.

16. The computer program product of claim 15 , wherein the targeted information includes an advertisement, the advertisement having a hyperlink selectable by the user to perform the action.

17. The computer program product of claim 15 , wherein to provide the relevant targeted information to the user, the computer readable program causes the computer to transmit the relevant targeted information for presentation on the mobile device.

18. The computer program product of claim 15 , wherein the document context further includes at least one from a group of: text appearing in the input image; previous targets a user hovered over; and hotspots surrounding the input image.

19. The computer program product of claim 15 , wherein the computer readable program causes the computer to determine the word relevancy weight for each word of the recognized text.

20. The computer program product of claim 19 , wherein the word relevancy weight is based on a distance from the center of the input image.

Assignees

• Ricoh Co Ltd

Inventors

• Graham Jamey
• Hull Jonathan J

Classifications

• G06F16/40

  of multimedia data, e.g. slideshows comprising image and additional audio data (retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70) · CPC title

• G06V30/1918

  Fusion techniques, i.e. combining data from various sources, e.g. sensor fusion · CPC title

• G06V30/1916

  Validation; Performance evaluation · CPC title

• G06V30/19113

  Selection of pattern recognition techniques, e.g. of classifiers in a multi-classifier system · CPC title

• G06F18/254

  of classification results, e.g. of results related to same input data · CPC title