Methods of and systems for content search based on environment sampling

Patent metadata

Field	Value
Publication number	US-9736524-B2
Application number	US-201213345119-A
Country	US
Kind code	B2
Filing date	Jan 6, 2012
Priority date	Jan 6, 2011
Publication date	Aug 15, 2017
Grant date	Aug 15, 2017

Abstract

The present disclosure provides user interface methods of and systems for displaying at least one available action overlaid on an image, comprising displaying an image; selecting at least one action and assigning a ranking weight thereto based on at least one of (1) image content, (2) current device location, (3) location at which the image was taken, (4) date of capturing the image; (5) time of capturing the image; and (6) a user preference signature representing prior actions chosen by a user and content preferences learned about the user; and ranking the at least one action based on its assigned ranking weight.

First claim

What is claimed is:

1. A computer-implemented user interface method of displaying at least one available action overlaid on an image, the method comprising:
generating for display, a live image and a visual guide overlaid on the live image;
identifying an object of interest in the live image based on a proximity of the object of interest to the visual guide;
identifying, by a processor, without receiving user input, a first plurality of actions of different types from a second plurality of actions for subsequent selection by a user, the first plurality of actions being identified automatically based at least in part on the object of interest and at least one of (1) current device location, (2) location at which the live image was taken, (3) date of capturing the live image, (4) time of capturing the live image, and (5) a user preference signature representing prior actions selected by a user and content preferences learned about the user associated with particular times or locations at which the prior actions were selected by the user;
assigning a ranking weight to the first plurality of actions based on a non-textual portion of the identified object of interest;
ranking the first plurality of actions based on its assigned ranking weight; and
presenting the first plurality of actions to a user as selectable options.

2. The method of claim 1 , wherein the presenting of the first plurality of actions to the user as selectable options includes displaying the first plurality of actions in an order based on the ranking.

3. The method of claim 2 , further comprising updating the user preference signature to include information about the action chosen by the user from among the first plurality of actions.

4. The method of claim 1 , wherein the live image is an image of a portion of an environment surrounding the user.

5. The method of claim 1 , wherein the identifying a first plurality of actions and assigning a ranking weight thereto includes determining the ranking weight by a machine learning process.

6. The method of claim 1 , further comprising selecting the highest ranked action within the first plurality of actions in response to activation of a hardware camera button.

7. The method of claim 1 , further comprising analyzing the live image to learn about the object of interest.

8. The method of claim 7 , further comprising using at least one of the location of the device and the location at which the live image was taken to augment the analyzing the live image to learn about the object of interest.

9. The method of claim 8 , wherein the first plurality of actions includes an action to purchase an item corresponding to the object of interest from an online storefront corresponding to a physical storefront, if the device's location is proximate to the physical storefront.

10. The method of claim 7 , wherein the analyzing the live image to learn about the object of interest includes comparing the live image against a collection of at least one sample image to determine the object of interest.

11. The method of claim 10 , wherein the analyzing the live image to learn about the object of interest includes using optical character recognition to learn about textual image content.

12. The method of claim 10 , wherein the analyzing the live image to learn about the object of interest includes analyzing at least one partial image selected based on a proximity of the at least one partial image to a visual field of interest for the user.

13. The method of claim 1 , wherein the first plurality of actions are presented at a first point in time, further comprising:
storing the live image to a memory along with data about at least one of the location of the device, the date at which the live image was captured, and the time at which the live image was captured; and
presenting the first plurality of actions at a second point in time in an order based on the ranking when the user later acts upon the stored live image after the first point in time.

14. The method of claim 1 , the method further comprising:
subsequent to identifying the first plurality of actions, identifying a second object in the live image, wherein the second object is farther from the visual guide than the object of interest;
identifying an alternate action associated with the second object; and
presenting the alternate action to a user as a selectable option.

15. A system for displaying at least one available action overlaid on an image, the system comprising:
a memory device that stores instructions; and
a processor circuitry that executes the instructions and is configured to:
generate, for display, a live image and a visual guide overlaid on the live image;
identify an object of interest in the live image based on the proximity of the object of interest to the visual guide;
identify, without receiving user input, a first plurality of actions of different types from a second plurality of actions for subsequent selection by the user, the first plurality of actions being identified automatically based at least in part on the object of interest and at least one of (1) current device location, (2) location at which the live image was taken, (3) date of capturing the live image, (4) time of capturing the live image, and (5) a user preference signature representing prior actions selected by a user and content preferences learned about the user associated with particular times or locations at which prior actions were selected by the user;
assign a ranking weight to the first plurality of actions based on a non-textual portion of the identified object of interest;
rank the first plurality of actions based on its assigned ranking weight; and
present the first plurality of actions to a user as selectable options.

16. The system of claim 15 , wherein the processor circuitry is further configured to present the first plurality of actions to a user as selectable options by displaying the first plurality of actions in an order based on the ranking.

17. The system of claim 16 , the processor circuitry further being configured to cause the computer system to update the user preference signature to include information about the action chosen by the user from among the first plurality of actions.

18. The system of claim 15 , wherein the live image is an image of a portion of an environment surrounding the user.

19. The system of claim 15 , wherein the processor circuitry is further configured to determine the ranking weight by a machine learning process.

20. The system of claim 15 , the processor circuitry being further configured to cause the computer system to select the highest ranked action within the first plurality of actions in response to activation of a hardware camera button.

21. The system of claim 15 , the processor circuitry being further configured to cause the computer system to analyze the live image to learn about the object of interest.

22. The system of claim 21 , the processor circuitry being further configured to cause the computer system to use at least one of the location of the device and the location at which the live image was taken to augment the analyzing of the live image to learn about the object of interest.

23. The system of claim 21 , wherein the processor circuitry is further configured to compare the live image against a collection of at least one sample image to determine the object of interest.

24. The system of claim 23 , wherein the processor cir

Assignees

• Aravamudan Murali
• Rajasekharan Ajit
• Veveo Inc

Inventors

• Aravamudan Murali
• Rajasekharan Ajit

Classifications

• H04N21/4223 Primary

  Cameras (H04N23/00 takes precedence) · CPC title

• H04N21/44218

  Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV programme (methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45) · CPC title

• H04N21/41407

  embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop (constructional details of equipment or arrangements specially adapted for portable computer application G06F1/1626; arrangements specially adapted for mobile receivers in broadcast systems H04H20/57) · CPC title

• H04N21/47815

  Electronic shopping (payment schemes, payment architectures or payment protocols for electronic shopping systems G06Q20/12) · CPC title

• H04N21/4532

  involving end-user characteristics, e.g. viewer profile, preferences (monitoring of user activities for profile generation for accessing a video database G06F16/739; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18) · CPC title