Item recognition using context data

Patent metadata

Field	Value
Publication number	US-10043069-B1
Application number	US-201414196669-A
Country	US
Kind code	B1
Filing date	Mar 4, 2014
Priority date	Mar 4, 2014
Publication date	Aug 7, 2018
Grant date	Aug 7, 2018

Abstract

A system for recognizing objects and/or text in image data may use context data to perform object/text recognition. The system may also use context data when determining potential functions to execute in response to recognizing the object/text. Context data may be gathered based on device sensor data, user profile data such as the behavior of a user or the behavior of those in a user's social network, or other factors. Recognition processing and/or function selection may be configured to account for context data when operating to improve output results.

First claim

What is claimed is:

1. A computer-implemented method, comprising:
receiving social network data associated with a first user profile corresponding to a mobile computing device, the social network data including information related to one or more different user profiles linked, within a social network, to the first user profile;
receiving image data from the mobile computing device;
performing image processing on the image data to identify text in the image data;
recognizing actionable text in the identified text;
determining a plurality of potential user-executable functions, wherein the determining is based at least in part on the actionable text and wherein the plurality of potential user-executable functions includes at least one of creating a new contact entry based on the actionable text, adding information in the actionable text to an existing contact, showing a map based on the actionable text, dialing a telephone number included in the actionable text, or accessing a website included in the actionable text;
determining, based at least in part on the social network data, a primary user-executable function of the plurality of potential user-executable functions;
causing the mobile computing device to display an icon, the icon including at least two user-selectable portions including:
a first user-selectable portion that, when selected, causes execution of the primary user-executable function, and
a second user-selectable portion that, when selected, causes execution of an action to display additional user-executable functions of the plurality of potential user-executable functions;

receiving an indication of a user selection of the first user-selectable portion corresponding to the primary user-executable function; and
executing the primary user-executable function.

2. The computer-implemented method of claim 1 , further comprising recognizing an address in the actionable text, and wherein:
the prior operation of the mobile computing device by the user comprises viewing content related to a geographic region on the mobile computing device; and
the recognizing the address further comprises recognizing one or more characters in the text based on the geographic region.

3. The computer-implemented method of claim 1 , wherein the actionable text comprises at least one of a physical address, a telephone number, a website address, or an email address.

4. The computer-implemented method of claim 1 , wherein the mobile computing device is caused to display the icon such that the first user-selectable portion is larger than the second user-selectable portion.

5. The computer-implemented method of claim 1 , wherein the determining the plurality of potential user-executable functions is based at least in part on the social network data.

6. A computing system, comprising:
at least one processor; and
a computer-readable medium encoded with instructions which, when executed by the at least one processor, cause the computing system to:
to receive image data, the image data including text data;
to receive social network data associated with a first user profile corresponding to the computing system, the social network data including information related to one or more different user profiles linked, within a social network, to the first user profile;
to perform character recognition of the text data;
to determine a plurality of potential user-selectable functions based at least in part on recognized characters of the text data;
to determine, based at least in part on the social network data, a first user-selectable function of the plurality of potential user-selectable functions;
to cause display of an icon to the user, the icon including at least two user-selectable portions including:
a first user-selectable portion that, when selected, causes execution of the first user-selectable function; and
a second user-selectable portion that, when selected, causes execution of an action to cause display of additional user-selectable functions of the plurality of potential user-selectable functions.

7. The computing system of claim 6 , wherein the computer-readable medium is encoded with additional instructions which, when executed by the at least one processor, further cause the computing system to determine a relative priority for each of the plurality of potential user-selectable functions based at least in part on the social network data.

8. The computing system of claim 7 , wherein the computer-readable medium is encoded with additional instructions which, when executed by the at least one processor, further cause the computing system to determine the first user-selectable function based at least in part on the relative priority.

9. The computing system of claim 6 , wherein the image data includes second text data and wherein the computer-readable medium is encoded with additional instructions which, when executed by the at least one processor, further cause the computing system:
to perform character recognition of the second text data;
to determine at least one second potential user-selectable function based at least in part on recognized characters of the second text data; and
to determine a relative priority between each of the plurality of potential user-selectable functions and the at least one second potential user-selectable function based at least in part on the social network data.

10. The computing system of claim 6 , wherein the computer-readable medium is encoded with additional instructions which, when executed by the at least one processor, further cause the computing system to determine a category of the text data, and wherein determination of the first user-selectable function is based at least in part on a previously selected user function associated with the category of the text data.

11. The computing system of claim 10 , wherein the previously selected user function is a function selected by the user associated with the image data.

12. The computing system of claim 6 , wherein the computer-readable medium is encoded with additional instructions which, when executed by the at least one processor, further cause the computing system:
to receive the image data from a user device; and
to identify the plurality of potential user-selectable functions in a communication to the user device.

13. The computing system of claim 6 , further comprising:
a camera to capture the image data; and
an interface to display the icon.

14. The computer-implemented method of claim 6 , wherein the computer-readable medium is encoded with additional instructions which, when executed by the at least one processor, further cause the computing system to cause the display of the icon such that the first user-selectable portion is larger than the second user-selectable portion.

15. The computer-implemented method of claim 6 , wherein the computer-readable medium is encoded with additional instructions which, when executed by the at least one processor, further cause the computing system to determine the plurality of potential user-executable functions based at least in part on the social network data.

16. A non-transitory computer-readable storage medium encoded with instructions which, when executed by at least one processor of a computing system, cause the computing system:
to receive image data, the image data including text data;
to receive social network data associated with a first user profile corresponding to the computing system, the social network data including information related to one or more different user profiles linked, within a social network, to the first user prof

Assignees

• Amazon Tech Inc

Inventors

• Liu Yue
• Prateek Utkarsh
• Sikka Avnish
• Hart Matthew Daniel
• Mcconville Emilie Noelle
• Jahagirdar Sonjeev

Classifications

• G06V10/23

  based on positionally close patterns or neighbourhood relationships · CPC title

• G06V30/40 Primary

  Document-oriented image-based pattern recognition · CPC title

• G06V10/235

  based on user input or interaction · CPC title

• G06K9/00422 Primary

  Physics · mapped topic

• G06V30/142

  using hand-held instruments; Constructional details of the instruments · CPC title