Augmented reality recommendations

Patent metadata

Field	Value
Publication number	US-9646335-B2
Application number	US-201514863325-A
Country	US
Kind code	B2
Filing date	Sep 23, 2015
Priority date	May 13, 2013
Publication date	May 9, 2017
Grant date	May 9, 2017

Abstract

Various embodiments enable a computing device to perform tasks such as processing an image to recognize text or an object in an image to identify a particular product or related products associated with the text or object. In response to recognizing the text or the object as being associated with a product available for purchase from an electronic marketplace, one or more advertisements or product listings associated with the product can be displayed to the user. Accordingly, additional information for the associated product can be displayed, enabling the user to learn more about and purchase the product from the electronic marketplace through the portable computing device.

First claim

What is claimed is:

1. A computing device, comprising:
a processor;
a camera, the camera having a field of view;
a display element; and
memory including instructions that, when executed by the processor, enable the computing device to:
capture image information by the camera, the image information including a representation of text within a field of view of the camera, the image information being displayed on a display element of the computing device;
determine a confidence score indicating a level of confidence that the text is associated with one of a plurality of items available for purchase from an electronic marketplace;
determine that the confidence score is below a threshold;
analyze the representation of the text to identify a first portion of the representation of the text and a second portion of the representation of the text;
determine that the first portion of the representation of the text is associated with a item listing; and
display, in a graphical overlay over a portion of the image information corresponding to the field of view of the camera and concurrently being displayed on the display element, the item listing identifying an item available for purchase from the electronic marketplace.

2. The computing device of claim 1 , wherein the instructions when executed by the processor further enable the computing device to:
determine that the first portion of the representation of the text is associated with a first confidence score;
determine that the second portion of the representation of the text is associated with a second confidence score; and
determine that the first confidence score is above the threshold and the second confidence score is below the threshold.

3. The computing device of claim 1 , wherein the first portion of the text and the second portion of the text are portions of one word.

4. The computing device of claim 1 , wherein the instructions when executed by the processor further enable the computing device to:
receive a selection of the item listed in the item listing;
display information associated with the item; and
enable purchasing of the item from the electronic marketplace.

5. The computing device of claim 1 , wherein the instructions when executed by the processor to analyze the representation further enable the computing device to:
compare the representation of the text to information associated with items from at least one of previously recognized items for a user, user browser history, user purchase history, or stated user preferences.

6. The computing device of claim 1 , wherein the item includes at least one of an advertisement, or a related item available for purchase from the electronic marketplace, wherein the related item is displayed based at least in part on one of the related item having at least one keyword in common with the item, stated user preferences, user browser history, or user purchase history.

7. The computing device of claim 6 , wherein the instructions when executed by the processor further enable the computing device to:
display, on the display element, the related item on at least one of an overlay, an advertisement, or a fixed-position listing.

8. The computing device of claim 1 , wherein the instructions when executed by the processor further cause the computing device to:
provide a user with a related item in at least one of an email or a product listing displayed in a personalized webpage of the electronic marketplace associated with an account of the user.

9. A method, comprising:
capturing image information by a camera of a computing device, the image information including a representation of text within a field of view of the camera, the image information being displayed on a display element of the computing device;
determining a confidence score indicating a level of confidence that the text is associated with one of a plurality of items available for purchase from an electronic marketplace;
determining that the confidence score is below a threshold;
analyzing the representation of the text to identify a first portion of the representation of the text and a second portion of the representation of the text;
determining that the first portion of the representation of the text is associated with an item listing; and
displaying, in a graphical overlay over a portion of the image information corresponding to the field of view of the camera and concurrently being displayed on the display element, the item listing identifying an item available for purchase from the electronic marketplace.

10. The method of claim 9 , further comprising:
determining that the first portion of the representation of the text is associated with a first confidence score;
determining that the second portion of the representation of the text is associated with a second confidence score; and
determining that the first confidence score is above the threshold and the second confidence score is below the threshold, wherein the first portion of the text and the second portion of the text are portions of one word.

11. The method of claim 9 , further comprising:
receiving a selection of the item listed in the item listing;
displaying information associated with the item; and
enabling purchasing of the item from the electronic marketplace, wherein the item includes at least one of an advertisement, or a related item available for purchase from the electronic marketplace, wherein the related item is displayed based at least in part on one of the related item having at least one keyword in common with the item, stated user preferences, user browser history, or user purchase history.

12. The method of claim 9 , wherein analyzing the representation further includes:
comparing the representation of the text to information associated with items from at least one of previously recognized items for a user, user browser history, user purchase history, or stated user preferences.

13. The method of claim 9 , further includes:
displaying, on the display element, a related item on at least one of an overlay, an advertisement, or a fixed-position listing.

14. The method of claim 9 , wherein analyzing the representation further includes:
providing a user with related item in at least one of an email or a product listing displayed in a personalized webpage of the electronic marketplace associated with an account of the user.

15. The method of claim 10 , wherein the second confidence score is below the threshold due to one of glare on the camera, a show on the text, or an optical aberration.

16. A non-transitory computer-readable storage medium storing instructions that, when executed by at least one processor, cause a computing device to:
capture image information by a camera of a computing device, the image information including a representation of text within a field of view of the camera, the image information being displayed on a display element of the computing device;
determine a confidence score indicating a level of confidence that the text is associated with one of a plurality of items available for purchase from an electronic marketplace;
determine that the confidence score is below a threshold;
analyze the representation of the text to identify a first portion of the representation of the text and a second portion of the representation of the text;
determine that the first portion of the representation of the text is associated with a item listing; and
display, in a graphical overlay over a portion of the image information corresponding to the field of view of the camera and concurrently being displayed on

Assignees

• A9 Com Inc

Inventors

• Lin Xiaofan
• Dhua Arnab Sanat Kumar
• Gray Douglas Ryan
• Kumar Atul
• Lou Yu

Classifications

• G06T11/60

  Creating or editing images; Combining images with text · CPC title

• G06Q30/0269

  based on user profile or attribute · CPC title

• G06Q30/0623 Primary

  by investigating goods or services · CPC title

• G06F3/005

  Input arrangements through a video camera · CPC title

• G06Q30/0641 Primary

  utilising user interfaces specially adapted for shopping · CPC title