Assisted text input for computing devices

Patent metadata

Field	Value
Publication number	US-10038839-B2
Application number	US-201715611405-A
Country	US
Kind code	B2
Filing date	Jun 1, 2017
Priority date	Dec 11, 2013
Publication date	Jul 31, 2018
Grant date	Jul 31, 2018

Abstract

Various approaches provide for detecting and recognizing text to enable a user to perform various functions or tasks. For example, a user could point a camera at an object with text, in order to capture an image of that object. The camera can be integrated with a portable computing device that is capable of taking the image and processing the image (or providing the image for processing) to recognize, identify, and/or isolate the text in order to send the image of the object as well as recognized text to an application, function, or system, such as an electronic marketplace.

First claim

What is claimed is:

1. A computing system, comprising:
at least one processor; and
a memory device including instructions that, when executed by the at least one processor, cause the computing system to:
obtain image data of an object, the object including text displayed on a surface of the object;
analyze the image data to determine a product category associated with the object;
retrieve a first set of words associated with the product category;
generate a product listing using the first set of words;
process the image data to recognize the text displayed on the surface of the object to identify recognized text;
determine a second set of words from the recognized text;
display an interface that includes the product listing and a selectable list of a subset of the second set of words;
receive a selection of a word from the selectable list;
generate a text field with the selection; and
provide the image data and the word to an application.

2. The computing system of claim 1 , wherein the instructions, when executed to process the image data further cause the computing system to:
determine a foreground area and a background area of the image data, the foreground area including a representation of the object;
apply a color to the background area;
mask the background area; and
analyze the foreground area to recognize the text associated with the object.

3. The computing system of claim 1 , wherein the instructions, when executed further cause the computing system to:
provide an interface that includes user-selectable elements; and
receive a selection of a user-selectable element to obtain additional image data of the object.

4. The computing system of claim 3 , wherein the user-selectable elements include functionality to submit a product for purchase to an electronic marketplace, copy the selection of the word to a clipboard, a notepad, or a browser, or store image data or additional image data to a database.

5. The computing system of claim 1 , wherein the text field includes at least one of a title field, a search term field, a product description field, a keyword search field, a quantity field, or a quality field.

6. The computing system of claim 1 , wherein the instructions, when executed further cause the computing system to
determine a relevance score for words of the set of words; and
organize the set of words based at least in part on respective relevance scores.

7. A computing device, comprising:
a camera;
a display screen;
at least one processor; and
a memory device including instructions that, when executed by the at least one processor, cause the computing device to:
obtain image data of an object using the camera, the object including text displayed on a surface of the object;
match the image data to a stored item in a catalog of items, the stored item offered through an electronic marketplace and associated with item information;
generate an item listing that includes at least a portion of the item information and a representation of the item obtained from the electronic marketplace;
process the image data to recognize the text displayed on the surface of the object to identify recognized text;
determine a set of words from the recognized text;
display an interface on the display screen that includes the item listing and a selectable list of a subset of the set of words;
receive a selection of a word from the selectable list;
generate a text field with the selection; and
provide the image data and the word to an application.

8. The computing device of claim 7 , wherein the instructions, when executed further cause the computing device to:
adjust at least one of image sharpness of the image data, image color of the image data, image brightness of the image data, image file size of the image data, image orientation of the image data, image rectification of the image data, or image up scaling of the image data.

9. The computing device of claim 7 , wherein the instructions, when executed further cause the computing device to:
determine an additional set of words from additional image data of the object; and
display, on the interface, a subset of the additional set of words and the set of words.

10. The computing device of claim 7 , wherein the instructions, when executed further cause the computing device to:
obtain additional image data that includes an additional portion of the object; and
generate, based at least in part on the image data and the additional image data, one of a three-dimensional (3D) representation of the object or two or more views of the object.

11. The computing device of claim 7 , wherein the instructions, when executed further cause the computing device to:
determine that the subset of the set of words is associated with a particular text input field,
wherein the item listing includes the subset of the set of words in the particular text input field.

12. The computing device of claim 7 , wherein the instructions, when executed further cause the computing device to:
determine a foreground area and a background area of the image data, the foreground area including a representation of the object;
apply a color to the background area;
mask the background area; and
analyze the foreground area to recognize the text associated with the object.

13. A non-transitory computer readable storage medium storing one or more sequences of instructions executable by one or more processors to perform a set of operations comprising:
obtaining image data of an object, the object including text displayed on a surface of the object;
matching the image data to a stored item in a catalog of items, the stored item offered through an electronic marketplace and associated with item information;
generating an item listing that includes at least a portion of the item information and a representation of the item obtained from the electronic marketplace;
processing the image data to recognize the text displayed on the surface of the object to identify recognized text;
determining a set of words from the recognized text;
displaying an interface that includes the item listing and a selectable list of a subset of the set of words;
receiving a selection of a word from the selectable list;
generating a text field with the selection; and
providing the image data and the word to an application.

14. The non-transitory computer readable storage medium of claim 13 , further comprising instructions executed by the one or more processors to perform the set of operations of:
determining a foreground area and a background area of the image data, the foreground area including a representation of the object;
applying a color to the background area;
masking the background area; and
analyzing the foreground area to recognize the text associated with the object.

15. The non-transitory computer readable storage medium of claim 13 , further comprising instructions executed by the one or more processors to perform the set of operations of:
providing an interface that includes user-selectable elements; and
receiving a selection of a user-selectable element to obtain additional image data of the object, wherein the user-selectable elements include functionality to submit a product for purchase to the electronic marketplace, copy the selection of the word to a clipboard, a notepad, or a browser, or store image data or additional image data to a database.

16. The non-transitory computer readable storage medium of claim 13 , further comprising instructions executed by the one or more processors to perform the set of operati

Assignees

• A9 Com Inc
• A 9 Com Inc

Inventors

• Kraft Adam Wiggen
• Ma Kathy Wing Lam
• Lin Xiaofan
• Dhua Arnab Sanat Kumar
• Lou Yu

Classifications

• H04N23/64 Primary

  Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image · CPC title

• G06F40/166

  Editing, e.g. inserting or deleting · CPC title

• H04N1/00

  Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof · CPC title

• G06T2210/22

  Cropping · CPC title

• G06T7/194

  involving foreground-background segmentation · CPC title