Automated highlighting of identified text

Patent metadata

Field	Value
Publication number	US-9582913-B1
Application number	US-201314037275-A
Country	US
Kind code	B1
Filing date	Sep 25, 2013
Priority date	Sep 25, 2013
Publication date	Feb 28, 2017
Grant date	Feb 28, 2017

Abstract

Various embodiments enable a computing device to perform tasks such as highlighting words in an augmented reality view that are important to a user. For example, word lists can be generated and the user, by pointing a camera of a computing device at a volume of text, can cause words from the word list within the volume of text to be highlighted in a live field of view of the camera displayed thereon. Accordingly, users can quickly identify textual information that is meaningful to them in an Augmented Reality view to aid the user in sifting through real-world text.

First claim

What is claimed is:

1. A non-transitory computer-readable storage medium storing instructions that, when executed by at least one processor, cause a computing device to:
access one or more user identified words and three or more single-character symbols each specifying an individual preference with respect to the one or more user identified words;
capture, using a camera of a computing device, image information for a volume of text;
display, on a display element of the computing device, at least a portion of the image information as corresponding to a live field of view of the camera;
recognize, using an optical character recognition (OCR) engine, text of the volume of text in the captured image information;
compare words, from the one or more user identified words, to the recognized text;
identify, from the one or more user identified words, at least one word from the one or more user identified words in the recognized text;
display, in the live field of view of the camera, an overlay proximate each occurrence of the at least one word from the one or more user identified words, each overlay including a visual attribute causing the at least one word to be identified as a user identified word and at least one of the single-character symbols; and
arranging the three or more single-character symbols so that a group preference with respect to the one or more user identified words can be ascertained.

2. The non-transitory computer-readable storage medium of claim 1 , wherein the overlay further comprises at least one of a bounding box around the at least one word, a graphical representation of the at least one word in color, or highlighting the word with a color.

3. The non-transitory computer-readable storage medium of claim 2 , wherein the instructions that, when executed by the at least one processor, further cause the computing device to:
receive the one or more user identified words from a user; and
enable the user to assign a first overlay to a first set of words and a second overlay to the second set of words of the one or more user identified words.

4. The non-transitory computer-readable storage medium of claim 3 , wherein the overlay for the first set of words is different in appearance relative to the overlay for the second set of words.

5. The non-transitory computer-readable storage medium of claim 1 , wherein the overlay proximate each occurrence of the at least one word moves in corresponding motion with the at least one word on the display element in the live field of view of the camera.

6. The non-transitory computer-readable storage medium of claim 1 , wherein the instructions that, when executed by the at least one processor, further cause the computing device to:
identify a category associated with the recognized text; and
apply, based at least in part on the identified category, the one or more user identified words.

7. A computer-implemented method, comprising:
accessing one or more user identified words and three or more single-character symbols each specifying an individual preference with respect to the one or more user identified words;
capturing, using a camera of a computing device, image information including a volume of text;
displaying, on a display element of the computing device, at least a portion of the image information representing at least a portion of the volume of text;
causing at least a portion of the volume of text in the image information to be recognized;
causing one or more words from a set of words to be compared to the recognized text;
modifying, in response to identifying at least one word from the set of words in the recognized text, a display of the at least one identified word in the displayed image information including displaying at least one single-character symbol; and
arranging the three or more single-character symbols so that a group preference with respect to the one or more user identified words can be ascertained.

8. The computer-implemented method of claim 7 , wherein the image information is displayed as at least one of a live field of view of the camera or a real-time video captured by the camera.

9. The computer-implemented method of claim 8 , wherein modifying the display of the at least one identified word further includes displaying at least one of a bounding box around the at least one identified word, or displaying a graphical representation of the at least one identified word.

10. The computer-implemented method of claim 8 , wherein modifying the display of the at least one identified word includes displaying a graphical overlay at least one of above or adjacent the at least one word.

11. The computer-implemented method of claim 10 , wherein the graphical overlay at least one of above or adjacent the at least one word moves in corresponding motion with the at least one word on the display element in the live field of view of the camera.

12. The computer-implemented method of claim 10 , wherein the graphical overlay includes a visual attribute causing the at least one word to stand out relative to a remainder of words of the text displayed on the display element.

13. The computer-implemented method of claim 10 , wherein the graphical overlay further comprises at least one of a bounding box around the at least one identified word, a graphical representation of the at least one identified word in color, or color highlighting.

14. The computer-implemented method of claim 10 , further comprising:
receiving the set of words from a user; and
enabling the user to assign an overlay to the set of words.

15. The computer-implemented method of claim 7 , further comprising:
identifying a category associated with the recognized text; and
applying the set of words based at least in part on the identified category.

16. A computing device, comprising:
a processor;
a camera;
a display screen; and
memory including instructions that, when executed by the processor, cause the computing device to:
access one or more user identified words and three or more single-character symbols specifying an individual preference with respect to the one or more identified words;
capture, using the camera, image information of text;
display, on the display screen, the image information of the text;
cause at least a portion of the text in the image information to be recognized;
compare words from a word list to the recognized text; and
modify, in response to identifying at least one word from the word list in the recognized text, a display of the at least one identified word in the displayed image information including displaying at least one single-character symbol,
arranging the three or more single-character symbols so that a group preference with respect to the one or more user identified words can be ascertained.

17. The computing device of claim 16 , wherein the instructions, when executed by the processor, further enable the computing device to:
identify a category associated with the recognized text; and
determine the word list based at least in part on the identified category.

18. The computing device of claim 16 , wherein modifying the display of the at least one identified word further includes displaying at least one of a bounding box around the at least one identified word, displaying a graphical representation of the at least one identified word in color, or color highlighting.

19. The computing device of claim 16 , wherein modifying the display of the at least one identified word includes displaying a graphical overlay at least one of above or

Assignees

• A9 Com Inc

Inventors

• Kraft Adam Wiggen
• Dhua Arnab Sanat Kumar
• Gray Douglas Ryan
• Lin Xiaofan
• Lou Yu
• Ramesh Sunil
• Taylor Colin Jon
• Mott David Creighton

Classifications

• G06T11/60 Primary

  Creating or editing images; Combining images with text · CPC title

• G06V20/80

  Recognising image objects characterised by unique random patterns · CPC title

• G06T19/006

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title