Automated highlighting of identified text

Patent metadata

Field	Value
Publication number	US-9870633-B2
Application number	US-201615387219-A
Country	US
Kind code	B2
Filing date	Dec 21, 2016
Priority date	Sep 25, 2013
Publication date	Jan 16, 2018
Grant date	Jan 16, 2018

Abstract

Various embodiments enable a computing device to perform tasks such as highlighting words in an augmented reality view that are important to a user. For example, word lists can be generated and the user, by pointing a camera of a computing device at a volume of text, can cause words from the word list within the volume of text to be highlighted in a live field of view of the camera displayed thereon. Accordingly, users can quickly identify textual information that is meaningful to them in an Augmented Reality view to aid the user in sifting through real-world text.

First claim

What is claimed is:

1. A non-transitory computer-readable storage medium storing instructions that, when executed by at least one processor, cause a computing device to:
receive a user preference, the user preference selected for an individual user;
access a user identified word of a plurality of user identified words, the user identified word associated with the user preference and a color selected for the individual user;
obtain image data that includes a representation of text;
use an object recognition algorithm to determine a recognized word represented in the image data;
associate the user identified word with the recognized word;
display the recognized word emphasized by a visual color attribute, the visual color attribute matching the color associated with the user identified word; and
arrange the recognized word with respect to a plurality of recognized words, individual recognized words of the plurality of recognized words associated with individual user preferences and individual colors selected for individual users, the plurality of recognized words arranged based on the color, thereby ascertaining a group preference associated with group behavior based on the individual recognized words with respect to the plurality of recognized words.

2. The non-transitory computer-readable storage medium of claim 1 , wherein the visual color attribute comprises at least one of a bounding box around the recognized word, a graphical representation of the recognized word in color, or a highlighting of the recognized word with a color.

3. The non-transitory computer-readable storage medium of claim 2 , wherein the instructions that, when executed by the at least one processor, further cause the computing device to:
receive the plurality of user identified words from a user; and
enable the user to assign a first overlay to a first set of words and a second overlay to a second set of words of the plurality of user identified words.

4. The non-transitory computer-readable storage medium of claim 3 , wherein the first overlay is different in appearance relative to the second overlay.

5. The non-transitory computer-readable storage medium of claim 1 , wherein an overlay associated with each occurrence of the recognized word moves in corresponding motion with at least one word of the representation of text on a display element in a live field of view of a camera.

6. The non-transitory computer-readable storage medium of claim 1 , wherein the instructions that, when executed by the at least one processor, further cause the computing device to:
identify a category associated with the recognized word; and
apply, based at least in part on the identified category, a list of one or more user identified words.

7. A computer-implemented method, comprising:
receiving a user preference, the user preference selected for an individual user;
accessing a user identified word of a plurality of user identified words, the user identified word associated with user preference and a color selected for the individual user;
obtaining image data that includes a representation of text;
using an object recognition algorithm to determine a recognized word represented in the image data;
associating the user identified word with the recognized word;
displaying the recognized word emphasized by a visual color attribute, the visual color attribute matching the color associated with the user identified word; and
arranging the recognized word with respect to a plurality of recognized words, individual recognized words of the plurality of recognized words associated with individual user preferences and individual colors selected for individual users, the plurality of recognized words arranged based on the color, thereby ascertaining a group preference associated with group behavior based on the individual recognized words with respect to the plurality of recognized words.

8. The computer-implemented method of claim 7 , wherein the image data is obtained by a camera, and wherein the image data is displayed on a display element as at least one of a live field of view of the camera or a real-time video captured by the camera.

9. The computer-implemented method of claim 8 , wherein the visual color attribute comprises displaying at least one of a bounding box around the recognized word, or displaying a graphical representation of the recognized word.

10. The computer-implemented method of claim 8 , wherein the visual color attribute comprises displaying a graphical overlay at least one of above or adjacent the recognized word.

11. The computer-implemented method of claim 10 , wherein the graphical overlay moves in corresponding motion with the recognized word on the display element in the live field of view of the camera.

12. The computer-implemented method of claim 10 , wherein the graphical overlay includes an additional visual attribute operable to cause the recognized word to be emphasized relative to a remainder of words of the representation of text displayed on the display element.

13. The computer-implemented method of claim 10 , wherein the graphical overlay further comprises at least one of a bounding box around the recognized word, a graphical representation of the recognized word in color, or color highlighting.

14. The computer-implemented method of claim 10 , further comprising:
receiving the plurality of user identified words from a user; and
enabling the user to assign an overlay to individual ones of the plurality of user identified words.

15. The computer-implemented method of claim 7 , further comprising:
identifying a category associated with the recognized word; and
applying a list of user identified words based at least in part on the identified category.

16. A computing device, comprising:
a processor;
a camera;
a display screen; and
memory including instructions that, when executed by the processor, cause the computing device to:
receive a user preference associated with behavior of an individual user;
access a user identified word of a plurality of user identified words, the user identified word associated with the user preference and a color selected for the individual user;
obtain image data that includes a representation of text;
use an object recognition algorithm to determine a recognized word represented in the image data;
associated the user identified word with the recognized word;
display associated with the recognized word a visual color attribute, the visual color attribute matching the color associated with the user identified word; and
arrange the recognized word with respect to a plurality of recognized words, individual recognized words of the plurality of recognized words associated with individual user preferences and individual colors selected for individual users, the plurality of recognized words arranged based on the color, thereby ascertaining a group preference associated with group behavior based on the individual recognized words with respect to the plurality of recognized words.

17. The computing device of claim 16 , wherein the instructions, when executed by the processor, further enable the computing device to:
identify a category associated with the recognized word; and
determine a user identified word list based at least in part on the identified category.

18. The computing device of claim 16 , wherein the visual color attribute comprises displaying at least one of a bounding box around the recognized word, displaying a graphical representation of the recognized word, or color highlighting.

Assignees

• A9 Com Inc

Inventors

• Kraft Adam Wiggen
• Dhua Arnab Sanat Kumar
• Gray Douglas Ryan
• Lin Xiaofan
• Lou Yu
• Ramesh Sunil
• Taylor Colin Jon
• Mott David Creighton

Classifications

• G06T11/60 Primary

  Creating or editing images; Combining images with text · CPC title

• G06K9/00577

  Physics · mapped topic

• G06T19/006

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06V20/80

  Recognising image objects characterised by unique random patterns · CPC title