Providing overlays based on text in a live camera view

Patent metadata

Field	Value
Publication number	US-2016005189-A1
Application number	US-201514850813-A
Country	US
Kind code	A1
Filing date	Sep 10, 2015
Priority date	Sep 21, 2012
Publication date	Jan 7, 2016
Grant date	—

Abstract

Approaches are described for rendering augmented reality overlays on an interface displaying the active field of view of a camera. The interface can display to a user an image or video, for example, and the overlay can be rendered over, near, or otherwise positioned with respect to any text or other such elements represented in the image. The overlay can have associated therewith at least one function or information, and when an input associated with the overlay is selected, the function can be performed (or caused to be performed) by the portable computing device.

First claim

What is claimed is:

1 . A computing system, comprising:
a display element; a camera; at least one processor; and memory including instructions that, when executed by the processor, cause the computing system to:
capture image data by the camera, the image data including a representation of text;
analyze the image data to locate the representation of text;
analyze the representation of the text to determine an action associated with the text;
display, on the display element, over a view being captured by the camera, a graphical representation of the text, the graphical representation of the text being rendered with respect to a location of the representation of the text and appearing more prominent than the representation of the text; and
perform, in response to a selection of the graphical representation, the action associated with the text.

2 . The computing system of claim 1 , wherein the instructions when executed by the processor further enable the computing system to:
detect an interaction with the graphical representation of the text; and perform the action associated with the text.

3 . The computing system of claim 1 , wherein the instructions when executed by the processor further enable the computing system to:
determine an application associated with the action; detect an interaction with the graphical representation of the text; instantiate an instance of the application; and perform the action by the instance of the application using the text.

4 . The computing system of claim 1 , wherein the instructions when executed by the processor further enable the computing system to:
analyze the image to determine a region that includes the representation of text; crop the region to generate a region of cropped text; binarize the region of cropped text to generate a binarized region of text; and provide the binarized region of text to a character recognition process for processing.

5 . The computing system of claim 4 , wherein the instructions when executed by the processor further enable the computing system to:
decode the binarized region of text to identify the text; and determine the action using the decoded binarized region of text.

6 . The computing system of claim 1 , wherein the instructions when executed by the processor further enable the computing system to:
detect a selection of the graphical representation; provide an alert to confirm performance of the action; receive confirmation; and perform the action.

7 . The computing system of claim 4 , wherein the representation of text corresponds to one of a phone number, an email address, a website URL, a physical address, a business card, a product model number, a book spine, a credit card, a word in a foreign language, a price in a foreign currency, a barcode, a nutrition label and a stock market ticker.

8 . The computing system of claim 4 , wherein the action correspond to calling a phone number, instantiating an instance of an email client, instantiating an instance of a web browser, instantiating an instance of a navigation application, or instantiating an instance of a shopping application.

9 . The computing system of claim 4 , wherein the location corresponds to one of directly above the representation of the text or up, down, right, or left of the representation of the text.

10 . The computing system of claim 4 , wherein appearing more prominent includes causing the graphical representation of the text to appear more bold than the representation of text, causing a size of the graphical representation of the text to appear larger than the representation of the text, causing a color to the graphical representation of the text to be different from the color of the representation of the text, or causing the graphical representation of the text to move in a path within a predetermined area about the representation of the text.

11 . A method, comprising:
capturing image data by a camera of a computing device, the image data including a representation of text; analyzing the image data to locate the representation of text; analyzing the representation of the text to determine an action associated with the text; displaying, on a display element of the computing device, over a view being captured by the camera, a graphical representation of the text, the graphical representation of the text being rendered with respect to a location of the representation of the text and appearing more prominent than the representation of the text; and performing, in response to a selection of the graphical representation, the action associated with the text.

12 . The method of claim 11 , wherein the method is performed locally on the computing device.

13 . The method of claim 11 , further comprising:
detecting an interaction with the graphical representation of the text; and performing the action associated with the text.

14 . The method of claim 11 , further comprising:
determining an application associated with the action; detecting an interaction with the graphical representation of the text; instantiating an instance of the application; and performing the action by the instance of the application using the text.

15 . The method of claim 11 , further comprising:
analyzing the image to determine a region that includes the representation of text; cropping the region to generate a region of cropped text; binarizing the region of cropped text to generate a binarized region of text; decoding the binarized region of text to identify the text; and determining the action using the decoded binarized region of text.

16 . The method of claim 11 , further comprising:
detecting a selection of the graphical representation; providing an alert to confirm performance of the action; receiving confirmation; and performing the action.

17 . A non-transitory computer readable storage medium storing one or more sequences of instructions executable by one or more processors to perform a set of operations comprising:
capturing image data by a camera of a computing device, the image data including a representation of text; analyzing the image data to locate the representation of text; analyzing the representation of the text to determine an action associated with the text; displaying, on a display element of the computing device, over a view being captured by the camera, a graphical representation of the text, the graphical representation of the text being rendered with respect to a location of the representation of the text and appearing more prominent than the representation of the text; and performing, in response to a selection of the graphical representation, the action associated with the text.

18 . The non-transitory computer readable storage medium of claim 17 , further comprising instructions executed by the one or more processors to perform the operations of:
determining an application associated with the action; detecting an interaction with the graphical representation of the text; instantiating an instance of the application; and performing the action by the instance of the application using the text.

19 . The non-transitory computer readable storage medium of claim 17 , further comprising instructions executed by the one or more processors to perform the operations of:
analyzing the image to determine a region that includes the representation of text; cropping the region to generate a region of cropped text; binarizing the region of cropped text to generate a binarized reg

Assignees

• A9 Com Inc

Inventors

• Gray Douglas R
• Dhua Arnab S
• Lou Yu
• Ramesh Sunil

Classifications

• G06T11/00 Primary

  Two-dimensional [2D] image generation · CPC title

• G06V20/20 Primary

  in augmented reality scenes · CPC title

• G06K9/46

  Physics · mapped topic

• H04N7/183

  for receiving images from a single remote source · CPC title

• G06K9/00456

  Physics · mapped topic