Augmented reality Camera Lucida

Patent metadata

Field	Value
Publication number	US-9965895-B1
Application number	US-201414221104-A
Country	US
Kind code	B1
Filing date	Mar 20, 2014
Priority date	Mar 20, 2014
Publication date	May 8, 2018
Grant date	May 8, 2018

Abstract

Approaches are described for enabling a user to create an accurate perspective rendering of a source (e.g., a scene, object, subject, point of interest, etc.) on a drawing surface. For example, various approaches enable superimposition of the source being viewed upon a drawing surface upon which a user is drawing. In this way, the user can view both the source and drawing surface simultaneously. This allows the user to duplicate key points of the source on the drawing surface by viewing a display of a device, thus aiding in the accurate rendering of perspective.

First claim

What is claimed is:

1. A computer implemented method, comprising:
obtaining first image data using a camera of a computing device, the first image data including a representation of a point of interest;
obtaining second image data of a live camera view that includes a representation of a physical surface;
analyzing the second image data to determine a boundary that includes the representation of the physical surface;
presenting a view of the point of interest overlaid on the representation of the physical surface and confined within the boundary;
using an image processing algorithm to generate an outline of the point of interest;
detecting, concurrently with obtaining the second image data, hand-written markings on the physical surface representing a drawing of the point of interest;
generating a model representing a relationship between the physical surface and the second image data based at least in part on the hand-written markings, the model including a first rendering of the drawing from hand-written markings;
detecting, concurrently with obtaining the second image data, additional hand-written markings;
updating the model to generate an updated model that represents a second rendering of the drawing, the second rendering of the drawing corresponding to a current rendering of the drawing;
comparing the updated model to the outline of the point of interest to determine an amount of unfinished drawing of the point of interest, the amount of unfinished drawing corresponding to a progress in creating the drawing of the point of interest; and
generating a prompt to indicate the amount of unfinished drawing of the point of interest in drawing of the point of interest.

2. The computer implemented method of claim 1 , wherein analyzing the second image data to determine the boundary further comprises:
analyzing the second image data to detect boundary markings that include one of a registration mark, one or more corners of the representation of the physical surface, or one or more edges of the representation of the physical surface.

3. The computer implemented method of claim 1 , further comprising:
analyzing the second image data to detect one or more marks on the representation of the physical surface, wherein the one or more marks correspond to a mark made from at least a writing instrument, a drawing instrument, or a painting instrument, and wherein the point of interest includes one of a scene or an object.

4. The computer implemented method of claim 1 , further comprising:
locking a placement of the representation of the point of interest over the representation of the physical surface.

5. The computer implemented method of claim 1 , further comprising:
detecting movement of a computing device, the computing device including a display element configured to display the view of the representation of the point of interest overlaid over the representation of the physical surface; and
tracking a location of the physical surface relative to the computing device,
wherein a placement of the representation of a point of interest remains in substantially a same location on the representation of a physical surface during the movement of the computing device.

6. The computer implemented method of claim 1 , further comprising:
comparing the model to the representation of the point of interest; and
generating at least one prompt, the at least one prompt including information indicating at least one or a progress in creating the point of interest, a number of errors in creating the point of interest, an indication of an error in creating the point of interest.

7. The computer implemented method of claim 1 , further comprising:
detecting a selection action being performed by a user with respect to at least one user-selectable element on a display element of a computing device, the at least one user-selectable element associated with one of a plurality of image editing algorithms;
enabling one of the plurality of image editing algorithms based at least on the selection action being detected; and
adjusting, based at least in part on the enabled one of the plurality of image editing algorithms, a current view of the display element.

8. The computer implemented method of claim 7 , wherein the plurality of image editing algorithms are configured to perform at least one of adjusting a zoom level of the first image data, adjusting a rotation of the first image data, adjusting an opacity of the first image data, displaying on outline of the first image data, or adjusting a contrast of the first image data.

9. The computer implemented method of claim 1 , wherein the first image data is acquired from one of a camera of a computing device, a data storage component of the computing device, derived from a three-dimensional model, or an application configured to operate on the computing device, and wherein the second image data includes a representation of a drawing surface acquired by a camera of a computing device.

10. The computer implemented method claim 5 , wherein tracking the location of the physical surface includes:
analyzing the first image data to identify a plurality of feature points representative of the point of interest; and
tracking the physical surface with respect to the computing device using at least a random sample consensus (RANSAC) algorithm and based at least in part on a portion of the plurality of feature points.

11. The computer implemented method of claim 1 , further comprising:
invoking version control as additional hand-written markings are detected.

12. A non-transitory computer readable storage medium storing one or more sequences of instructions executable by one or more processors to perform a set of operations comprising:
obtaining first image data using a camera of a computing device, the first image data including a representation of a point of interest;
obtaining second image data of a live camera view that includes a representation of a physical surface;
analyzing the second image data to determine a boundary that includes the representation of the physical surface;
presenting a view of the point of interest overlaid on the representation of the physical surface and confined within the boundary;
using an image processing algorithm to generate an outline of the point of interest;
detecting, concurrently with obtaining the second image data, hand-written markings on the physical surface representing a drawing of the point of interest;
generating a model representing a relationship between the physical surface and the second image data based at least in part on the hand-written markings, the model including a first rendering of the drawing from hand-written markings;
detecting, concurrently with obtaining the second image data, additional hand-written markings;
updating the model to generate an updated model that represents a second rendering of the drawing, the second rendering of the drawing corresponding to a current rendering of the drawing;
comparing the updated model to the outline of the point of interest to determine an amount of unfinished drawing of the point of interest, the amount of unfinished drawing corresponding to a progress in creating the drawing of the point of interest; and
generating a prompt to indicate the amount of unfinished drawing of the point of interest in drawing of the point of interest.

13. The non-transitory computer readable storage medium of claim 12 , further comprising instructions executed by the one or more processors to perform the operations of:
determining a boundary that corresponds to the representation of the physical surface; and
analyzing the second

Assignees

• A9 Com Inc

Inventors

• Gray Douglas Ryan

Classifications

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06T11/20

  Drawing from basic elements · CPC title

• G06T2219/2012

  Colour editing, changing, or manipulating; Use of colour codes · CPC title

• G06T2219/2016

  Rotation, translation, scaling · CPC title

• G06T2219/2021

  Shape modification · CPC title