Augmented reality product preview

Patent metadata

Field	Value
Publication number	US-9734634-B1
Application number	US-201414498832-A
Country	US
Kind code	B1
Filing date	Sep 26, 2014
Priority date	Sep 26, 2014
Publication date	Aug 15, 2017
Grant date	Aug 15, 2017

Abstract

Systems and methods for displaying 3D containers in a computer generated environment are described. A computing device may provide a user with a catalog of objects which may be purchased. In order to view what an object may look like prior to purchasing the object, a computing device may show a 3D container that has the same dimensions as the object. As discussed herein, the 3D container may be located and oriented based on a two-dimensional marker. Moreover, some 3D containers may contain a representation of an object, which may be a 2D image of the object.

First claim

What is claimed is:

1. A computing device comprising:
at least one processor;
at least one camera;
a display screen; and
memory including instructions that, when executed by the at least one processor, cause the computing device to perform a set of operations comprising:
capture, using the at least one camera, at least one frame of a video including a representation of a real-world environment, the at least one frame including a representation of a two-dimensional physical marker located in the real-world environment;
determine one or more dimensions of the two-dimensional physical marker in the at least one frame;
determine an orientation of the computing device with respect to the real-world environment;
determine a plane of the two-dimensional physical marker in the frame with respect to the computing device;
receive a selection of a product from an electronic catalog of products being presented on the display screen to identify a selected product;
determine one or more dimensions of the selected product with respect to a camera environment of the real-world environment based at least in part on metadata associated with the product, the metadata describing at least one or more real-world dimensions corresponding to the selected product;
render the camera environment of the real-world environment for display on the display screen based at least in part on the plane of the two-dimensional physical marker and the orientation of the computing device, the camera environment including a representation of a semi-transparent container within which a graphical representation of the selected product is presented, wherein the semi-transparent container is rendered in the camera environment based at least in part on the one or more dimensions of the two-dimensional physical marker and the one or more dimensions of the selected product;
determine whether the semi-transparent container overlaps an object located in the real-world environment that is represented in the camera environment; and
generate an alert in response to determining an overlap with the object, the alert indicating whether the selected product would physically contact the object in the real-world environment when positioned as represented in the camera environment.

2. The computing device of claim 1 , wherein the instructions, when executed by the processor, further enable the computing device to:
receive user input to change a location of the semi-transparent container rendered in the camera environment; and
based on the user input, render the semi-transparent container in a new location in the camera environment, wherein the semi-transparent container in the new location includes the graphical representation of the selected product.

3. The computing device of claim 1 , wherein the instructions, when executed by the processor, further enable the computing device to:
receive input associated with a smaller container,
display objects from the electronic catalog of products that are smaller than the semi-transparent container within which a graphical representation of the selected product is presented and larger than the smaller container.

4. A computer-implemented method, comprising:
acquiring an image of a real-world environment containing a two-dimensional marker using at least one camera of a computing device;
determining at least a distance and orientation of a representation of the two-dimensional marker in the image;
displaying, on a display screen of the computing device, an interface in which the image of the real-world environment is presented, and wherein the interface also includes a listing of products;
receiving a selection of a product from the listing of products;
determining one or more dimensions of the product based at least in part on metadata associated with the product;
displaying, on the display screen, a representation of the product through the interface, the representation of the product appearing within an at least one virtual container, wherein a size of the at least one virtual container is based at least in part on the one or more dimensions associated with the product, and wherein the representation of the product is based at least in part on the distance and orientation of the two-dimensional marker in the image; and
determine whether the virtual container overlaps an object located in the real-world environment that is represented in the interface; and
generate an alert in response to determining an overlap with the object, the alert indicating whether the product would physically contact the object in the real-world environment when orientated as represented in the interface.

5. The computer-implemented method of claim 4 , further comprising:
receiving input to move the representation of the product displayed on the display screen; and
based on the input, moving the representation of the product within the display either along a plane that the two-dimensional marker is located on or along a plane that is parallel to the plane that the two-dimensional marker is located on.

6. The computer-implemented method of claim 4 , wherein the representation of the product is based at least in part on the distance of the two-dimensional marker in the image, the orientation of the two-dimensional marker in the image, and a direction of little g.

7. The computer-implemented method of claim 4 , wherein the representation is a first representation, the method further comprising:
acquiring a second image of the real-world environment containing the two-dimensional marker;
determining a second distance and a second orientation of the two-dimensional marker in the second image; and
displaying a second representation of the product to be displayed in the real-world environment in the interface, the second representation of the product appearing within the virtual container, wherein the second representation of the product being displayed in the interface is based at least in part on the second distance and second orientation of the two-dimensional marker in the second image.

8. The computer-implemented method of claim 7 , wherein the second representation of the product being displayed includes a different perspective of the product based at least in part on a perspective associated with the second image creating visual consistency between planes of representation such that a transition from displaying the first representation of the product to displaying the second representation of the product is smooth.

9. The computer-implemented method of claim 4 , further comprising:
acquiring metadata associated with the product; and
determining the size of the product based on the acquired metadata.

10. The computer-implemented method of claim 4 , further comprising:
receiving a selection of a second product from the listing of product;
determining a size associated with the selected second product; and
displaying a representation of the selected second product to be displayed in the interface, the representation of the selected second product appearing within the virtual container, wherein the representation of the selected second product being displayed in the interface is based at least in part on the distance and orientation of the two-dimensional marker in the image.

11. The computer-implemented method of claim 10 , wherein determining a selected second product is further based on the size of the second product, wherein the size of the second product is less than the product.

12. The computer-implemented method of claim 4 , further comprising:
identifying a plurality of objects in the real-world environment; and
causing the representation of the object to be displ

Assignees

• A9 Com Inc

Inventors

• Mott David Creighton
• Dhua Arnab Sanat Kumar
• Taylor Colin Jon
• Lou Yu
• Wang Chun-Kai
• Pantham Sudeshna
• Arora Himanshu
• Zhang Xi

Classifications

• G06T2215/16

  Using real world measurements to influence rendering · CPC title

• G06F3/011

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06T19/20

  Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts · CPC title

• G02B27/017

  Head mounted · CPC title

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title