Visual editor for designing augmented-reality effects

Patent metadata

Field	Value
Publication number	US-10360736-B2
Application number	US-201715615471-A
Country	US
Kind code	B2
Filing date	Jun 6, 2017
Priority date	Mar 15, 2017
Publication date	Jul 23, 2019
Grant date	Jul 23, 2019

Abstract

In one embodiment, a method for designing an augmented-reality effect may include receiving a model definition of a virtual object. The virtual object may be rendered in a 3D space based on the model definition. The system may display the virtual object in the 3D space from a first perspective in a first display area of a user interface. The system may display the virtual object in the 3D space from a second perspective, different from the first, in a second display area of the user interface. The system may receive a user command input by a user through the first display area for adjusting the virtual object. The virtual object may be adjusted according to the user command. The system may display the adjusted virtual object in the 3D space from the first perspective in the first display area and from the second perspective in the second display area.

First claim

What is claimed is:

1. A method, comprising:
by a computing system, receiving a model definition of a virtual object;
by the computing system, rendering the virtual object in a 3D space based on the model definition;
by the computing system, displaying the virtual object in the 3D space from a first perspective in a first display area of a user interface;
by the computing system, displaying the virtual object in the 3D space from a second perspective in a second display area of the user interface, the second perspective being different from the first perspective;
by the computing system, receiving a user command for adjusting the virtual object, the user command being input by a user through the first display area;
by the computing system, adjusting the virtual object according to the user command;
by the computing system, displaying the adjusted virtual object in the 3D space from the first perspective in the first display area;
by the computing system, displaying the adjusted virtual object in the 3D space from the second perspective in the second display area; and
by the computing system, outputting an adjusted model definition of the adjusted virtual object.

2. The method of claim 1 , further comprising:
by the computing system, receiving a second user command for adjusting the virtual object, the user command being input by a user through the second display area;
by the computing system, further adjusting the adjusted virtual object according to the second user command;
by the computing system, displaying the further adjusted virtual object in the 3D space from the first perspective in the first display area; and
by the computing system, displaying the further adjusted virtual object in the 3D space from the second perspective in the second display area.

3. The method of claim 1 , further comprising:
by the computing system, displaying a representation of a mobile device and its focal plane in the 3D space from the first perspective in the first display area; and
by the computing system, displaying the representation of the mobile device and its focal plane in the 3D space from the second perspective in the second display area;
wherein the representation of the focal plane corresponds to a focal plane of a first camera of the mobile device.

4. The method of claim 3 , further comprising:
by the computing system, receiving a second user command to enable a second camera of the mobile device and to disable the first camera of the mobile device;
by the computing system, in response to the second user command:
displaying a second representation of a second focal plane in the 3D space from the first perspective in the first display area;
displaying the second representation of the second focal plane in the 3D space from the second perspective in the second display area; and
ceasing to display the representation of the focal plane;

wherein the second representation of the second focal plane corresponds to a focal plane of the second camera.

5. The method of claim 3 , further comprising:
by the computing system, receiving a second user command to change an orientation of the representation of the mobile device; and
by the computing system, in response to the second user command, changing the orientation of the displayed representation of the mobile device and its focal plane in the first display area and the second display area.

6. The method of claim 5 , wherein the second user command to change the orientation is based on gyroscope data received from a device communicatively coupled to the computing system, wherein the device comprises a gyroscope.

7. The method of claim 1 , wherein the 3D space comprises a representation of a mobile device, the method further comprising:
by the computing system, displaying in a third display area a scene in the 3D space, the scene representing a view from the mobile device through its focal plane.

8. The method of claim 1 , wherein the 3D space comprises a video sphere centered at the mobile device, the method further comprising:
by the computing system, displaying a background in the third display area, the background being a portion of the video sphere selected based on an orientation of the mobile device.

9. The method of claim 8 , further comprising:
by the computing system, overlaying a video of a person on top of the background.

10. The method of claim 9 , wherein an effect associated with the virtual object is configured to be triggered by motions of the person.

11. The method of claim 7 , wherein the virtual object is configured to be displayed in the focal plane of the mobile device regardless of the mobile device's orientation.

12. The method of claim 7 , wherein the virtual object is configured to be displayed in the third display area when mobile device is oriented such that the virtual object is visible through the focal plane.

13. The method of claim 1 , wherein the virtual object is configurable to be associated with a first camera of a mobile device or a second camera of the mobile device;
wherein when the virtual object is associated with the first camera, the virtual object is displayable in a scene captured by the first camera and not displayable in a scene captured by the second camera; and
wherein when the virtual object is associated with the second camera, the virtual object is displayable in a scene captured by the second camera and not displayable in a scene captured by the first camera.

14. One or more computer-readable non-transitory storage media embodying software that is operable when executed to:
receive a model definition of a virtual object;
render the virtual object in a 3D space based on the model definition;
display the virtual object in the 3D space from a first perspective in a first display area of a user interface;
display the virtual object in the 3D space from a second perspective in a second display area of the user interface, the second perspective being different from the first perspective;
receive a user command for adjusting the virtual object, the user command being input by a user through the first display area;
adjust the virtual object according to the user command;
display the adjusted virtual object in the 3D space from the first perspective in the first display area;
display the adjusted virtual object in the 3D space from the second perspective in the second display area; and
output an adjusted model definition of the adjusted virtual object.

15. The media of claim 14 , wherein the software is further operable when executed to:
display a representation of a mobile device and its focal plane in the 3D space from the first perspective in the first display area; and
display the representation of the mobile device and its focal plane in the 3D space from the second perspective in the second display area;
wherein the representation of the focal plane corresponds to a focal plane of a first camera of the mobile device.

16. The media of claim 15 , wherein the software is further operable when executed to:
receive a second user command to enable a second camera of the mobile device and to disable the first camera of the mobile device;
in response to the second user command:
display a second representation of a second focal plane in the 3D space from the first perspective in the first display area;
display the second representation of the second focal plane in the 3D space from the second perspective in the second display area; and
cease to display the representation of the focal plane;

wherein the second representation of the second focal plan

Assignees

• Facebook Inc

Inventors

• Smet Stef Marc
• Falola Dolapo Omobola
• Slater Michael
• Krug Samantha P
• Gigniak Volodymyr
• Verlinde Hannes Luc Herman
• Anpilov Sergei Viktorovich
• Gontovnik Danil
• Ng Yu Hang
• Hanchar Siarhei
• Dzhumerov Milen Georgiev

Classifications

• G06F3/14 Primary

  Digital output to display device {; Cooperation and interconnection of the display device with other functional units} · CPC title

• G06F30/00

  Computer-aided design [CAD] · CPC title

• G06T2200/24

  involving graphical user interfaces [GUIs] · CPC title

• G06T19/20 Primary

  Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts · CPC title

• G06T2215/16

  Using real world measurements to influence rendering · CPC title