Augmented reality enabled dynamic product presentation

Patent metadata

Field	Value
Publication number	US-11935202-B2
Application number	US-202217824828-A
Country	US
Kind code	B2
Filing date	May 25, 2022
Priority date	May 25, 2022
Publication date	Mar 19, 2024
Grant date	Mar 19, 2024

Abstract

Systems and methods described herein allow a customer to employ AR/VR software to generate virtual representations of physical spaces (e.g., house) and sub-spaces (e.g., living room) to preview virtual objects situated in AR/VR virtual environments. A commerce system (or mobile app associated with the commerce system) may generate virtualized environments representing a physical space (e.g., house, apartment) and regions (e.g., living room, kitchen) based on source images uploaded to or otherwise captured by the commerce system. The end-user may operate the software on a client device and interacts with VR or AR presentations of the virtual environment using a voice-based interface recognized by the software. For example, the end-user may say the name of room (region) or an object and the system retrieves data of the identified room or an appropriate room, such as virtual representations of furniture or objects situated in the room.

First claim

What is claimed is:

1. A computer-implemented method comprising:
receiving, by a computer, an utterance indicating an object;
selecting, by the computer, a three-dimensional virtual environment based upon the utterance, wherein the three-dimensional virtual environment represents a region of a physical space; and
providing for display, by the computer, a virtual object representing the object, the virtual object situated in the three-dimensional virtual environment at a position corresponding to the region of the physical space.

2. The method according to claim 1 , wherein the three-dimensional virtual environment comprises the virtual object in an augmented representation of an image of the region from a camera.

3. The method according to claim 2 , wherein the utterance is received from a client device, and further comprising:
determining, by the computer, a location of the client device,
wherein the computer generates the augmented representation of the image from the camera in response to determining that the client device is located in the region.

4. The method according to claim 1 , further comprising determining, by the computer, a spoken instruction in the utterance by applying a speech recognition function and a natural language processing (NLP) function on the utterance.

5. The method according to claim 1 , wherein selecting the three-dimensional virtual environment based upon the instruction in the utterance includes:
identifying, by the computer, in a database the three-dimensional virtual environment associated with a verbal identifier as indicated by the utterance.

6. The method according to claim 1 , wherein selecting the three-dimensional virtual environment based upon the instruction includes:
determining, by the computer, an object type for the object indicated by the utterance; and
identifying, by the computer, in a database the three-dimensional virtual environment associated with the object type.

7. The method according to claim 1 , wherein the three-dimensional virtual environment is selected from a plurality of three-dimensional virtual environments associated with an end-user.

8. The method according to claim 1 , further comprising generating, by the computer, the three-dimensional virtual environment representing the region based upon the one or more source images depicting the region.

9. The method according to claim 1 , further comprising generating, by the computer, in a database one or more object tables corresponding to one or more regions associated with an end-user, wherein an object table corresponding to the region indicates a set of one or more objects associated with the region.

10. The method according to claim 9 , wherein the object in the utterance is associated with the set of one or more objects associated with the region, and wherein the computer selects the three-dimensional virtual environment representing the region using the object table corresponding to the region.

11. The method according to claim 1 , further comprising determining, by the computer, a region type of the region based upon one or more attributes of the region, the one or more attributes of the region including at least one of: a set of one or more objects in the region or a set of one or more spatial features.

12. The method according to claim 11 , further comprising generating, by the computer, one or more identifiers for the region based upon the region type as determined by the computer based upon the one or more attributes of the region.

13. The method according to claim 11 , wherein the computer determines an object type of the object indicated by the utterance, and
wherein the computer determines that the object type of the object is associated with the set of one or more objects associated with the region type of the region.

14. The method according to claim 1 , further comprising:
obtaining, by the computer, a plurality of virtual objects corresponding to a plurality of objects identified in source image data for the region; and
determining, by the computer, the positions of the virtual objects according to a plurality of spatial parameters of a three-dimensional coordinate system of the three-dimensional virtual environment corresponding to the region of the physical space.

15. The method according to claim 14 , wherein the position of the virtual object for the object indicated by the utterance is relative to one or more attributes of a second virtual object of the plurality of objections, the one or more attributes of the second virtual object including at least one of: an object type or a spatial feature.

16. The method according to claim 14 , further comprising:
identifying, by the computer, a positioning collision in the three-dimensional coordinate system based upon each position determined for a second virtual object of the plurality of virtual objects and the virtual object for the object indicated by the utterance; and
modifying, by the computer, the position of the virtual object for the objection indicated by the utterance responsive to the position collision.

17. A system comprising:
a computer comprising a processor configured to:
receive an utterance indicating an object;
select a three-dimensional virtual environment based upon the utterance, wherein the three-dimensional virtual environment represents a region of a physical space; and
provide for display a virtual object representing the object, the virtual object situated in the three-dimensional virtual environment at a position corresponding to the region of the physical space.

18. The system according to claim 17 , wherein the three-dimensional virtual environment comprises the virtual object in an augmented representation of an image of the region from a camera.

19. The system according to claim 18 , wherein the utterance is received from a client device, wherein the processor of the computer is further configured to determine a location of the client device, and wherein the computer generates the augmented representation of the image from the camera in response to determining that the client device is located in the region.

20. The system according to claim 17 , further comprising determining, by the computer, a spoken instruction in the utterance by applying a speech recognition function and a natural language processing (NLP) function on the utterance.

21. The system according to claim 17 , wherein when selecting the three-dimensional virtual environment based upon the instruction in the utterance the processor of the computer is further configured to identify in a database the three-dimensional virtual environment associated with a verbal identifier as indicated by the utterance.

22. The system according to claim 17 , wherein when selecting the three-dimensional virtual environment based upon the instruction the processor of the computer is further configured to:
determine an object type for the object indicated by the utterance; and
identify in a database the three-dimensional virtual environment associated with the object type.

23. The system according to claim 17 , wherein the three-dimensional virtual environment is selected from a plurality of three-dimensional virtual environments associated with an end-user.

24. The system according to claim 17 , wherein the processor of the computer is further configured to generate the three-dimensional virtual environment representing the region based upon the one or more sourc

Assignees

• Shopify Inc

Inventors

• Beauchamp Daniel
• Delgado Byron Leonel
• Wade Jonathan

Classifications

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06F3/167

  Audio in a user interface, e.g. using voice commands for navigating, audio feedback · CPC title

• G06Q30/0643 Primary

  graphically representing goods, e.g. 3D product representation · CPC title

• G06V20/20

  in augmented reality scenes · CPC title