Systems and methods for providing augmented media

Patent metadata

Field	Value
Publication number	US-12198286-B2
Application number	US-202318308324-A
Country	US
Kind code	B2
Filing date	Apr 27, 2023
Priority date	Oct 28, 2020
Publication date	Jan 14, 2025
Grant date	Jan 14, 2025

Abstract

The present disclosure relates, in part, to spatially aware media that includes three-dimensional (3D) spatial information pertaining to a real-world space. The spatially aware media may map this 3D spatial information to media such as an image, for example, to provide 3D spatial context for the media. This may allow users to more flexibly and efficiently interact with virtual content in real-world spaces that are relevant to them. According to one embodiment, spatially aware media is augmented to provide an image of a real-world space overlaid with a render of a 3D model defined relative to the 3D spatial features of the real-world space. Before augmenting the spatially aware media, a recommended position for the 3D model relative to the 3D spatial features of the real-world space may be determined based on the 3D model and/or on the spatially aware media.

First claim

The invention claimed is:

1. A computer-implemented method comprising:
obtaining a three-dimensional (3D) model of a product;
obtaining stored media including: a first image of a real-world space from a first perspective, a second image of the real-world space from a different second perspective, and information identifying 3D spatial features of the real-world space;
generating a first instance of augmented media including the first image of the real-world space from the first perspective overlaid with a first render of the 3D model defined relative to the 3D spatial features of the real-world space, the first render of the 3D model showing a first portion of the product and not a second portion of the product;
generating a second instance of augmented media including the second image of the real-world space from the second perspective overlaid with a different second render of the 3D model defined relative to the 3D spatial features of the real-world space, the second render of the 3D model showing the second portion of the product; and
outputting both the first instance of augmented media and the second instance of augmented media for simultaneous display on a device.

2. The computer-implemented method of claim 1 , wherein the first render of the 3D model and the second render of the 3D model are both at a same position relative to the 3D spatial features of the real-world space.

3. The computer-implemented method of claim 2 , wherein the position is a recommended position for the 3D model relative to the 3D spatial features of the real-world space, and wherein the method further comprises determining the recommended position.

4. The computer-implemented method of claim 3 , wherein the recommended position is based on at least one of: the first image, the second image, or the 3D model.

5. The computer-implemented method of claim 1 , wherein the first image of the real-world space and the second image of the real-world space have a mapping to a 3D scan of the real-world space associated with the information identifying the 3D spatial features of the real-world space.

6. The computer-implemented method of claim 1 , further comprising receiving an instruction to move the 3D model to a new position, and in response updating both the first instance of augmented media and the second instance of augmented media to render the 3D model at the new position in both the first instance of augmented media and the second instance of augmented media.

7. The computer-implemented method of claim 1 , further comprising obtaining results of a search query, and obtaining the 3D model based on the results of the search query.

8. The computer-implemented method of claim 7 , further comprising selecting spatially aware media based on a search string or a search result associated with the search query, wherein the spatially aware media comprises the first image, the second image, and the information identifying the 3D spatial features of the real-world space.

9. The computer-implemented method of claim 2 , wherein the 3D model is a first 3D model, and wherein the method further comprises obtaining a different second 3D model and updating both the first instance of augmented media and the second instance of augmented media to render the second 3D model instead of the first 3D model in both the first instance of augmented media and the second instance of augmented media.

10. The computer-implemented method of claim 9 , wherein the render of the second 3D model is at the same position relative to the 3D spatial features of the real-world space.

11. A system comprising:
at least one processor; and
a memory storing processor-executable instructions that, when executed by the at least one processor, cause the system to:
obtain a three-dimensional (3D) model of a product;
obtain stored media including: a first image of a real-world space from a first perspective, a second image of the real-world space from a different second perspective, and information identifying 3D spatial features of the real-world space;
generate a first instance of augmented media including the first image of the real-world space from the first perspective overlaid with a first render of the 3D model defined relative to the 3D spatial features of the real-world space, the first render of the 3D model showing a first portion of the product and not a second portion of the product;
generate a second instance of augmented media including the second image of the real-world space from the second perspective overlaid with a different second render of the 3D model defined relative to the 3D spatial features of the real-world space, the second render of the 3D model showing the second portion of the product; and
output both the first instance of augmented media and the second instance of augmented media for simultaneous display on a device.

12. The system of claim 11 , wherein the first render of the 3D model and the second render of the 3D model are both at a same position relative to the 3D spatial features of the real-world space.

13. The system of claim 12 , wherein the position is a recommended position for the 3D model relative to the 3D spatial features of the real-world space, and wherein the processor-executable instructions, when executed, further cause the system to determine the recommended position.

14. The system of claim 13 , wherein the recommended position is based on at least one of: the first image, the second image, or the 3D model.

15. The system of claim 11 , wherein the first image of the real-world space and the second image of the real-world space have a mapping to a 3D scan of the real-world space associated with the information identifying the 3D spatial features of the real-world space.

16. The system of claim 11 , wherein the processor-executable instructions, when executed, further cause the system to receive an instruction to move the 3D model to a new position, and in response update both the first instance of augmented media and the second instance of augmented media to render the 3D model at the new position in both the first instance of augmented media and the second instance of augmented media.

17. The system of claim 11 , wherein the processor-executable instructions, when executed, further cause the system to obtain results of a search query, and obtain the 3D model based on the results of the search query.

18. The system of claim 17 , wherein the processor-executable instructions, when executed, further cause the system to select spatially aware media based on a search string or a search result associated with the search query, wherein the spatially aware media comprises the first image, the second image, and the information identifying the 3D spatial features of the real-world space.

19. The system of claim 12 , wherein the 3D model is a first 3D model, and wherein the processor-executable instructions, when executed, further cause the system to obtain a different second 3D model and update both the first instance of augmented media and the second instance of augmented media to render the second 3D model instead of the first 3D model in both the first instance of augmented media and the second instance of augmented media.

20. A non-transitory computer-readable medium having stored thereon computer-executable instructions that, when executed by a computer, cause the computer to the perform operations comprising:
obtaining a three-dimensional (3D) model of a product;
obtaining stored media including: a first image of a real-world space from a

Assignees

• Shopify Inc

Inventors

• Delgado Byron Leonel
• Leroux Stephan
• Beauchamp Daniel

Classifications

• G06T19/003

  Navigation within 3D models or images · CPC title

• G06Q30/0643

  graphically representing goods, e.g. 3D product representation · CPC title

• G06V20/20

  in augmented reality scenes · CPC title

• G06F16/9538

  Presentation of query results · CPC title

• G06Q30/0623

  by investigating goods or services · CPC title