Computing platform for facilitating augmented reality experiences with third party assets

Patent metadata

Field	Value
Publication number	US-12039679-B2
Application number	US-202217993334-A
Country	US
Kind code	B2
Filing date	Nov 23, 2022
Priority date	Jan 7, 2021
Publication date	Jul 16, 2024
Grant date	Jul 16, 2024

Abstract

Systems and methods for data asset acquisition and obfuscation can be helpful for retrieving augmented reality rendering data assets from third parties. The sending of a software development kit and receiving back data assets can ensure the data assets are compatible with the augmented reality rendering experience in the user interface. The data acquisition system with obfuscation can also ensure the code generated by third parties is stripped of semantics and has reduced readability.

First claim

What is claimed is:

1. A computing system comprising:
one or more processors;
one or more non-transitory computer readable media that collectively store instructions that, when executed by the one or more processors, cause the computing system to perform operations, the operations comprising:
receiving a software development kit, wherein the software development kit comprises a template for generating an augmented reality experience that leverages one or more rendering effect shaders, wherein the software development kit comprises one or more per-product presets, wherein the template and the one or more per-product presets are associated with products in a particular product class, wherein the one or more per-product presets comprise one or more parameters associated with uniform values and textures for the particular product class;
receiving one or more inputs to modify the template, wherein modifying the software development kit generates a product-specific augmented reality experience;
generating data assets based at least in part on the one or more inputs, wherein the data assets are associated with one or more particular products of the particular product class, wherein generating the data assets comprise generating one or more rendering subgraphs for the one or more particular products, wherein the one or more rendering subgraphs when processed with a perception subgraph generates a rendering of the one or more particular products, wherein the perception subgraph was generated by a facilitator computing system, and wherein the perception subgraph is compatible with a plurality of different rendering subgraphs associated with a plurality of different products; and
sending the data assets comprising the one or more rendering subgraphs to a second computing system, wherein the second computing system stores the perception subgraph, and wherein the second computing system uses the one or more rendering subgraphs with the perception subgraph to provide an augmented reality rendering experience;
wherein the one or more rendering subgraphs comprise a plurality of product-specific shaders generated based on modifying the template based on the one or more inputs, and wherein the one or more rendering subgraphs comprise one or more render calculators that process a camera feed, the plurality of product-specific shaders, one or more meshes generated by the perception subgraph, and one or more segmentation masks generated by the perception subgraph to provide the augmented reality rendering experience.

2. The system of claim 1 , wherein the software development kit comprises a joint interface for editing and previewing an augmented reality experience.

3. The system of claim 1 , wherein the operations further comprise:
providing the product-specific augmented-reality experience for display.

4. The system of claim 1 , wherein the operations further comprise:
processing the data assets to generate obfuscated code; and
sending the obfuscated code to the second computing device.

5. The system of claim 1 , wherein the operations further comprise:
testing the augmented reality experience, wherein testing the augmented reality experience comprises:
obtaining training data;
processing the training data with the augmented reality experience to generate augmented reality media;
evaluating a loss function based at least in part on a comparison between the augmented reality media and ground truth data; and
adjusting one or more parameters based at least in part on the loss function.

6. The system of claim 1 , wherein the data assets comprise code;
wherein the operations further comprise:
determining one or more comments in the code comprise text descriptive of code semantics; and
removing the one or more comments that comprise text descriptive of code semantics.

7. The system of claim 1 , wherein the software development kit is received from the second computing system.

8. The system of claim 1 , wherein generating the data assets based at least in part on the one or more inputs comprises generating a renderable compressed file that comprises the data assets that are associated with rendering a product-specific rendering effect.

9. The system of claim 1 , wherein product-specific augmented reality experience comprises an augmented-reality try-on experience that renders the one or more products in user image data.

10. The system of claim 1 , wherein the data assets comprise a rendering subgraph associated with the one or more products, and wherein the rendering subgraph is processed with the perception subgraph to provide the product-specific augmented reality experience.

11. A computer-implemented method, the method comprising:
receiving, by a computing system comprising one or more processors, a software development kit, wherein the software development kit comprises a template for generating an augmented reality experience that leverages one or more rendering effect shaders, wherein the software development kit comprises one or more per-product presets, wherein the template and the one or more per-product presets are associated with products in a particular product class, wherein the one or more per-product presets comprise one or more parameters associated with uniform values and textures for the particular product class;
receiving, by the computing system, one or more inputs to modify the template, wherein modifying the software development kit generates a product-specific augmented reality experience;
generating, by the computing system, data assets based at least in part on the one or more inputs, wherein the data assets are associated with one or more particular products of the particular product class, wherein generating the data assets comprise generating one or more rendering subgraphs for the one or more particular products, wherein the one or more rendering subgraphs when processed with a perception subgraph generates a rendering of the one or more particular products, wherein the perception subgraph was generated by a facilitator computing system, wherein the perception subgraph is compatible with a plurality of different rendering subgraphs associated with a plurality of different products; and
sending, by the computing system, the data assets comprising the one or more rendering subgraphs to a second computing device, wherein the second computing system stores the perception subgraph, and wherein the second computing system uses the one or more rendering subgraphs with the perception subgraph to provide an augmented reality rendering experience;
wherein the one or more rendering subgraphs comprise a plurality of product-specific shaders generated based on modifying the template based on the one or more inputs, and wherein the one or more rendering subgraphs comprise one or more render calculators that process a camera feed, the plurality of product-specific shaders, one or more meshes generated by the perception subgraph, and one or more segmentation masks generated by the perception subgraph to provide the augmented reality rendering experience.

12. The method of claim 11 , further comprising:
obfuscating, by the computing system, the data assets, wherein obfuscating comprises:
removing, by the computing system, whitespace from code of the data assets; and
renaming, by the computing system, one or more terms in the code.

13. The method of claim 12 , wherein renaming, by the computing system, one or more terms in the code comprises uniform renaming across files.

14. The method of claim 11 , wherein the data assets comprise anchoring landmark data associated with a location to render a rendering effect.

Assignees

• c/o Google LLC
• Google Llc

Inventors

• Neulander Ivan
• Roth Ian Joseph
• Wang Hao
• Venezuela Agustin Iii Olivan
• Mathur Subramanian Shyamsunder
• Zhao Xuemei
• Koshi Valdrin
• Singh James Sraw

Classifications

• G06T15/005

  General purpose rendering architectures · CPC title

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06F11/3668

  Testing of software · CPC title

• G06F21/14 Primary

  against software analysis or reverse engineering, e.g. by obfuscation · CPC title

• G06T15/80

  Shading · CPC title