Augmented reality object rendering based on camera quality

Patent metadata

Field	Value
Publication number	US-2025153754-A1
Application number	US-202519024189-A
Country	US
Kind code	A1
Filing date	Jan 16, 2025
Priority date	Jun 17, 2022
Publication date	May 15, 2025
Grant date	—

Abstract

Systems and embodiments herein describe an augmented reality (AR) object rendering system. The AR object rendering system receives an image, generates a set of noise parameters and a set of blur parameters for the image using a neural network trained on a paired dataset of images, identifies an AR object associated with the image, modifies the AR object using the set of noise parameters and the set of blur parameters, displays the modified augmented reality object within the image.

First claim

What is claimed is:

1 . A method comprising:
accessing an image from a computing device; generating a set of blur parameters for the image using a blur estimation neural network, the blur estimation neural network trained based on a first dataset of clear images and a second dataset of blurry images, the blur estimation neural network trained to estimate a blur of a blurry image and generate a modified clear image by applying the estimated blur to a clear image; identifying an augmented reality object associated with the image; modifying the augmented reality object using the generated set of blur parameters; and causing display, on a graphical user interface of the computing device, of the modified augmented reality object within the image.

2 . The method of claim 1 , further comprising:
training a blur discriminator to identify images with a same blur; and comparing the modified clear image with the blurry image using the blur discriminator.

3 . The method of claim 1 , further comprising:
applying a high-pass filter to the modified clear image and the blurry image to separate image brightness from image blurriness in the modified clear image and the blurry image.

4 . The method of claim 1 , wherein the blur estimation neural network is trained with a blur discriminator that is removed from the blur estimation neural network prior to generating the set of blur parameters.

5 . The method of claim 1 , further comprising:
causing display, on the graphical user interface of the computing device, a selectable user interface element to apply a blur effect, wherein the modified augmented reality object is displayed within the image in response to an interaction with the selectable user interface element.

6 . The method of claim 1 , further comprising:
generating a set of noise parameters for the image using a noise estimation neural network, the noise estimation neural network trained based on the first dataset of clear images and a third dataset of noisy images, the noise estimation neural network trained to estimate a noise of a noisy image and generate a second modified clear image by applying the estimated noise to the clear image; and modifying the augmented reality object using the generated set of noise parameters.

7 . The method of claim 6 , further comprising:
training a noise discriminator to identify images with a same noise; and comparing the second modified clear image with the noisy image using the noise discriminator.

8 . The method of claim 6 , wherein the blur estimation neural network is trained with a blur discriminator that is removed from the blur estimation neural network prior to generating the set of blur parameters.

9 . The method of claim 6 , further comprising:
generating a supervised dataset with known blur output results and known noise output results using the blur estimation neural network and the noise estimation neural network; and training an augmented reality object rendering network using the supervised dataset and the first dataset of clear images.

10 . The method of claim 1 , wherein the image is accessed from one or more image sensors of the computing device.

11 . A system comprising:
one or more processors; and memory storing instructions that, when executed by the one or more processors, cause the system to perform operations comprising:
accessing an image from a computing device;
generating a set of blur parameters for the image using a blur estimation neural network, the blur estimation neural network trained based on a first dataset of clear images and a second dataset of blurry images, the blur estimation neural network trained to estimate a blur of a blurry image and generate a modified clear image by applying the estimated blur to a clear image;
identifying an augmented reality object associated with the image;
modifying the augmented reality object using the generated set of blur parameters; and
causing display, on a graphical user interface of the computer device, of the modified augmented reality object within the image.

12 . The system of claim 11 , the operations further comprising:
training a blur discriminator to identify images with a same blur; and comparing the modified clear image with the blurry image using the blur discriminator.

13 . The system of claim 11 , the operations further comprising:
applying a high-pass filter to the modified clear image and the blurry image to separate image brightness from image blurriness in the modified clear image and the blurry image.

14 . The system of claim 11 , wherein the blur estimation neural network is trained with a blur discriminator that is removed from the blur estimation neural network prior to generating the set of blur parameters.

15 . The system of claim 11 , the operations further comprising:
causing display, on the graphical user interface of the computing device, a selectable user interface element to apply a blur effect, wherein the modified augmented reality object is displayed within the image in response to an interaction with the selectable user interface element.

16 . The system of claim 11 , the operations further comprising:
generating a set of noise parameters for the image using a noise estimation neural network, the noise estimation neural network trained based on the first dataset of clear images and a third dataset of noisy images, the noise estimation neural network trained to estimate a noise of a noisy image and generate a second modified clear image by applying the estimated noise to the clear image; and modifying the augmented reality object using the generated set of noise parameters.

17 . The system of claim 16 , the operations further comprising:
training a noise discriminator to identify images with a same noise; and comparing the second modified clear image with the noisy image using the noise discriminator.

18 . The system of claim 16 , wherein the blur estimation neural network is trained with a blur discriminator that is removed from the blur estimation neural network prior to generating the set of blur parameters.

19 . The system of claim 16 , the operations further comprising:
generating a supervised dataset with known blur output results and known noise output results using the blur estimation neural network and the noise estimation neural network; and training an augmented reality object rendering network using the supervised dataset and the first dataset of clear images.

20 . A non-transitory computer-readable storage medium comprising instructions that, when executed by a computing system, cause the computing system to perform operations comprising:
accessing an image from a computing device; generating a set of blur parameters for the image using a blur estimation neural network, the blur estimation neural network trained based on a first dataset of clear images and a second dataset of blurry images, the blur estimation neural network trained to estimate a blur of a blurry image and generate a modified clear image by applying the estimated blur to a clear image; identifying an augmented reality object associated with the image; modifying the augmented reality object using the generated set of blur parameters; and causing display, on a graphical user interface of the computer device, of the modified augmented reality object within the image.

Assignees

• Snap Inc

Inventors

• Demyanov Sergey
• Hu Yunqing
• Marton Istvan
• Ostashev Daniil
• Podkin Aleksei

Classifications

• B66F11/04

  for movable platforms or cabins, e.g. on vehicles, permitting workmen to place themselves in any desired position for carrying out required operations ({Working platforms on fork-lift trucks B66F9/127; } vehicle aspects of service vehicles B60P3/14; platforms for cleaning windows A47L3/02; devices for rescuing persons from buildings A62B1/02; liftable or lowerable platforms for use on ladders E06C7/16; maintenance travellers for bridges E01D19/10; scaffolds on an extensible sub-structure E04G1/22) · CPC title

• B62B2202/30

  Furniture · CPC title

• B62B5/0086

  arranged independently on either side of the transported load · CPC title

• B62B3/0618 Primary

  using fluid lifting mechanisms · CPC title

• A47B31/00

  Service or tea tables, trolleys, or wagons ({serving trays A47G23/06}; features relating to running gear or to movement by hand B62B) · CPC title