Augmented reality object rendering based on camera quality

Patent metadata

Field	Value
Publication number	US-2024246590-A1
Application number	US-202418625568-A
Country	US
Kind code	A1
Filing date	Apr 3, 2024
Priority date	Jun 17, 2022
Publication date	Jul 25, 2024
Grant date	—

Abstract

Systems and embodiments herein describe an augmented reality (AR) object rendering system. The AR object rendering system receives an image, generates a set of noise parameters and a set of blur parameters for the image using a neural network trained on a paired dataset of images, identifies an AR object associated with the image, modifies the AR object using the set of noise parameters and the set of blur parameters, displays the modified augmented reality object within the image.

First claim

What is claimed is:

1 . A method comprising:
accessing an image from a computing device; generating a set of noise parameters using a noise estimation neural network trained to:
receive a first dataset of clear images and a second dataset of noisy images;
for each clear image in the first dataset of clear images and for each noisy image in the second dataset of noisy images:
estimate random gaussian noise of the noisy image;
generate a modified clear image, the modified clear image generated by applying the estimated random gaussian noise to the clear image; and
compare the modified clear image with the noisy image using a noise discriminator network;

generating a set of blur parameters for the image; identifying an augmented reality object associated with the image; receiving a first selection of a first selectable user interface element corresponding to the generated set of noise parameters; receiving a second selection of a second selectable user interface element corresponding to the generated set of blur parameters; in response to the first selection and the second selection, modifying the augmented reality object using the generated set of noise parameters and the generated set of blur parameters; and causing display, on a graphical user interface of the computer device, of the modified augmented reality object within the image.

2 . The method of claim 1 , wherein modifying the augmented reality object further comprises:
first modifying the augmented reality object using the set of noise parameters; and subsequently modifying the augmented reality object using the set of blur parameters.

3 . The method of claim 1 , wherein the image is accessed from one or more image sensors of a computer device.

4 . The method of claim 1 , wherein the noise estimation neural network is trained in a generative adversarial network (GAN) style.

5 . The method of claim 1 , wherein generating the set of blur parameters further comprises:
training a blur estimation neural network, the training comprising: receiving the first dataset of clear images and a third dataset of blurry images; for each clear image in the first dataset of clear images and for each blurry image in the third dataset of blurry images: estimating random gaussian blur of the blurry image; generating a modified clear image, the modified clear image generated by applying the estimated random gaussian blur to the clear image; and comparing the modified clear image with the blurry image using a blur discriminator network.

6 . The method of claim 5 , wherein the blur estimation neural network is trained in a GAN style.

7 . The method of claim 1 , wherein the first selectable user interface element and the second selectable user interface element are toggle buttons.

8 . The method of claim 5 , wherein generating the modified clear image further comprises:
generating an adjusted image, the adjusted image generated by passing the modified clear image to a high pass filter; and comparing the adjusted image with the blurry image using the blur discriminator.

9 . The method of claim 8 , wherein the high pass filter disentangles amounts of brightness of the modified clear image and the blurry image from amounts of blur of the modified clear image and the blurry image.

10 . A computing system comprising:
a processor; and a memory storing instructions that, when executed by the processor, configure the system to perform operations comprising:
accessing an image from a computing device;
generating a set of noise parameters using a noise estimation neural network trained to:
receive a first dataset of clear images and a second dataset of noisy images;
for each clear image in the first dataset of clear images and for each noisy image in the second dataset of noisy images:
estimate random gaussian noise of the noisy image;
generate a modified clear image, the modified clear image generated by applying the estimated random gaussian noise to the clear image; and
compare the modified clear image with the noisy image using a noise discriminator network;

generating a set of blur parameters for the image; identifying an augmented reality object associated with the image; receiving a first selection of a first selectable user interface element corresponding to the generated set of noise parameters; receiving a second selection of a second selectable user interface element corresponding to the generated set of blur parameters; in response to the first selection and the second selection, modifying the augmented reality object using the generated set of noise parameters and the generated set of blur parameters; and causing display, on a graphical user interface of the computer device, of the modified augmented reality object within the image.

11 . The computing system of claim 10 , wherein modifying the augmented reality object further comprises:
first modifying the augmented reality object using the set of noise parameters; and subsequently modifying the augmented reality object using the set of blur parameters.

12 . The computing system of claim 10 , wherein the image is accessed from one or more image sensors of a computer device.

13 . The computing system of claim 10 , wherein the noise estimation neural network is trained in a generative adversarial network (GAN) style.

14 . The computing system of claim 10 , wherein generating the set of blur parameters further comprises:
training a blur estimation neural network, the training comprising: receiving the first dataset of clear images and a third dataset of blurry images; for each clear image in the first dataset of clear images and for each blurry image in the third dataset of blurry images: estimating random gaussian blur of the blurry image; generating a modified clear image, the modified clear image generated by applying the estimated random gaussian blur to the clear image; and comparing the modified clear image with the blurry image using a blur discriminator network.

15 . The computing system of claim 14 , wherein the blur estimation neural network is trained in a GAN style.

16 . The computing system of claim 10 , wherein the first selectable user interface element and the second selectable user interface element are toggle buttons.

17 . The computing system of claim 14 , wherein generating the modified clear image further comprises:
generating an adjusted image, the adjusted image generated by passing the modified clear image to a high pass filter; and comparing the adjusted image with the blurry image using the blur discriminator.

18 . The computing system of claim 17 , wherein the high pass filter disentangles amounts of brightness of the modified clear image and the blurry image from amounts of blur of the modified clear image and the blurry image.

19 . A non-transitory computer-readable storage medium, the computer-readable storage medium including instructions that when executed by a computer, cause the computer to perform operations comprising:
accessing an image from a computing device; generating a set of noise parameters using a noise estimation neural network trained to:
receive a first dataset of clear images and a second dataset of noisy images;
for each clear image in the first dataset of clear images and for each noisy image in the second dataset of noisy images:
estimate random gaussian noise of the noisy image;
generate a modified clear image, the modified clear image generated by applying the estimated random

Assignees

• Snap Inc

Inventors

• Demyanov Sergey
• Hu Yunqing
• Marton Istvan
• Ostashev Daniil
• Podkin Aleksei

Classifications

• B66F11/04

  for movable platforms or cabins, e.g. on vehicles, permitting workmen to place themselves in any desired position for carrying out required operations ({Working platforms on fork-lift trucks B66F9/127; } vehicle aspects of service vehicles B60P3/14; platforms for cleaning windows A47L3/02; devices for rescuing persons from buildings A62B1/02; liftable or lowerable platforms for use on ladders E06C7/16; maintenance travellers for bridges E01D19/10; scaffolds on an extensible sub-structure E04G1/22) · CPC title

• B62B2202/30

  Furniture · CPC title

• B62B5/0086

  arranged independently on either side of the transported load · CPC title

• B62B3/0618 Primary

  using fluid lifting mechanisms · CPC title

• A47B31/00

  Service or tea tables, trolleys, or wagons ({serving trays A47G23/06}; features relating to running gear or to movement by hand B62B) · CPC title