Image enhancement using one or more neural networks

Patent metadata

Field	Value
Publication number	US-12400291-B2
Application number	US-202117406922-A
Country	US
Kind code	B2
Filing date	Aug 19, 2021
Priority date	Sep 3, 2020
Publication date	Aug 26, 2025
Grant date	Aug 26, 2025

Abstract

Apparatuses, systems, and techniques are presented to generate images with one or more visual effects applied. In at least one embodiment, one or more visual effects are applied to one or more images having a resolution that is less than a first resolution and those visual effects approximated for one or more images having a resolution that is greater than or equal to the first resolution.

First claim

What is claimed is:

1. A method of image processing comprising:
obtaining initial training images;
generating blurred training images comprising intentionally blurring one or more of the initial training images;
generating low resolution training images or high resolution training images or both and comprising using the blurred training images;
training a neural network to generate blurred high resolution pixel values comprising using the low resolution training images or high resolution training images or both; and
inputting lower resolution images to the neural network while separately generating weights or biases or both of the neural network based on statistics of the low resolution images.

2. The method of claim 1 , further comprising:
blurring the initial training images comprising changing image data of the initial training images.

3. The method of claim 1 , further comprising:
blurring the initial training images comprising applying a function-related value to image data of the initial training images.

4. The method of claim 1 , further comprising:
training the neural network to convert a resolution on input blurred lower resolution images.

5. The method of claim 1 , further comprising:
processing a blurred image to generate a desired size of a lower resolution training image or higher resolution training image or both.

6. At least one non-transitory article having at least one computer readable medium comprising a plurality of instructions that in response to being executed on a computing device, cause the computing device to operate by:
obtaining initial training images;
generating blurred training images comprising intentionally blurring one or more of the initial training images;
generating lower resolution training images or high resolution training images or both and comprising using the blurred training images;
training a neural network to generate blurred higher resolution pixel values comprising
using the lower resolution training images or higher resolution training images or both; and
inputting lower resolution images to the neural network while separately generating weights or biases or both of the neural network based on statistics of the low resolution images.

7. The at least one non-transitory article of claim 6 , wherein the instructions, in response to being executed, further cause the computing device to operate by generating a factor to compensate for reducing a size of the initial training images.

8. The at least one non-transitory article of claim 7 , wherein the factor is used to convolve the initial training image to form the blurred training images.

9. The at least one non-transitory article of claim 6 , wherein generating the blurred training images comprises intentionally blurring less than all of one or more of the initial training images.

10. A method, comprising:
generating one or more blurred images by, at least, blurring pixel values of one or more training images;
generating at least lower resolution training images or high resolution training images or both and comprising using the blurred images;
training a neural network to generate blurred images at a higher resolution using at least the lower resolution training images or higher resolution training images; and
inputting lower resolution images to the neural network while separately generating weights or biases or both of the neural network based on statistics of the low resolution images.

11. The method of claim 10 , further comprising: blurring the training images at least in part by changing image data of the training images.

12. The method of claim 10 , further comprising:
blurring the training images at least in part by applying a function-related value to image data of the training images.

13. The method of claim 10 , further comprising:
training the neural network to convert a resolution on input blurred lower resolution images.

14. The method of claim 10 , further comprising:
processing a blurred image to generate a desired size of at least one of a lower resolution training image or higher resolution training image.

15. The method of claim 10 , wherein a first parameterized function is determined for the one or more images having a resolution less than a first resolution, and wherein a second parameterized function is determined based on the first parameterized function and used to approximate one or more visual effects for the one or more training images having a resolution greater than or equal to the first resolution.

16. The method of claim 15 , further comprising:
applying one or more enhancements to the one or more training images, having a resolution greater than or equal to the first resolution, after one or more visual effects are approximated.

17. The method of claim 10 , further comprising:
downscaling the one or more training images, having less than a first resolution, to a resolution of an initial aliased training image generated by a rendering engine.

Assignees

• Nvidia Corp

Inventors

• Pottorff Robert
• Tarjan David
• Tao Andrew
• Catanzaro Bryan

Classifications

• G06T5/80

  Geometric correction · CPC title

• G06T5/73

  Deblurring; Sharpening · CPC title

• G06T5/70

  Denoising; Smoothing · CPC title

• G06T5/20

  using local operators · CPC title

• G06T15/503

  Blending, e.g. for anti-aliasing · CPC title