Machine learning to process Monte Carlo rendered images

Patent metadata

Field	Value
Publication number	US-10832091-B2
Application number	US-201816219340-A
Country	US
Kind code	B2
Filing date	Dec 13, 2018
Priority date	Apr 30, 2015
Publication date	Nov 10, 2020
Grant date	Nov 10, 2020

Abstract

A method of rendering an image includes Monte Carlo rendering a scene to produce a noisy image. The noisy image is processed to render an output image. The processing applies a machine learning model that utilizes colors and/or features from the rendering system for denoising the noisy image and/or to for adaptively placing samples during rendering.

First claim

The invention claimed is:

1. A computer-implemented method of rendering an image, the method comprising: Monte Carlo rendering a scene with a rendering system to produce a noisy image; processing the noisy image to render an output image, wherein said processing comprises applying a machine learning model that utilizes colors and/or features from the rendering system for denoising the noisy image and/or for adaptively placing samples during rendering, wherein the applying machine learning comprises:
applying an error metric to measure the distance between filtered images and ground truth images;
applying backpropagation to minimize an energy function on results of the error metric.

2. The method of claim 1 , wherein the machine learning model had been trained with ground truth sample images prior to the applying.

3. The method of claim 1 wherein the denoising uses sample colors from the rendering system.

4. The method of claim 1 wherein the denoising uses sample features from the rendering system.

5. The method of claim 1 wherein the applying the machine learning comprises applying a machine learning algorithm directly to the noisy image to compute denoised pixel values of the output image.

6. The method of claim 4 , where the machine learning algorithm uses secondary features derived from the colors and/or features from the rendering system to compute the denoised pixel values of the output image.

7. The method of claim 1 wherein the denoising is implemented with an explicit filter and the applying machine learning comprises obtaining optimal parameters for the filter.

8. The method of claim 7 wherein the explicit filter comprises a cross-bilateral filter and the applying machine learning comprises obtaining optimal parameters for the cross-bilateral filter.

9. The method of claim 1 wherein the explicit filter comprises cross non-local means filter and the applying machine learning comprises obtaining optimal parameters for the cross non-local means filter.

10. The method of claim 1 , wherein the neural network is a one of a neural network, a support vector machine, a random forest, deep neural network, multi-layer perceptron, convolutional network, deep convolutional network, recurrent neural network, autoencoder neural network, long short-term memory networks, and generative adversarial network.

11. The method of claim 1 , wherein the features from the rendering system comprise illumination or texture.

Assignees

• Univ California

Inventors

• Sen Pradeep
• Bako Steve
• Kalantari Nima Khademi

Classifications

• G06N3/084 Primary

  Backpropagation, e.g. using gradient descent · CPC title

• G06V10/774

  Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06V10/82

  using neural networks · CPC title

• G06F18/214

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06N7/01

  Probabilistic graphical models, e.g. probabilistic networks · CPC title