Machine learning to process Monte Carlo rendered images

Patent metadata

Field	Value
Publication number	US-10192146-B2
Application number	US-201715840754-A
Country	US
Kind code	B2
Filing date	Dec 13, 2017
Priority date	Apr 30, 2015
Publication date	Jan 29, 2019
Grant date	Jan 29, 2019

Abstract

A method of rendering an image includes Monte Carlo rendering a scene to produce a noisy image. The noisy image is processed to render an output image. The processing applies a machine learning model that utilizes colors and/or features from the rendering system for denoising the noisy image and/or to for adaptively placing samples during rendering.

First claim

The invention claimed is:

1. A method of producing an output image, the method comprising:
obtaining training images;
using machine learning incorporating a filter on the training images to output final filter parameters, wherein the using machine learning comprises training a neural network, and the training comprises:
extracting, determining and/or computing features from the training images;
computing test filter parameters using a machine learning model including applying the filter using the features to create a denoised image;
applying an error metric to the denoised image;
correcting the machine learning model based on the error metric including updating the testing filter parameters;
repeating the computing, the applying and the correcting to determine final filter parameters;

receiving a Monte Carlo rendered image that has noise;
executing the filter on the noisy image using the final filter parameters to generate an output image.

2. The method of claim 1 , wherein the training images include both ground truth training images and noisy training images.

3. The method of claim 1 wherein the extracting, determining and/or computing features includes:
determining primary features of the training images;
extracting and/or computing secondary features of the training images using the primary features.

4. The method of claim 3 , wherein the primary features include features selected from the group consisting: positions, colors, world positions, visibility, shading normals, texture values.

5. The method of claim 4 , wherein the secondary features include features selected from the group consisting of: variances and noise approximation in local regions, mean of primary features at various block sizes, standard deviation of the primary features at various block sizes, gradients of primary features, mean deviation of the primary features, median absolute deviation (MAD) of primary features, sampling rate.

6. The method of claim 1 , wherein the training images comprise ground truth sample images.

7. The method of claim 1 , wherein the filter comprises a cross-bilateral filter.

8. The method of claim 1 , wherein the filter comprises a cross non-local means filter.

9. The method of claim 1 , wherein the neural network is a one of a support vector machine, a random forest, deep neural network, multi-layer perceptron, convolutional network, deep convolutional network, recurrent neural network, autoencoder neural network, long short-term memory networks, and generative adversarial network.

10. The method of claim 1 , wherein the features comprise color, illumination or texture.

Assignees

• Univ California

Inventors

• Sen Pradeep
• Bako Steve
• Kalantari Nima Khademi

Classifications

• G06N3/084 Primary

  Backpropagation, e.g. using gradient descent · CPC title

• G06V10/774

  Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06V10/82

  using neural networks · CPC title

• G06F18/214

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06N7/01

  Probabilistic graphical models, e.g. probabilistic networks · CPC title