Volume denoising with feature selection

Patent metadata

Field	Value
Publication number	US-12450697-B2
Application number	US-202217958198-A
Country	US
Kind code	B2
Filing date	Sep 30, 2022
Priority date	Oct 1, 2021
Publication date	Oct 21, 2025
Grant date	Oct 21, 2025

Abstract

A system includes a hardware processor and a system memory storing software code and one or more machine learning (ML) models. The hardware processor is configured to execute the software code to train a first ML model of the one or more ML models as a denoising feature selector, generate, using the trained first ML model a plurality of candidate feature sets, and identify a best volumetric feature set of the plurality of candidate feature sets using a predetermined selection criterion. The hardware processor is further configured to execute the software code to train, using the identified best volumetric feature set, one of the first ML model or a second ML model of the one or more ML models as a denoiser, receive an image including noise due to rendering, and denoise, using the trained denoiser, the noise due to rendering to produce a denoised image.

First claim

What is claimed is:

1. A system comprising:
a hardware processor and a system memory storing a software code and one or more machine learning (ML) models;
the hardware processor configured to execute the software code to:
train a first ML model of the one or more ML models as a denoising feature selector;
generate, using the trained first ML model, a plurality of candidate feature sets;
identify a best volumetric feature set of the plurality of candidate feature sets using a predetermined selection criterion;
train, using the identified best volumetric feature set, one of the first ML model or a second ML model of the one or more ML models as a denoiser;
receive a noisy image including a noise due to rendering; and
transform the noisy image to a denoised image;
wherein transforming the noisy image to the denoised image comprises:
decomposing color included in the noisy image into a surface contribution to the color and a volumetric contribution to the color;
denoising, using the trained denoiser, the volumetric contribution to the color to provide a denoised volumetric color result;
separately denoising the surface contribution to the color to provide a denoised surface color result; and
combining the denoised surface color result with the denoised volumetric color result.

2. The system of claim 1 , wherein the first ML model is trained as the denoiser.

3. The system of claim 1 , wherein the one or more ML models include the first ML model and the second ML model, and wherein the second ML model is trained as the denoiser.

4. The system of claim 1 , wherein generating the plurality of candidate feature sets comprises generating candidate feature sets of different sizes.

5. The system of claim 1 , wherein generating the plurality of candidate feature sets comprises generating candidate feature sets of progressively increasing size.

6. The system of claim 1 , wherein the predetermined selection criterion is a smallest denoising error.

7. The system of claim 1 , wherein the predetermined selection criterion is a predetermined balance between a denoising quality and a volumetric feature set size.

8. A method for use by a system including a hardware processor and a system memory storing a software code and one or more machine learning (ML) models, the method comprising:
training, by the software code executed by the hardware processor, a first ML model of the one or more ML models as a denoising feature selector;
generating, by the software code executed by the hardware processor and using the trained first ML model, a plurality of candidate feature sets;
identifying, by the software code executed by the hardware processor, a best volumetric feature set of the plurality of candidate feature sets using a predetermined selection criterion;
training, by the software code executed by the hardware processor and using the identified best volumetric feature set, one of the first ML model or a second ML model of the one or more ML models as a denoiser;
receiving a noisy image, by the software code executed by the hardware processor, the noisy image including a noise due to rendering; and
transforming, by the software code executed by the hardware processor, the noisy image to a denoised image
wherein transforming the noisy image to the denoised image comprises:
decomposing color included in the noisy image into a surface contribution to the color and a volumetric contribution to the color;
denoising, using the trained denoiser, the volumetric contribution to the color to provide a denoised volumetric color result;
separately denoising the surface contribution to the color to provide a denoised surface color result; and
combining the denoised surface color result with the denoised volumetric color result.

9. The method of claim 8 , wherein the first ML model is trained as the denoiser.

10. The method of claim 8 , wherein the one or more ML models include the first ML model and the second ML model, and wherein the second ML model is trained as the denoiser.

11. The system of claim 8 , wherein generating the plurality of candidate feature sets comprises generating candidate feature sets of different sizes.

12. The method of claim 8 , wherein generating the plurality of candidate feature sets comprises generating candidate feature sets of progressively increasing size.

13. The method of claim 8 , wherein the predetermined selection criterion is a smallest denoising error.

14. The method of claim 8 , wherein the predetermined selection criterion is a predetermined balance between a denoising quality and a volumetric feature set size.

15. A computer-readable non-transitory storage medium having stored thereon instructions, which when executed by a hardware processor, instantiates a method comprising:
training a first ML model of the one or more ML models as a denoising feature selector;
generating, using the trained first ML model, a plurality of candidate feature sets;
identifying a best volumetric feature set of the plurality of candidate feature sets using a predetermined selection criterion;
training, using the identified best volumetric feature set, one of the first ML model or a second ML model of the one or more ML models as a denoiser;
receiving a noisy image including a noise due to rendering; and
transforming the noisy image to a denoised image;
wherein transforming the noisy image to the denoised image comprises:
decomposing color included in the noisy image into a surface contribution to the color and a volumetric contribution to the color;
denoising, using the trained denoiser, the volumetric contribution to the color to provide a denoised volumetric color result;
separately denoising the surface contribution to the color to provide a denoised surface color result; and
combining the denoised surface color result with the denoised volumetric color result.

16. The computer-readable non-transitory storage medium of claim 15 , wherein the first ML model is trained as the denoiser.

17. The computer-readable non-transitory storage medium of claim 15 , wherein the one or more ML models include the first ML model and the second ML model, and wherein the second ML model is trained as the denoiser.

18. The computer-readable non-transitory storage medium of claim 15 , wherein generating the plurality of candidate feature sets comprises generating candidate feature sets of different sizes.

19. The computer-readable non-transitory storage medium of claim 15 , wherein generating the plurality of candidate feature sets comprises generating candidate feature sets of progressively increasing size.

20. The computer-readable non-transitory storage medium of claim 15 , wherein the predetermined selection criterion is one of a smallest denoising error or a predetermined balance between a denoising quality and a volumetric feature set size.

Assignees

• Disney Entpr Inc
• Eth Zuerich Eidgenoessische Technische Hochschule Zuerich
• Eth Zurich Eidgenoessische Technische Hochschule Zuerich

Inventors

• Papas Marios
• Zhang Xianyao
• Ott Melvin
• Manzi Marco

Classifications

• G06N20/20

  Ensemble learning · CPC title

• G06T2207/20081

  Training; Learning · CPC title

• G06T5/60

  using machine learning, e.g. neural networks · CPC title

• G06N20/00

  Machine learning · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title