System and method for image processing using automatically estimated tuning parameters

Patent metadata

Field	Value
Publication number	US-10032256-B1
Application number	US-201615355777-A
Country	US
Kind code	B1
Filing date	Nov 18, 2016
Priority date	Nov 18, 2016
Publication date	Jul 24, 2018
Grant date	Jul 24, 2018

Abstract

Imaging processing techniques using a trained convolution neural network (CNN) are described. In one or more implementations, an image processing system and method are provided for applying an image processing algorithm to a dataset of training images to generate a plurality of performance curves, constructing a loss function based upon the plurality of performance curves, training a convolutional neural network (CNN) to optimize the loss function to establish a trained convolutional neural network, predicting a specific tuning parameter for an image of interest using the trained convolution neural network and performing image processing of the image of interest using the specific tuning parameter and the image processing algorithm to generate a processed image of interest.

First claim

What is claimed is:

1. An image processing method, the method comprising:
dividing each training image of a dataset of training images into a plurality of training images patches;
applying an image processing algorithm, over a range of possible tuning parameters, to each of the plurality of training image patches to generate a plurality of performance curves;
constructing a loss function based upon the plurality of performance curves;
training a convolutional neural network (CNN) to optimize the loss function to establish a trained convolutional neural network;
predicting a specific tuning parameter for an image of interest using the trained convolutional neural network; and
performing image processing of the image of interest using the specific tuning parameter and the image processing algorithm to generate a processed image of interest.

2. The method of claim 1 , further comprising:
dividing the image of interest into a plurality of image patches, and wherein predicting a specific tuning parameter for an image of interest using the trained convolutional neural network further comprises predicting a specific tuning parameter for each of the image patches of the image of interest using the trained convolutional neural network.

3. The method of claim 2 , wherein performing image processing of the image of interest using the specific turning parameter and the image processing algorithm to generate a processed image of interest further comprises, performing image processing of the image of interest using the specific turning parameter of each of the image patches of the image of interest and the image processing algorithm to generate a plurality of processed image patches and combining the plurality of processed image patches to generate the processed image of interest.

4. The method of claim 1 , wherein the image processing algorithm is selected from a denoising algorithm, an image super-resolution imaging algorithm, an image segmentation algorithm and an image inpainting algorithm.

5. The method of claim 1 , wherein the image of interest is a noisy image of interest and wherein the processed image of interest is a denoised image of interest.

6. The method of claim 1 , wherein the image processing algorithm is a denoising algorithm selected from a Block-Matching and 3D Filtering (BM3D) denoising algorithm, an Active Random Field algorithm, a Bi-Level Optimization Algorithm and a Multi-Layer Perceptron algorithm.

7. The method of claim 1 , wherein the loss function is a function for maximizing peak signal-to-noise ratio (PSNR) loss over the dataset of training images.

8. The method of claim 1 , wherein the loss function is a function for the maximizing variance over the dataset of training images.

9. An image denoising method, the method comprising:
dividing each training image of a dataset of training images into a plurality of training images patches;
applying an image denoising algorithm, over a range of possible tuning parameters, to each of the plurality of training image patches to generate a plurality of performance curves;
constructing a loss function based upon the plurality of performance curves;
training a convolutional neural network (CNN) to optimize the loss function to establish a trained convolutional neural network;
dividing a noisy image of interest into a plurality of image patches;
predicting a specific tuning parameter for each of the image patches of the noisy image of interest using the trained convolutional neural network;
performing image denoising of each of the image patches of the noisy image of interest using the specific tuning parameter of each of the image patches of the noisy image of interest and the image processing algorithm to generate a plurality of denoised image patches; and
combining the plurality of denoised image patches to generate a denoised image of interest.

10. The method of claim 9 , wherein the image denoising algorithm is selected from a Block-Matching and 3D Filtering (BM3D) denoising algorithm, an Active Random Field algorithm, a Bi-Level Optimization Algorithm and a Multi-Layer Perceptron algorithm.

11. The method of claim 9 , wherein the loss function is a function for maximizing peak signal-to-noise ratio (PSNR) loss over the dataset of training images.

12. The method of claim 9 , wherein the loss function is a function for the maximizing variance over the dataset of training images.

13. A system for image processing, the system comprising:
one or more modules implemented at least partially in hardware, the one or more modules configured to perform operations comprising:
dividing each training image of a dataset of training images into a plurality of training images patches;
applying an image processing algorithm, over a range of possible tuning parameters, to the plurality of training images patches to generate a plurality of performance curves;
constructing a loss function based upon the plurality of performance curves;
training a convolutional neural network (CNN) to optimize the loss function to establish a trained convolutional neural network;
predicting a specific tuning parameter for an image of interest using the trained convolutional neural network, and
performing image processing of the image of interest using the specific tuning parameter and the image processing algorithm to generate a processed image of interest.

14. The system of claim 13 , wherein the image processing algorithm is selected from a denoising algorithm, an image super-resolution imaging algorithm, an image segmentation algorithm and an image inpainting algorithm.

15. The system of claim 13 , wherein the image of interest is a noisy image of interest and wherein the processed image of interest is a denoised image of interest.

16. The system of claim 13 , wherein the image processing algorithm is a denoising algorithm selected from a Block-Matching and 3D Filtering (BM3D) denoising algorithm, an Active Random Field algorithm, a Bi-Level Optimization Algorithm and a Multi-Layer Perceptron algorithm.

17. The system of claim 13 , wherein the dataset of training images is a RENOIR dataset of training images.

18. The system of claim 13 , wherein the loss function is a function for maximizing peak signal-to-noise ratio (PSNR) loss over the dataset of training images.

19. The system of claim 13 , wherein the loss function is a function for the maximizing variance over the dataset of training images.

Assignees

• Univ Florida State Res Found Inc

Inventors

• Anaya Josue
• Barbu Adrian

Classifications

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title

• G06T5/002 Primary

  Physics · mapped topic

• G06T7/0081

  Physics · mapped topic

• G06T2207/20182

  Noise reduction or smoothing in the temporal domain; Spatio-temporal filtering · CPC title

• G06T3/4053

  based on super-resolution, i.e. the output image resolution being higher than the sensor resolution · CPC title