Image processing apparatus and method using machine learning

Patent metadata

Field	Value
Publication number	US-10360664-B2
Application number	US-201715441145-A
Country	US
Kind code	B2
Filing date	Feb 23, 2017
Priority date	Jan 12, 2017
Publication date	Jul 23, 2019
Grant date	Jul 23, 2019

Abstract

Disclosed are an image processing apparatus and an image processing method. The image processing method performed in the image processing apparatus comprises receiving a blurred image; generating an intermediate image from the blurred image, the intermediate image including artifacts with less sensitivity to changes in a blur kernel than the blurred image; and generating a first corrected image by removing the artifacts of the intermediate image using an artifact removal model. Therefore, it is made possible to learn how to remove artifacts of the intermediate image without the image processing apparatus learning a direct deconvolution method.

First claim

What is claimed is:

1. A machine learning method in which an image processing apparatus learns image processing, the method comprising:
receiving a clear test image;
generating an intermediate image from a blurred test image modified from the clear test image, the intermediate image including artifacts with less sensitivity to changes in a blur kernel than the blurred test image; and
generating an artifact removal model that removes artifacts of an image from a result of machine learning performed by using the intermediate image and the clear test image,
wherein the artifact removal model is generated from the result of the machine learning using a convolutional neural network which includes at least one convolution layer for generating a plurality of characteristic maps using a plurality of convolution filters.

2. The machine learning method according to claim 1 , further comprising generating the blurred test image from a sum of a convolution of the clear test image and the blur kernel and a noise.

3. The machine learning method according to claim 1 , wherein the intermediate image is obtained by applying a Wiener filter to the blurred test image.

4. The machine learning method according to claim 3 , wherein, in the generating the artifact removal model, a back propagation process of the convolutional neural network is performed in consideration of a regularization term obtained from a gradient of an output result of the convolutional neural network.

5. The machine learning method according to claim 1 , wherein the convolutional neural network includes at least one node for summing an output result of an intermediate layer of the convolutional neural network and an output result of a layer preceding the intermediate layer.

6. The machine learning method according to claim 1 , wherein the convolutional neural network includes a node for summing an output result of a last layer of the convolutional neural network and data of the intermediate image input to the convolutional neural network.

7. An image processing method performed in an image processing apparatus, the method comprising:
receiving a blurred image;
generating an intermediate image from the blurred image, the intermediate image including artifacts with less sensitivity to changes in a blur kernel than the blurred image; and
generating a first corrected image by removing the artifacts of the intermediate image using an artifact removal model,
wherein the artifact removal model removes the artifacts of the intermediate image by using a convolutional neural network which includes at least one convolution layer for generating a plurality of characteristic maps using a plurality of convolution filters.

8. The image processing method according to claim 7 , wherein the intermediate image is obtained by applying a Wiener filter to the blurred image.

9. The image processing method according to claim 7 , wherein the convolutional neural network includes at least one node for summing an output result of an intermediate layer of the convolutional neural network and an output result of a layer preceding the intermediate layer.

10. The image processing method according to claim 7 , wherein the convolutional neural network includes a node for summing an output result of a last layer of the convolutional neural network and data of the intermediate image input to the convolutional neural network.

11. The image processing method according to claim 7 , further comprising generating a second corrected image which corrects detail of the first corrected image, wherein the second corrected image satisfies a condition of minimizing a linear sum of a squared error of the blurred image and a convolution of the second corrected image and the blur kernel, and a squared error of the first corrected image and the second corrected image.

12. An image processing apparatus deconvoluting a blurred image, the apparatus comprising:
a processor; and
a memory storing at least one instruction executed by the processor,
wherein the at least one instruction is configured to receive a clear test image; generate, from a blurred test image modified from the clear test image, an intermediate training image with less sensitivity to changes in a blur kernel than the blurred test image; generate an artifact removal model from a result of machine learning performed by using the intermediate training image and the clear test image; generate, from the blurred image, an intermediate image including artifacts with less sensitivity to changes in a blur kernel than the blurred image; and generate a first corrected image by removing the artifacts of the intermediate image using the artifact removal model.

13. The image processing apparatus according to claim 12 , wherein the at least one instruction is further configured to remove the artifacts of the intermediate image by using a convolutional neural network, and the convolutional neural network includes a node for summing an output result of a last layer of the convolutional neural network and data of the intermediate image input to the convolutional neural network.

14. The image processing apparatus according to claim 12 , wherein the at least one instruction is further configured to remove the artifacts of the intermediate image by using a convolutional neural network, and the convolutional neural network includes at least one node for summing an output result of an intermediate layer of the convolutional neural network and an output result of a layer preceding the intermediate layer.

15. The image processing apparatus according to claim 12 , wherein the at least one instruction is further configured to generate a second corrected image which corrects detail of the first corrected image, wherein the second corrected image satisfies a condition of minimizing a linear sum of a squared error of the blurred image and a convolution of the second corrected image and the blur kernel, and a squared error of the first corrected image and the second corrected image.

Assignees

• Postech Acad Ind Found

Inventors

• Lee Seung Yong
• Son Hyeong Seok

Classifications

• G06V10/82

  using neural networks · CPC title

• F16M7/00 Primary

  Details of attaching or adjusting engine beds, frames, or supporting-legs on foundation or base; Attaching non-moving engine parts, e.g. cylinder blocks (elastic or equivalent mounting for absorbing vibrations F16F, especially F16F15/04) · CPC title

• F16M1/04

  for rotary engines or similar machines · CPC title

• G06T2207/10024

  Color image · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title