Dictionary creation device, image processing device, image processing system, dictionary creation method, image processing method, and program

Patent metadata

Field	Value
Publication number	US-9436981-B2
Application number	US-201214363367-A
Country	US
Kind code	B2
Filing date	Dec 10, 2012
Priority date	Dec 12, 2011
Publication date	Sep 6, 2016
Grant date	Sep 6, 2016

Abstract

A dictionary creation device including a blurred image generation unit which outputs a blurred image generated by performing a blurring process to a learning image together with a blur parameter indicating a blurring state of the blurred image, a patch pair generation unit which generates a restoration patch and a blurred patch as a patch pair that is composed of the patches located at the corresponding positions of the learning image and the blurred image, and a registration unit which associates the patch pair with a blur parameter corresponding to the blurred patch in the patch pair and registers them in a dictionary.

First claim

The invention claimed is:

1. A dictionary creation device comprising:
a processor configured to:
output a blurred image to a learning image together with a blur parameter indicating a blurring state of the blurred image;
generate a restoration patch and a blurred patch as a patch pair that is composed of the patches located at the corresponding positions of the learning image and the blurred image; and
associate the patch pair with the blur parameter and register the patch pair and the blur parameter in a dictionary.

2. The dictionary creation device according to claim 1 wherein the processor is further configured to generate the blurred image by performing at least one among a size reduction process, a lightness reducing process, a high frequency component eliminating process, and a posture variation process to the learning image and
the blur parameter includes a value indicating at least one among a scaling factor, lightness, a degree of blur, and a posture slope.

3. The dictionary creation device according to claim 1 wherein the processor is further configured to generate the patch pair by using the restoration patch that is generated by shifting the patch by one pixel column in the learning image.

4. An image processing device comprising:
a processor configured to:
estimate a value of a blur parameter indicating a blurring state with respect to an input image that is an image received from outside;
divide the input image and generate a plurality of input patches;
refer to a dictionary storing two or more sets of data in which a blurred patch generated from a learning image, a restoration patch, and the blur parameter of the blurred patch are associated with each other and select the restoration patch for each input patch based on a value indicating a feature similarity and a value indicating a similarity of the blur parameter between the input patch and the blurred patch in the dictionary; and
combine the restoration patches selected for each input patch and generate the restored image.

5. The image processing device according to claim 4 wherein the blur parameter includes values indicating a scaling factor, lightness, a degree of blur, and a posture slope.

6. The image processing device according to claim 5 wherein the processor is further configured to estimate the value of the blur parameter of the input image by
calculating the scaling factor based on an aspect ratio of the input image and an aspect ratio of the learning image,
calculating the lightness based on an average value of brightness values of the input image,
calculating the degree of blur based on a magnitude of a high frequency component of the input image, or
calculating the posture slope by estimating a direction.

7. The image processing device according to claim 5 wherein the processor is further configured to calculate the value indicating the similarity of the blur parameter by using a value weighted by a weighting factor set for each blur parameter.

8. The image processing device according to claim 7 wherein the processor is further configured to set a higher weighting factor for the scaling factor and the lightness than for the degree of blur and the posture slope.

9. The image processing device according to claim 7 wherein the processor is further configured to receive an input of the weighting factor set by a user.

10. The image processing device according to claim 4 wherein the processor is further configured to select the restoration patch in only a range in which it is shown that the blur parameter is similar that is determined by a comparison between a value indicating a similarity of the blur parameter and a predetermined threshold value.

11. An image processing system including a dictionary creation device and an image processing device wherein
the dictionary creation device configured to:
output a blurred image generated by performing a blurring process to a learning image together with a blur parameter indicating a blurring state of the blurred image,
generate a restoration patch and a blurred patch as a patch pair that is composed of the patches located at the corresponding positions of the learning image and the blurred image, and
associate the patch pair with the blur parameter and register the patch pair and the blur parameter in a dictionary and
the image processing device is configured to:
estimate value of a blur parameter indicating the blurring state with respect to an input image that is an image received from outside,
divide the input image and generate a plurality of input patches,
refer to the dictionary created by the dictionary creation device and select the restoration patch for each input patch based on a value indicating a feature similarity and a value indicating a similarity of the blur parameter between the input patch and the blurred patch in the dictionary, and
combine the restoration patches selected for each input patch and generate the restored image.

12. A method for creating a dictionary comprising:
outputting a blurred image generated by performing a blurring process to a learning image together with a blur parameter indicating a blurring state of the blurred image;
generating a restoration patch and a blurred patch as a patch pair that is composed of the patches located at the corresponding positions of the learning image and the blurred image; and
associating the patch pair with the blur parameter and register the patch pair and the blur parameter in a dictionary.

13. A method for processing an image comprising:
estimating a value of a blur parameter indicating a blurring state with respect to an input image that is an image received from outside;
dividing the input image and generating a plurality of input patches;
referring to a dictionary storing two or more sets of data in which a blurred patch that is a patch of a blurred image generated from a learning image, a restoration patch, and the blur each input patch based on a value indicating a feature similarity and a value indicating a similarity of the blur parameter between the input patch and the blurred patch in the dictionary; and
combining the restoration patches selected for each input patch and generating a restored image.

14. A non-transitory computer-readable storage medium storing a program for causing a computer to execute, comprising:
outputting a blurred image generated by performing a blurring process to a learning image together with a blur parameter indicating a blurring state of the blurred image;
generating a restoration patch and a blurred patch as a patch pair that is composed of the patches located at the corresponding positions of the learning image and the blurred image; and
associating the patch pair with the blur parameter and register the patch pair and the blur parameter in a dictionary.

15. Anon-transitory computer-readable storage medium storing a program for causing a computer to execute, comprising:
estimating a value of a blur parameter indicating a blurring state with respect to an input image that is an image received from outside;
dividing the input image and generating a plurality of input patches;
referring to a dictionary storing two or more sets of data in which a blurred patch that is a patch of a blurred image generated from a learning image, a restoration patch that is a patch of the learning image, and the blur parameter of the blurred patch are associated with each other and selecting the restoration patch for each input patch based on a value indicating a feature similarity and a value indicating a similarity of the blur parameter between the input patc

Assignees

• Shibata Takashi
• Iketani Akihiko
• Senda Shuji
• Nec Corp

Inventors

• Shibata Takashi
• Iketani Akihiko
• Senda Shuji

Classifications

• G06T2207/20081

  Training; Learning · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title

• G06N99/005

  Physics · mapped topic

• G06T5/003 Primary

  Physics · mapped topic

• G06K9/00275

  Physics · mapped topic