Degradation restoration system, degradation restoration method and program

Patent metadata

Field	Value
Publication number	US-9280807-B2
Application number	US-201214233973-A
Country	US
Kind code	B2
Filing date	Jun 21, 2012
Priority date	Jul 21, 2011
Publication date	Mar 8, 2016
Grant date	Mar 8, 2016

Abstract

For each pixel of an input image, a direction toward which variation in pixel value between the relevant pixel and its peripheral pixels is largest and an amount of the variation in pixel value are computed, where a direction unit vector that indicates the direction toward which the variation is largest and a variation vector that represents the amount of the largest variation in the pixel value are computed. For each pixel, a regularization strength is computed utilizing the direction unit vector, the variation vector, and a regularization strength computing formula so that the larger the amount of the variation in pixel value toward a direction indicated by the direction unit vector, the less the regularization strength. An optimization function is determined based on the input image and the regularization strength for each pixel thereof and assigns a value for each pixel, by which the determined optimization function has the minimum value, to the corresponding pixel of a restored image to be generated.

First claim

The invention claimed is:

1. A degradation restoration system that generates a restored image based on an optimization function which includes a plurality of regularization terms, that reduce variation in pixel value of the restored image toward different directions assigned to the individual regularization terms, and an error term where the smaller the difference between an input image and an image obtained by virtually degrading the restored image in accordance with a pre-assumed degradation step, the less the value of the error term, wherein the system comprises:
a variation computation unit that computes, for each pixel of the input image, a direction toward which variation in pixel value between the relevant pixel and its peripheral pixels is largest and an amount of the variation in pixel value between the relevant pixel and its peripheral pixels, where the variation computation unit computes a direction unit vector that indicates, for each pixel, the direction toward which the variation is largest and a variation vector that represents, for each pixel, the amount of the largest variation between the relevant pixel and its peripheral pixels;
a regularization strength computation unit that computes, for each pixel, a regularization strength utilizing the direction unit vector, the variation vector, and a regularization strength computing formula in a manner such that the greater the degree of coincidence between a direction toward which each regularization term reduces the variation in pixel value and a direction of the variation in pixel value, which is indicated by the direction unit vector computed by the variation computation unit, and also the larger the amount of the variation in pixel value, the less the regularization strength; and
an image restoration unit that determines the optimization function based on the input image and the regularization strength assigned to each pixel of the input image and assigns a value for each pixel, by which the determined optimization function has a minimum value, to the corresponding pixel of the restored image so as to generate the restored image.

2. The degradation restoration system in accordance with claim 1 , wherein:
the individual regularization terms reduce, not only the variation in pixel value of the restored image toward the different directions, but also variation in pixel value of the restored image for different scales assigned to the individual regularization terms; and
the variation computation unit computes, for each pixel, the amount of the variation in pixel value between the relevant pixel and its peripheral pixels for the different scales; and
in the computation manner of the regularization strength computation unit, not only the greater the degree of coincidence between the direction toward which each regularization term reduces the variation in pixel value and the direction of the variation in pixel value indicated by the direction unit vector computed by the variation computation unit and the larger the amount of the variation in pixel value, the less the regularization strength, but also the larger the scale determined by the variation computation unit, the less the regularization strength.

3. The degradation restoration system in accordance with claim 1 , further comprising:
a category classification unit that classifies the individual pixels of the input image into a plurality of predetermined categories,
wherein the regularization strength computation unit appropriately computes, for each pixel, the regularization strength in accordance with the category into which the pixel was classified by the category classification unit in a manner such that the larger the variation in pixel value toward the direction computed by the variation computation unit, the less the regularization strength.

4. The degradation restoration system in accordance with claim 3 , wherein:
the category classification unit classifies the individual pixels of the input image into the predetermined categories and computes reliability for each result of the category classification; and
the system further comprises:
a synthesis ratio computation unit that computes a synthesis ratio between the restored image and the input image based on the reliability computed by the category classification unit; and
an image synthesis unit that generates a synthesized image based on the input image input via an image input unit, the restored image restored by the image restoration unit, and the synthesis ratio computed by the synthesis ratio computation unit so that in the synthesized image, only a region having a relatively high degree of reliability is close to the corresponding region in the restored image.

5. A degradation restoration method utilized in a degradation restoration system that generates a restored image based on an optimization function which includes a plurality of regularization terms and an error term, wherein:
a variation computation step comprising computing, for each pixel of an input image, a direction toward which variation in pixel value between the relevant pixel and its peripheral pixels is largest and an amount of the variation in pixel value between the relevant pixel and its peripheral pixels, where the variation computation unit computes a direction unit vector that indicates, for each pixel, the direction toward which the variation is largest and a variation vector that represents, for each pixel, the amount of the largest variation between the relevant pixel and its peripheral pixels;
a regularization strength computation step comprising computing, for each pixel, a regularization strength utilizing the direction unit vector, the variation vector, and a regularization strength computing formula in a manner such that the larger the amount of the variation in pixel value toward a direction indicated by the direction unit vector computed by the variation computation unit, the less the regularization strength; and
an image restoration step comprising determining the optimization function based on the input image and the regularization strength assigned to each pixel of the input image and assigns a value for each pixel, by which the determined optimization function has a minimum value, to the corresponding pixel of the restored image so as to generate the restored image.

6. A non-transitory computer-readable medium having a program stored thereon that makes a computer in a degradation restoration system, that generates a restored image based on an optimization function which includes a plurality of regularization terms and an error term, execute:
a variation computation step that computes, for each pixel of an input image, a direction toward which variation in pixel value between the relevant pixel and its peripheral pixels is largest and an amount of the variation in pixel value between the relevant pixel and its peripheral pixels, where the variation computation step computes a direction unit vector that indicates, for each pixel, the direction toward which the variation is largest and a variation vector that represents, for each pixel, the amount of the largest variation between the relevant pixel and its peripheral pixels;
a regularization strength computation step that computes, for each pixel, a regularization strength utilizing the direction unit vector, the variation vector, and a regularization strength computing formula in a manner such that the larger the amount of the variation in pixel value toward a direction indicated by the direction unit vector computed by the variation computation step, the less the regularization strength; and
an image restoration step that determines the optimization function based on the input image and the regularization strength assigned to each pixel of the input image and assigns a value for each

Assignees

• Shibata Takashi
• Iketani Akihiko
• Senda Shuji
• Nec Corp

Inventors

• Shibata Takashi
• Iketani Akihiko
• Senda Shuji

Classifications

• H04N23/81

  for suppressing or minimising disturbance in the image signal generation · CPC title

• H04N5/217

  Electricity · mapped topic

• G06T5/003 Primary

  Physics · mapped topic

• G06T2207/20192

  Edge enhancement; Edge preservation · CPC title

• G06T5/002

  Physics · mapped topic