Image processing apparatus, image processing method, and storage medium

Patent metadata

Field	Value
Publication number	US-11055564-B2
Application number	US-201816201740-A
Country	US
Kind code	B2
Filing date	Nov 27, 2018
Priority date	Nov 30, 2017
Publication date	Jul 6, 2021
Grant date	Jul 6, 2021

Abstract

An image processing apparatus includes a setting unit, a detection unit, a matrix calculation unit, a calculation unit, an obtaining unit, a deriving unit, a patch correction unit, and a generation unit. The setting unit sets a target patch in an input image. The detection unit detects a plurality of similar patches in the input image. The matrix calculation unit calculates a covariance matrix representing correlation between pixels based on the plurality of similar patches. The calculation unit calculates eigenvalues and eigenvectors of the covariance matrix. The obtaining unit obtains a noise amount in the input image. The deriving unit derives a correction matrix based on the eigenvalues, the eigenvectors, the noise amount, and the number of similar patches. The patch correction unit corrects values of pixels in the similar patches based on the correction matrix. The generation unit generates an output image by combining the corrected similar patches.

First claim

What is claimed is:

1. An image processing apparatus comprising:
a processor coupled to a memory to perform operations including:
setting a target patch in an input image, wherein the set target patch includes a number of pixels corresponding to characteristics of a target pixel in the input image and a pixel group in a vicinity of the target pixel as the target patch relative to the target pixel,
detecting a plurality of similar patches which are similar to the target patch set in the input image, wherein the plurality of similar patches includes pixels,
calculating, as a matrix calculation, a covariance matrix representing correlation between pixels based on the detected plurality of similar patches,
calculating a plurality of eigenvalues and a plurality of eigenvectors of the calculated covariance matrix,
obtaining a noise amount in the input image,
deriving a correction matrix that is different from the calculated covariance matrix and, in a case where eigenvectors corresponding to eigenvalues which are larger than a threshold value, the derived correction matrix is based on the plurality of eigenvalues, the plurality of eigenvectors, the noise amount, a number of detected similar patches, the eigenvectors corresponding to the eigenvalues which are larger than the threshold value, and, in a case of correcting eigenvalues smaller than the threshold value, the corrected eigenvalues and eigenvectors corresponding to the corrected eigenvalues,
correcting, as a patch correction, values of pixels in at least one of the plurality of similar patches based on the derived correction matrix, and
generating an output image by combining the similar patches having pixels values corrected by the patch correction,
wherein the threshold value is set in accordance with the noise amount in the plurality of eigenvalues and the number of pixels included in the plurality of similar patches, and
wherein deriving includes correcting eigenvalues smaller than the threshold value and deriving the correction matrix based on the corrected eigenvalues and eigenvectors corresponding to the corrected eigenvalues.

2. The image processing apparatus according to claim 1 , wherein the threshold value further is set based on the number of detected similar patches.

3. The image processing apparatus according to claim 1 , wherein deriving includes correcting eigenvalues, which are equal to or smaller than the corrected eigenvalues, to values corresponding to the noise amount.

4. The image processing apparatus according to claim 1 , wherein deriving includes setting the threshold value smaller as the number of detected similar patches is larger.

5. The image processing apparatus according to claim 1 , wherein deriving includes correcting eigenvalues of the plurality of eigenvalues in accordance with the noise amount and the number of pixels included in the plurality of similar patches.

6. The image processing apparatus according to claim 1 , wherein deriving includes setting the threshold value, selecting eigenvectors by comparing the threshold value with individual eigenvalues, and deriving the correction matrix using the selected eigenvectors.

7. The image processing apparatus according to claim 1 , wherein deriving includes correcting eigenvalues using a function of converting an input eigenvalue into a corrected eigenvalue.

8. The image processing apparatus according to claim 7 , wherein the function of converting the input eigenvalue into the corrected eigenvalue has a characteristic corresponding to noise variance and the number of detected similar patches.

9. The image processing apparatus according to claim 8 , wherein deriving includes deriving a matrix represented by the following expression as the derived correction matrix:
H≡σ 2 ( E 1 E 2 . . . E M ) t diag(η 1 −1 ,η 2 −1 , . . . , η M −1 )( E 1 E 2 . . . E M )

where H denotes the derived correction matrix, a denotes a noise amount in the input image, t denotes a matrix transpose operator, M represents the number of detected similar patches, diag represent a square matrix function, and (E 1 E 2 . . . E M ) denotes the plurality of eigenvectors corresponding to a plurality of corrected eigenvalues (η 1 −1 , η 2 −1 , . . . , η M −1 ).

10. The image processing apparatus according to claim 9 , wherein the patch correction includes correcting a processing target patch in accordance with the following expression:
O i ≡P i −H ( P i −Q )
where i represents a number identifier of a patch to be corrected, O i denotes a corrected patch, P i denotes the processing target patch included in the plurality of similar patches, H denotes the derived correction matrix, and Q denotes an average patch generated by averaging a corresponding pixel group in the plurality of similar patches.

11. The image processing apparatus according to claim 1 , wherein deriving includes calculating a projection matrix as the derived correction matrix using the plurality of eigenvectors and the plurality of eigenvalues.

12. The image processing apparatus according to claim 1 , wherein obtaining includes storing, in a table, the noise amount corresponding to parameters obtained when an imaging apparatus captures the input image.

13. A method for an image processing apparatus, the method comprising:
setting a target patch in an input image, wherein the set target patch includes a number of pixels corresponding to characteristics of a target pixel in the input image and a pixel group in a vicinity of the target pixel as the target patch relative to the target pixel;
detecting a plurality of similar patches which are similar to the target patch set in the input image, wherein the plurality of similar patches includes pixels;
calculating, as a matrix calculation, a covariance matrix representing correlation between pixels based on the detected plurality of similar patches;
calculating a plurality of eigenvalues and a plurality of eigenvectors of the calculated covariance matrix;
obtaining a noise amount in the input image;
deriving a correction matrix that is different from the calculated covariance matrix and, in a case where eigenvectors corresponding to eigenvalues which are larger than a threshold value, the derived correction matrix is based on the plurality of eigenvalues, the plurality of eigenvectors, the noise amount, a number of detected similar patches, the eigenvectors corresponding to the eigenvalues which are larger than the threshold value, and, in a case of correcting eigenvalues smaller than the threshold value, the corrected eigenvalues and eigenvectors corresponding to the corrected eigenvalues;
correcting, as a patch correction, values of pixels in at least one of the plurality of similar patches based on the derived correction matrix; and
generating an output image by combining the similar patches having pixels values corrected by the patch correction,
wherein the threshold value is set in accordance with the noise amount in the plurality of eigenvalues and the number of pixels included in the plurality of similar patches, and
wherein deriving includes correcting eigenvalues smaller than the threshold value and deriving the correction matrix based on the corrected eigenvalues and eigenvectors corresponding to the corrected eigenvalues.

14. A non-transitory computer-readable storage medium storing a program to cause a computer to perform a method for an image processing apparatus, the method comprising:
setting a target patch in an input image, wherein the set target patch includes a number of pixels corresponding to characteristics of a target pixel in

Assignees

• Canon Kk

Inventors

• Yahata Kazuhiro

Classifications

• G06V10/30

  Noise filtering · CPC title

• G06V10/50

  by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis · CPC title

• G06V10/76

  based on eigen-space representations, e.g. from pose or different illumination conditions; Shape manifolds · CPC title

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title

• G06T2207/20076

  Probabilistic image processing · CPC title