Method and device for image correction

Patent metadata

Field	Value
Publication number	US-10748252-B2
Application number	US-201615779901-A
Country	US
Kind code	B2
Filing date	Nov 29, 2016
Priority date	Dec 2, 2015
Publication date	Aug 18, 2020
Grant date	Aug 18, 2020

Abstract

Methods, apparatuses and computer programs for image processing are provided. A sequence of images, in particular, is processed in this case. The images are subdivided into tiles and the tiles are transformed into the frequency domain. By evaluating the argument of the spectral density in the frequency domain, it is possible to identify and rectify disturbances which, for example, are caused by air disturbances (flickering).

First claim

What is claimed is:

1. A method of image processing by a computing device, for a temporal sequence of images, the method comprising:
transforming a plurality of images of the temporal sequence of images into the spatial frequency space using a spectral transformation, to obtain a spectral density for each of the plurality of images,
identifying at least one shift by evaluating a frequency-proportional change of the argument of the spectral density over the plurality of images in spatial frequency space,
compensating the frequency-proportional change of the argument of the spectral density to compensate the shift, and
inverse transforming at least one of the plurality of images with compensated shift back into real space using the spectral transformation, to obtain at least one improved image.

2. The method of claim 1 , further comprising refraining from compensating the frequency-proportional change of the argument of the spectral density to compensate the shift if the shift has a known characteristic indicative that the shift is not to be compensated.

3. The method of claim 2 , wherein the known characteristic is a characteristic of a motion of an object.

4. The method of claim 1 , wherein the compensating further comprises calculating a mean value for the spectral density and using a known characteristic of at least one parameter of the shift as a function of time.

5. The method of claim 4 , wherein the known characteristic of the at least one parameter is a periodic characteristic, and wherein the compensating comprises removing the component proportional to frequency caused by the shift using a level fit in respect of the change of the argument of the spectral density from image to image in the plurality of images.

6. The method of claim 4 , wherein the known characteristic of the at least one parameter is linearly proportional to time and wherein the compensating comprises separately averaging the argument of the spectral density and the absolute value of the spectral density over time.

7. The method of claim 1 , further comprising subdividing each image of the plurality of images into a plurality of tiles, wherein the transforming comprises transforming the plurality of tiles for each image, and the identifying comprises evaluating the frequency-proportional change of the argument of the spectral density of corresponding tiles over the plurality of images, and where the inverse transformation comprises inverse transforming the plurality of tiles.

8. The method as claimed in claim 7 , wherein the subdividing, transforming, evaluating, compensating, and inverse transforming is repeated iteratively in N iterations, N > 1 , wherein a size of the one or more tiles is incrementally reduced for each iteration, wherein for each subsequent iteration the sequence of images is provided based on the at least one improved image resulting from the previous iteration,
wherein for each iteration the identifying of shifts and the compensating of shifts is carried out for the respective tile size, wherein the identifying and the compensating of the shifts is performed based on different tile sizes in the different iterations.

9. The method as claimed in claim 1 , wherein the images are color images, wherein the transforming, identifying, compensating, and inverse transforming are carried out separately for each color channel of the color images, wherein the compensation for the color channels is carried out separately on the basis of the results of the identifying.

10. The method as claimed in claim 1 , wherein the images are color images, wherein the method comprises averaging color channels of the color images to obtain grayscale values, where the identifying is performed on the basis of the grayscale values obtained by averaging color channels.

11. An apparatus for image processing, comprising:
a computing device having at least one processor and a memory, wherein a sequence of temporal images is storable in the memory, and wherein the processor is configured to:
receive the temporal sequence of images,
transform a plurality of images of the temporal sequence of images into the spatial frequency space using a spectral transformation, to obtain a spectral density for each of the plurality of images,
identify at least one shift by evaluating a frequency-proportional change of the argument of the spectral density over the plurality of images in spatial frequency space,
compensate the frequency-proportional change of the argument of the spectral density to compensate the shift, and
inverse transform at least one of the plurality of images with compensated shift back into real space using the spectral transformation, to obtain at least one improved image.

12. The apparatus as claimed in claim 11 , where the processor is further configured to refrain from compensating the frequency-proportional change of the argument of the spectral density to compensate the shift, if the shift has a known characteristic indicative that the shift is not to be compensated.

13. The apparatus as claimed in claim 12 , wherein the known characteristic is a characteristic of a motion of an object.

14. The apparatus as claimed in claim 11 , wherein, to compensate the shift, the processor is configured to calculate a mean value for the spectral density and use a known characteristic of at least one parameter of the shift as a function of time.

15. The apparatus as claimed in claim 14 , wherein the known characteristic of the at least one parameter is a periodic characteristic, and wherein, to compensate the shift, the processor is configured to remove the component proportional to frequency caused by the shift, using a level fit in respect of the change of the argument of the spectral density from image to image in the plurality of images.

16. The apparatus as claimed in claim 14 , wherein the known characteristic of the at least one parameter is linearly proportional to time and wherein, to compensate the shift, the processor is configured to separately average the argument of the spectral density and the absolute value of the spectral density over time.

17. The apparatus as claimed in claim 11 , where the processor is further configured to subdivide each image of the plurality of images into a plurality of tiles and transform the plurality of tiles for each image, and wherein, to identify the at least one shift, the processor is configured to evaluate the frequency-proportional change of the argument of the spectral density of corresponding tiles over the plurality of images, and further wherein, to inverse transform the at least one of the plurality of images, the processor is configured to inverse transform the plurality of tiles.

18. The apparatus as claimed in claim 17 , where the processor is further configured to perform a sharpening over the one or more tiles in spatial frequency space,
wherein the sharpening comprises amplifying absolute values of higher frequency spectral density components of the spectral density function in the spatial frequency space.

19. The apparatus as claimed in claim 17 , wherein the processor is further configured to: iteratively repeat subdividing, transforming, evaluating, compensating, and inverse transforming in N iterations, N>1,
reduce a size of the one or more tiles incrementally in each iteration, wherein for each subsequent iteration the processor is configured to:
provide the sequence of images based on the at least one improved image resulting from the previous iteration, and
perform the identifying of shifts and to carry ou

Assignees

• Zeiss Carl Ag

Inventors

• Derndinger Eberhard

Classifications

• G06F17/18

  for evaluating statistical data {, e.g. average values, frequency distributions, probability functions, regression analysis (forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04)} · CPC title

• G06T2207/20216

  Image averaging · CPC title

• G06T3/608 Primary

  by skew deformation, e.g. two-pass or three-pass rotation · CPC title

• G06T5/10

  using non-spatial domain filtering · CPC title

• G06T2207/20021

  Dividing image into blocks, subimages or windows · CPC title