Method of image evaluation for sim microscopy and sim microscopy method

Patent metadata

Field	Value
Publication number	US-11954831-B2
Application number	US-202117467912-A
Country	US
Kind code	B2
Filing date	Sep 7, 2021
Priority date	Sep 10, 2020
Publication date	Apr 9, 2024
Grant date	Apr 9, 2024

Abstract

A method of image evaluation when performing SIM microscopy on a sample includes: A) providing n raw images of the sample, which were each generated by illuminating the sample with an individually positioned SIM illumination pattern and imaging the sample in accordance with a point spread function, B) providing (S1) n illumination pattern functions, which each describe one of the individually positioned SIM illumination patterns, C) providing (S1) the point spread function and D) Carrying out an iteration method, which includes following iteration steps a) to e), as follows: a) providing an estimated image of the sample, b) generating simulated raw images, in each case by image processing of the estimated image using the point spread function and one of the n illumination pattern functions such that n simulated raw images are obtained, c) assigning each of the n simulated raw images to that of the n provided raw images which was generated by the illumination pattern that corresponds to the illumination pattern function used to generate the simulated raw image, and calculating n correction raw images by the comparison of each provided raw image with the simulated raw image assigned thereto, d) generating a correction image by combining image processing of the n correction raw images using the point spread function and the n illumination pattern functions, wherein a filtering step is carried out in each implementation of iteration step d), said filtering step suppressing a spatial fundamental frequency of the illumination pattern, and e) reconstructing the estimated image of the sample by means of the correction image and using the corrected estimated image of the sample as the estimated image of the sample in iteration step a) in the next run through the iteration.

First claim

What is claimed is:

1. A method of image evaluation when performing SIM microscopy on a sample, comprising:
A) providing n raw images of the sample, which were each generated by illuminating the sample with an individually positioned SIM illumination pattern and imaging the sample in accordance with a point spread function,
B) providing n illumination pattern functions, which each describe one of the individually positioned SIM illumination patterns,
C) providing the point spread function, and
D) carrying out an iteration method, which comprises following iteration steps a) to e) as follows:
a) providing an estimated image of the sample,
b) generating simulated raw images, in each case by image processing of the estimated image using the point spread function and one of the n illumination pattern functions such that n simulated raw images are obtained,
c) assigning each of the n simulated raw images to that of the n provided raw images which was generated by the illumination pattern that corresponds to the illumination pattern function used to generate the simulated raw image, and calculating n correction raw images by the comparison of each provided raw image with the simulated raw image assigned thereto,
d) generating a correction image by combining image processing of the n correction raw images using the point spread function and the n illumination pattern functions, and
e) reconstructing the estimated image of the sample by means of the correction image and using the corrected estimated image of the sample as the estimated image of the sample in iteration step a) in the next run through the iteration,
carrying out a filtering step in a plurality of implementations of iteration step d), said filtering step attenuating or suppressing a signal component corresponding to an order of diffraction of the illumination pattern.

2. The method as claimed in claim 1 , wherein the filtering step attenuates or suppresses the signal component which corresponds to the zero order of diffraction of the illumination pattern.

3. The method as claimed in claim 1 , wherein the filtering step attenuates or suppresses signal components which correspond to higher orders of diffraction of the illumination pattern.

4. The method as claimed in claim 1 , wherein the filtering step attenuates or suppresses signal components corresponding to a plurality of orders of diffraction of the illumination pattern.

5. The method as claimed in claim 4 , wherein, for the filtering step, a degree of the suppression and/or a frequency bandwidth of the suppression are chosen differently for the orders of diffraction.

6. The method as claimed in claim 5 , wherein the degree of suppression and/or the frequency bandwidth is greater for the zero order of diffraction than for higher orders of diffraction.

7. The method as claimed in claim 1 , wherein, in iteration step b), the generation of the simulated raw images in each case comprises a convolution of the estimated image with the point spread function and one of the n illumination pattern functions and wherein, in iteration step d), the reconstruction of the correction image comprises a deconvolution of the n correction raw images using the point spread function and the n illumination pattern functions, wherein the filtering is carried out in the deconvolution.

8. The method as claimed in claim 1 , wherein the iteration method comprises a maximum likelihood estimation.

9. The method as claimed in claim 1 , wherein, in iteration step d), the assignment is implemented on the basis of a phase angle and—to the extent this is present—a relative rotational position of the illumination pattern.

10. A method for performing SIM microscopy on a sample, wherein a microscope whose imaging properties are characterized by a point spread function is used and the method further comprises:
A) providing n raw images of the sample, which were each generated by illuminating the sample with an individually positioned SIM illumination pattern and imaging the sample in accordance with a point spread function,
B) providing n illumination pattern functions, which each describe one of the individually positioned SIM illumination patterns,
C) providing the point spread function and
D) carrying out an iteration method, which comprises following iteration steps a) to e):
a) providing an estimated image of the sample,
b) generating simulated raw images, in each case by image processing of the estimated image using the point spread function and one of the n illumination pattern functions such that n simulated raw images are obtained,
c) assigning each of the n simulated raw images to that of the n provided raw images which was generated by the illumination pattern that corresponds to the illumination pattern function used to generate the simulated raw image, and calculating n correction raw images by the comparison of each provided raw image with the simulated raw image assigned thereto,
d) generating a correction image by combining image processing of the n correction raw images using the point spread function and the n illumination pattern functions, and
e) reconstructing the estimated image of the sample by means of the correction image and using the corrected estimated image of the sample as the estimated image of the sample in iteration step a) in the next run through the iteration,
carrying out a filtering step in a plurality of implementations of iteration step d), said filtering step attenuating or suppressing a signal component corresponding to an order of diffraction of the illumination pattern, and
providing the n raw images of the sample each by illuminating the sample with one of the individually positioned SIM illumination patterns and imaging the sample by means of the microscope.

11. A computer program stored on a non-transitory computer readable storage device comprising commands which, when the program is executed by a computer, cause the latter to read n raw images of a sample which were each generated by illuminating the sample with an individually positioned SIM illumination pattern and imaging the sample in accordance with a point spread function, n illumination pattern functions, which each describe one of the individually positioned SIM illumination patterns, and the point spread function, and to carry out an iteration method, which comprises following iteration steps a) to e):
a) providing an estimated image of the sample,
b) generating simulated raw images, in each case by image processing of the estimated image using the point spread function and one of the n illumination pattern functions such that n simulated raw images are obtained,
c) assigning each of the n simulated raw images to that of the n provided raw images which was generated by the illumination pattern that corresponds to the illumination pattern function used to generate the simulated raw image, and calculating n correction raw images by the comparison of each provided raw image with the simulated raw image assigned thereto,
d) generating a correction image by combining image processing of the n correction raw images using the point spread function and the n illumination pattern functions, and
e) reconstructing the estimated image of the sample by means of the correction image and using the corrected estimated image of the sample as the estimated image of the sample in iteration step a) in the next run through the iteration
wherein the commands cause the computer to carry out a filtering step in a plurality of implementations of iteration step d), said filtering step attenuating or suppressing a signal component corresponding to an order of diffraction of the illumination pattern.

12.

Assignees

• Zeiss Carl Microscopy Gmbh

Inventors

• Kleppe Ingo
• Novikau Yauheni
• Schaefer Lutz

Classifications

• G06T5/20 Primary

  using local operators · CPC title

• G01B11/25

  by projecting a pattern, e.g. {one or more lines,} moiré fringes on the object (G01B11/255 takes precedence {; image analysis for depth or shape recovery G06T7/50}) · CPC title

• G02B21/06

  Means for illuminating specimens · CPC title

• G02B21/367 Primary

  providing an output produced by processing a plurality of individual source images, e.g. image tiling, montage, composite images, depth sectioning, image comparison · CPC title

• G06T5/50

  using two or more images, e.g. averaging or subtraction · CPC title