Apparatus and method of estimating values from images

Patent metadata

Field	Value
Publication number	US-12204086-B2
Application number	US-202318236088-A
Country	US
Kind code	B2
Filing date	Aug 21, 2023
Priority date	Dec 6, 2019
Publication date	Jan 21, 2025
Grant date	Jan 21, 2025

Abstract

A method is used to generate a distortion model for a structured illumination microscopy (SIM) optical system. A sliding window is moved in relation to a plurality of images to define a plurality of sub-tiles. Each sub-tile represents a portion of the corresponding image. Parameters are estimated for each sub-tiles. The parameters include two or more parameters selected from the group consisting of modulation, angle, spacing, phase offset, and phase deviation. A full width at half maximum (FWHM) value associated with each sub-tile is estimated. A distortion model is estimated, based at least in part on a combination of the estimated parameters and FWHM values stored in the predetermined format and an estimated center window parameter. A two-dimensional image may be generated, based at least in part on the estimated distortion model. The two-dimensional image may include representations indicating where distortions occur in the optical system.

First claim

What is claimed is:

1. A method including:
receiving a plurality of images captured using structured illumination microscopy (SIM) in an optical system, each image of the plurality of images having a first field of view;
defining a window, the window defining a second field of view representing a portion of the first field of view such that the second field of view is smaller than the first field of view;
moving the window in relation to each image of a plurality of images;
capturing a plurality of sub-tiles from each image of the plurality of images while moving the window in relation to each image of the plurality of images,
each sub-tile of the plurality of plurality of sub-tiles representing a portion of the corresponding image of the plurality of images, the portion represented by each sub-tile of the plurality of sub-tiles being defined by the second field of view at a position corresponding to a moment at which the sub-tile of the plurality of sub-tiles is captured;

estimating parameters associated with each sub-tile of the plurality of sub-tiles, the parameters comprising an angle, a spacing, and a phase offset; and
storing the estimated parameters in a predetermined format.

2. The method of claim 1 , the parameters further including a parameter selected from the group consisting of modulation and phase deviation.

3. The method of claim 1 , further comprising:
estimating a full width at half maximum (FWHM) value associated with each sub-tile of the plurality of sub-tiles; and
storing the FWHM values in a predetermined format.

4. The method of claim 1 , further comprising:
estimating a center window parameter, the center window parameter corresponding to a central region within the first field of view; and
estimating a distortion model based at least in part on a combination of the estimated parameters stored in the predetermined format and the estimated center window parameter, the estimating the distortion model including subtracting the estimated center window parameter from the estimated parameters stored in the predetermined format.

5. The method of claim 4 , further comprising:
capturing a subsequent plurality of images using SIM in the optical system; and
generating a high-resolution image based at least in part on the plurality of images, the generating the high-resolution image including adjusting data from the subsequent plurality of images based at least in part on the estimated distortion model.

6. A method including:
receiving a plurality of images captured using structured illumination microscopy (SIM) in an optical system, each image of the plurality of images having a first field of view;
defining a window, the window defining a second field of view representing a portion of the first field of view such that the second field of view is smaller than the first field of view;
moving the window in relation to each image of a plurality of images;
capturing a plurality of sub-tiles from each image of the plurality of images while moving the window in relation to each image of the plurality of images, each sub-tile of the plurality of plurality of sub-tiles representing a portion of the corresponding image of the plurality of images, the portion represented by each sub-tile of the plurality of sub-tiles being defined by the second field of view at a position corresponding to a moment at which the sub-tile of the plurality of sub-tiles is captured;
estimating a first set of parameters for a first sub-tile of the plurality of sub-tiles,
the first set of parameters comprising an angle and a spacing;

mapping the first set of parameters to a second sub-tile of the plurality of sub-tiles;
estimating a phase offset for the second sub-tile of the plurality of sub-tiles using the mapped first set of parameters; and
storing the mapped first set of parameters and the estimated phase offset for the second sub-tile in a predetermined format.

7. The method of claim 6 , the parameters further including a parameter selected from the group consisting of modulation, phase offset, and phase deviation.

8. The method of claim 6 , further comprising:
estimating a full width at half maximum (FWHM) value associated with each sub-tile of the plurality of sub-tiles; and
storing the FWHM values in a predetermined format.

9. The method of claim 6 , further comprising:
estimating a center window parameter, the center window parameter corresponding to a central region within the first field of view; and
estimating a distortion model based at least in part on a combination of the estimated parameters stored in the predetermined format and the estimated center window parameter, the estimating the distortion model including subtracting the estimated center window parameter from the estimated parameters stored in the predetermined format.

10. The method of claim 9 , further comprising:
capturing a subsequent plurality of images using SIM in the optical system; and
generating a high-resolution image based at least in part on the plurality of images, the generating the high-resolution image including adjusting data from the subsequent plurality of images based at least in part on the estimated distortion model.

11. An apparatus comprising:
a first optical assembly to emit structured illumination toward a target, the first optical assembly including:
a light emitting assembly,
a first phase mask to impart a first pattern to light emitted by the light emitting assembly,
a second phase mask to impart a second pattern to light emitted by the light emitting assembly, and
a phase adjustment assembly to adjust a phase of light structured by the first phase mask and the second phase mask;

a second optical assembly, the second optical assembly including an image sensor to capture images of the target as illuminated by the first optical assembly; and
a processor, the processor to perform the following:
receive a plurality of images captured using the image sensor, each image of the plurality of images having a first field of view,
define a window, the window defining a second field of view representing a portion of the first field of view such that the second field of view is smaller than the first field of view,
move the window in relation to each image of a plurality of images,
capture a plurality of sub-tiles from each image of the plurality of images while moving the window in relation to each image of the plurality of images, each sub-tile of the plurality of plurality of sub-tiles representing a portion of the corresponding image of the plurality of images, the portion represented by each sub-tile of the plurality of sub-tiles being defined by the second field of view at a position corresponding to a moment at which the sub-tile of the plurality of sub-tiles is captured,
estimate a first set of parameters for a first sub-tile of the plurality of sub-tiles, the first set of parameters comprising an angle and a spacing,
map the first set of parameters to a second sub-tile of the plurality of sub-tiles,
estimate a phase offset for the second sub-tile of the plurality of sub-tiles using the mapped first set of parameters, and
store the mapped first set of parameters and the estimated phase offset for the second sub-tile in a predetermined format.

12. The apparatus of claim 11 , the first set of parameters further including a parameter selected from the group consisting of modulation, phase offset, and phase deviation.

13. The apparatus of claim 11 , the processor further configured to:
estimate a full width at half maximum (FWHM) value associated with each sub-tile of the plurality of sub-tiles, and
store the estimat

Assignees

• Illumina Inc

Inventors

• Langlois Robert
• Lu Bo
• Ren Hongji
• Pinto Joseph
• Prince Simon
• Corbett Austin

Classifications

• G06T2207/30024

  Cell structures in vitro; Tissue sections in vitro · CPC title

• G06T2207/10056

  Microscopic image · CPC title

• G06T7/0012

  Biomedical image inspection · CPC title

• G02B21/14

  affording illumination for phase-contrast observation · CPC title

• G01N21/6458

  Fluorescence microscopy (fluorescence microscopes per se G02B21/0076 and G02B21/16) · CPC title