Predicting structured illumination parameters

Patent metadata

Field	Value
Publication number	US-11525993-B2
Application number	US-202117396478-A
Country	US
Kind code	B2
Filing date	Aug 6, 2021
Priority date	Jun 29, 2018
Publication date	Dec 13, 2022
Grant date	Dec 13, 2022

Abstract

Implementations of the disclosure are directed to predicting structured illumination parameters for a particular point in time, space, and/or temperature using estimates of structured illumination parameters obtained from structured illumination images captured by a structured illumination system. Particular implementations are directed to predicting structured illumination frequency, phase, orientation, and/or modulation order parameters.

First claim

What is claimed is:

1. An imaging device, comprising:
a light emitter to emit light;
a beam splitter to split light emitted by the light emitter to project a structured illumination pattern on a sample;
a controller configured to cause the imaging device to perform operations comprising:
capturing a first image of the sample at a first time, a first sample position, or a first sample temperature;
estimating, using at least the captured first image, a first structured illumination parameter;
capturing a second image of the sample at a second time, a second sample position, or a second sample temperature;
estimating, using at least the captured second image, a second structured illumination parameter; and
predicting, using at least the first structured illumination parameter or the second structured illumination parameter, a third structured illumination parameter corresponding to a third image of the sample captured at a third time, a third sample position, or a third sample temperature, wherein each of the first, second, and third structured illumination parameters comprises a phase, a frequency, an orientation, or a modulation order.

2. The imaging device of claim 1 , wherein:
the first image is captured at the first time;
the second image is captured at the second time after the first time;
the third image is captured at the third time between the first time and the second time; and
predicting the third structured illumination parameter comprises:
determining a rate of change from the first structured illumination parameter at the first time to the second structured illumination at the second time; and
predicting, using at least the determined rate of change, the third structured illumination parameter at the third time.

3. The imaging device of claim 1 , wherein:
the first image is captured at the first time;
the second image is captured at the second time after the first time;
the third image is captured at the third time after both the first time and the second time; and
predicting the third structured illumination parameter comprises: predicting, using at least the first structured illumination parameter and the second structured illumination parameter, the third structured illumination parameter at the third time.

4. The imaging device of claim 3 , wherein:
the imaging device further comprises an optical phase modulator or phase shifter that is positioned after the beam splitter in an optical path of the imaging device; and
the operations further comprise: prior to capturing the third image at the third time, adjusting, using at least the predicted third structured illumination parameter, the optical phase modulator or phase shifter to adjust a phase or orientation of the structured illumination pattern.

5. The imaging device of claim 4 , wherein
the optical phase modulator or phase shifter is a rotating mirror; and
the beam splitter comprises a diffraction grating.

6. The imaging device of claim 3 , wherein:
the beam splitter comprises a diffraction grating;
the imaging device further comprises a translation stage carrying the diffraction grating; and
the operations further comprise: prior to capturing the third image at the third time, adjusting, using at least the predicted third structured illumination parameter, the translation stage to adjust a phase or orientation of the structured illumination pattern.

7. The imaging device of claim 3 , wherein:
the imaging device further comprises a sample translation stage carrying the sample; and
the operations further comprise: prior to capturing the third image at the third time, adjusting, using at least the predicted third structured illumination parameter, the sample translation stage to adjust a phase or orientation of the structured illumination pattern.

8. The imaging device of claim 3 , wherein:
the controller comprises a memory; and
the operations further comprise:
storing in the memory: the first structured illumination parameter, the second structured illumination parameter, and the third structured illumination parameter; and
reducing, using one or more of the stored first structured illumination, the stored second structured illumination parameter, the stored third structured illumination parameter, or a stored value based on known physical characteristics of the imaging device, a search space for a fourth structured illumination parameter for a fourth image.

9. The imaging device of claim 1 , wherein:
the first image of the sample is captured at the first sample temperature;
the first structured illumination parameter is estimated at the first sample temperature;
the second image of the sample is captured at the second sample temperature;
the second structured illumination parameter is estimated at the second sample temperature; and
predicting the third structured illumination parameter, comprises: predicting the third structured illumination parameter at the third sample temperature.

10. The imaging device of claim 1 , wherein the operations further comprise:
dividing the first image of the sample into a plurality of image subsections;
estimating, at the computing device, using at least a first image subsection of the plurality of image subsections, a fourth structured illumination parameter;
estimating, at the computing device, using at least a second image subsection of the plurality of image subsections, a fifth structured illumination parameter; and
predicting, at the computing device, using at least the fourth structured illumination parameter or the fifth structured illumination parameter, a sixth structured illumination parameter corresponding to a third image subsection of the plurality of image subsections.

11. The imaging device of claim 1 , wherein the operations further comprise:
dividing the first image of the sample into a plurality of image subsections;
estimating, at the computing device, using at least a first image subsection of the plurality of image subsections, a fourth structured illumination parameter; and
using the estimated fourth structured illumination parameter as a predicted structured illumination parameter of a second image subsection of the plurality of image subsections.

12. An imaging device, comprising:
a light emitter to emit light;
a diffraction grating to split light emitted by the light emitter to project a structured illumination pattern on a sample;
a processor; and
a non-transitory computer-readable medium having executable instructions stored thereon that, when executed by the processor, cause the imaging device to perform operations comprising:
capturing a first image of the sample;
estimating, using at least the captured first image, a first structured illumination parameter;
capturing a second image of the sample;
estimating, using at least the captured second image, a second structured illumination parameter; and
predicting, using at least the first structured illumination parameter and the second structured illumination parameter, a third structured illumination parameter corresponding to a third image of the sample, wherein each of the first, second, and third structured illumination parameters comprises a phase, a frequency, an orientation, or a modulation order.

13. The imaging device of claim 12 , wherein:
the first image is captured at a first time;
the second image is captured at a second time after the first time; and
the third image is captured at a third time between the first time and the second time.

14. The imaging device of claim 12 , wherein:
the first image is captured at a first time;
the second image is captur

Assignees

• Illumina Inc

Inventors

• Carney Michael J
• Hong Stanley S
• Langlois Robert
• Ren Hongji
• Bartig Kevin Wayne
• Otto Rico
• Souverneva Olga Andreevna

Classifications

• G02B21/26

  Stages; Adjusting means therefor · CPC title

• G06T7/11

  Region-based segmentation · CPC title

• G06T2207/10056

  Microscopic image · CPC title

• G02B21/367

  providing an output produced by processing a plurality of individual source images, e.g. image tiling, montage, composite images, depth sectioning, image comparison · CPC title

• G02B21/361

  Optical details, e.g. image relay to the camera or image sensor (G02B21/364 takes precedence; illumination details G02B21/06 and subgroups) · CPC title