Method and system for event-based imaging

Patent metadata

Field	Value
Publication number	US-12432437-B2
Application number	US-202118016806-A
Country	US
Kind code	B2
Filing date	Jul 21, 2021
Priority date	Jul 22, 2020
Publication date	Sep 30, 2025
Grant date	Sep 30, 2025

Abstract

The present invention refers to a computer-implemented method for event-based imaging at least one specimen to record only structures of interest as events, the method comprising: providing an automated image capturing device coupled with a specimen holder, a controller coupled with the image capturing device, at least one processor in an operative conjunction with the controller, and a computer-readable medium comprising instructions that, when executed by the at least one processor, cause the controller and/or the at least one processor to: a) acquire ( 102 ), by the image capturing device, at least one initial image of the at least one specimen ( 100 ) carried by the specimen holder, b) search ( 103 ) the at least one initial image for structures of interest, using an image processing algorithm, C) upon detection of one or more structures of interest, control an imaging capturing software of the image capturing device to acquire ( 104 ) at least one main image of the detected structures of interest, respectively, d) classify ( 105 ) the detected structures of interest in the at least one main image, using a classifying algorithm, c) evaluate ( 106 ) the classified structures of interest, and f) output ( 107 ) a result of the evaluated structures of interest, executing the instructions by the at least one processor, wherein at least steps a) to c) are repeated until a pre-given number k of structures of interest has been detected, with k being an integer.

First claim

The invention claimed is:

1. A computer-implemented method for event-based imaging multiple specimens to record only structures of interest as events, the method comprising:
providing an automated image capturing device providing a light beam and a specimen holder carrying the multiple specimens, a controller coupled with the image capturing device, at least one processor in an operative conjunction with the controller, and a computer-readable medium comprising instructions that, when executed by the at least one processor, cause the controller and/or the at least one processor to:
a) place one of the multiple specimens into the light beam,
b) acquire, by the image capturing device, at least one initial image of the specimen placed in the light beam using an objective lens,
c) search the at least one initial image for structures of interest, using an image processing algorithm,
d) upon detection of one or more structures of interest, control an imaging capturing software of the image capturing device to acquire at least one main image of the detected structures of interest, respectively, using a different objective lens than for the at least one initial image, the at least one main image being a three-dimensional image of multiple focal planes within the specimen placed in the light beam,
e) classify the detected structures of interest in the at least one main image in a three-dimensional space, using a classifying algorithm,
f) repeat steps a) to e) for at least one different specimen of the multiple specimens,
g) evaluate the classified structures of interest, and
h) output a result of the evaluated structures of interest, and

executing the instructions by the at least one processor.

2. The method according to claim 1 , wherein the at least one initial image of a specimen is acquired as a number of partial images of the respective specimen wherein the partial images of one or more specimens are searched, for each specimen as a whole or partial images of the specimen successively, for structures of interest until a pre-defined number k of structures of interest has been detected.

3. The method according to claim 2 , wherein the partial images of the one or more specimens are searched for structures of interest of a specific classification until a predefined number m of structures of interest of the specific classification is detected, with m being an integer, and m≤k.

4. The method according to claim 1 , wherein the at least one initial image is acquired as a number of temporary partial images.

5. The method according to claim 1 , wherein each of the at least one initial image is acquired as a respective low magnification image.

6. The method according to claim 1 , wherein searching the at least one initial image also includes differentiating between different structures of interest so that the at least one main image is already acquired by taking multiple source images at a number of different focal distances adapted to the detected different structures of interest.

7. The method according to claim 1 , wherein the at least one initial image is acquired by taking multiple source images at a predetermined number a of different focal distances, the number a being predetermined depending on the respective specimen and/or expected detectable structures of interest.

8. The method according to claim 7 wherein, in the case that no structures of interest are detected in a first initial image of a specimen which is composed of multiple source images taken at a first number of different focal distances, the image capturing device is controlled to acquire a next initial image by taking multiple source images at a next higher number of different focal distances and/or to move, for acquiring the next initial image, to another location of the respective specimen or to another specimen of the specimen placed into the light beam.

9. The method according to claim 1 , wherein, in the case that structures of interest are detected in at least one of the at least one initial image, the at least one main image of the detected structures of interest is acquired as a high magnification image, respectively.

10. The method according to claim 1 , wherein in the case that structures of interest are detected in at least one of the at least one initial image, each of the at least one main image of the detected structures of interest is acquired as an image by taking multiple source images at a variable number of different focal distances, wherein the acquisition of the respective image and the classification of the detected structures of interest are executed simultaneously so that the number of different focal distances of each of the at least one main image is chosen depending on a respective classification of the detected structures of interest.

11. The method according to claim 1 , wherein the steps a) to e) are executed for one specimen and/or for different specimens until a predetermined number of structures of interest has been detected, before steps f) and g) are executed.

12. A system for event-based imaging multiple specimens to record only structures of interest as events, the system comprising at least:
an automated image capturing device providing a light beam and a specimen holder carrying the multiple specimens,
a controller which is configured to be coupled with the image capturing device,
at least one processor in an operative conjunction with the controller, and
a computer-readable medium comprising instructions that when executed by the at least one processor, cause the controller coupled with the image capturing device and/or the at least one processor to:
a) place one of the multiple specimens into the light beam,
b) acquire, by the image capturing device, at least one initial image of the specimen placed in the light beam using an objective lens,
c) search the at least one initial image for structures of interest, using an image processing algorithm,
d) upon detection of one or more structures of interest, control an imaging capturing software of the image capturing device to acquire at least one main image of the detected structures of interest, respectively, using a different objective lens than for the at least one initial image, the at least one main image being a three-dimensional image of multiple focal planes within the specimen placed in the light beam,
e) classify the detected structures of interest in the at least one main image in a three-dimensional space, using a classifying algorithm,
f) repeat steps a) to e) for at least one different specimen of the multiple specimens,
g) evaluate the classified structures of interest, and
h) output a result of the evaluated structures of interest,

wherein the at least one processor is configured to execute the instructions.

13. The system according to claim 12 , wherein the instructions that when executed by the at least one processor, further cause the controller coupled with the image capturing device and/or the at least one processor to control a gripping device, particularly a robot arm to place the specimen on a specimen holder into a light beam of the image capturing device.

14. The system according to claim 12 further comprising an output device that is configured to output the result to a user or a gripping device which is configured to be controlled by the at least one processor or the controller to place the specimen on the specimen holder into a light beam of the image capturing device or a multi-slide chamber which is configured to be coupled with the image capturing device and designed to accommodate the specimen before or after the specimen on the specimen hol

Assignees

• Zeiss Carl Microscopy Gmbh

Inventors

• Misch Nicola Verena
• Flachmann Ralf
• Herwig Anne-Christina
• Klukas Christian

Classifications

• G06V20/698 Primary

  Matching; Classification · CPC title

• H04N23/61 Primary

  based on recognised objects · CPC title

• G06F18/2413

  based on distances to training or reference patterns · CPC title