Microscopy system and method for checking input data

Patent metadata

Field	Value
Publication number	US-12340505-B2
Application number	US-202117493980-A
Country	US
Kind code	B2
Filing date	Oct 5, 2021
Priority date	Oct 9, 2020
Publication date	Jun 24, 2025
Grant date	Jun 24, 2025

Abstract

A microscopy system comprises a microscope configured to capture raw data and a computing device configured to derive at least one microscope image from the raw data and to run a processing program for processing the microscope image. The computing device is configured to first supply a microscope image which is to be processed with the processing program to a checking program, which checks the supplied microscope image with respect to an evaluation criterion predefined for the processing program. The microscope image is only transmitted to the processing program in the event of a positive checking result.

First claim

What is claimed is:

1. A microscopy system comprising:
a microscope configured to capture raw data; and
a computing device configured to form at least one microscope image from the raw data and to run a processing program for processing the microscope image;
wherein the computing device is further configured to first supply a microscope image which is to be processed with the processing program to a checking program executing on at least one of the computing device or a second computing device;
wherein the checking program, when executed on the at least one of the computing device or the second computing device, is configured to:
check the supplied microscope image with respect to an evaluation criterion predefined for the processing program, wherein the checking program determines a deviation of the microscope image from known microscope images as a measure of resemblance between the microscope image and training data used for training the processing program for the evaluation criterion;
transmit the microscope image to the processing program in the event of a positive checking result; and
not transmit the microscope image to the processing program in the event of a negative checking result, wherein the checking program does not transmit the microscope image to the processing program if the deviation indicates that the microscope image falls outside a statistical distribution of the training data used for training the processing program.

2. A method for checking input data, comprising:
a first computing device supplying measurement data to be processed with a processing program to a checking program executing on at least one of the first computing device or a second computing device, the measurement data including a microscope image,
wherein, when executed on the at least one of the computing device or the second computing device, the checking program:
checks the supplied measurement data with respect to an evaluation criterion predefined for the processing program, wherein the checking program determines a deviation of the microscope image from known microscope images as a measure of resemblance between the microscope image and training data used for training the processing program for the evaluation criterion;
transmits the measurement data to the processing program in the event of a positive checking result; and
does not transmit the measurement data to the processing program in the event of a negative checking result, wherein the checking program does not transmit the microscope image to the processing program if the deviation indicates that the microscope image falls outside a statistical distribution of the training data used for training the processing program.

3. The method according to claim 2 ,
wherein the processing program is an image processing program, and
wherein the evaluation criterion takes into account one or more of the following aspects: image noise, image brightness, and an illumination distribution.

4. The method according to claim 3 ,
wherein the evaluation criterion additionally takes into account one or more of the following aspects:
presence of artefacts in the microscope image; and
missing structures in the microscope image.

5. The method according to claim 3 ,
wherein the checking program also takes into account contextual data regarding the microscope image together with the microscope image.

6. The method according to claim 3 ,
wherein the checking program checks image regions of the microscope image separately from one another and only transmits image regions for which a positive checking result is determined to the image processing program.

7. The method according to claim 3 ,
wherein the checking program checks image regions of the microscope image separately from one another and, for image regions for which a negative checking result is determined, carries out or suggests a new image capture with modified capture parameters.

8. The method according to claim 3 ,
wherein a plurality of image processing programs are provided to process the microscope image in parallel or in sequence in relation to one another,
wherein a respective checking program is used for each of the image processing programs and the checking programs differ with respect to their respective evaluation criteria depending on the associated image processing program.

9. The method according to claim 3 ,
wherein, in the event of a positive checking result, the image processing program generates an image processing result from the microscope image,
wherein the image processing result is fed to a verification model designed to check a plausibility of the image processing result.

10. The method according to claim 9 ,
wherein the verification model is formed by a machine learning model trained with training data comprising the image processing results of the image processing program.

11. The method according to claim 2 ,
wherein the checking program comprises a machine learning model, wherein the evaluation criterion is defined by model parameter values of the machine learning model based on training data.

12. The method according to claim 11 ,
wherein the machine learning model is designed as an anomaly detector, a one-class classifier or an autoencoder and is trained in an unsupervised training using training data, wherein all training data constitute suitable input data for the processing program.

13. The method according to claim 11 ,
wherein the machine learning model is trained or has been trained in a supervised training using training data, wherein a part of the training data is labelled with a positive annotation indicating that the training data in question represents suitable input data for the processing program, and wherein another part of the training data is labelled with a negative annotation indicating that the training data in question does not represent suitable input data for the processing program.

14. The method according to claim 13 ,
wherein a reason is additionally indicated for each negative annotation of training data, whereby the machine learning model learns to provide a reason for the negative checking result in the event of a negative checking result.

15. The method according to claim 11 ,
wherein training data is labelled with a positive annotation or a negative annotation as a function of a noise level, a brightness, a brightness distribution or a contrast of said training data.

16. The method according to claim 15 ,
wherein, in order to determine annotations for the training data, the training data is first entered into the processing program and positive or negative annotations are then assigned via an evaluation of processing results of the processing program.

17. The method according to claim 11 ,
wherein training data is generated by respectively generating a plurality of images with different levels of interference as training data from one or more images captured by a microscope, by adding different levels of noise or by setting different levels of underexposure or overexposure via image processing.

18. The method according to claim 11 ,
wherein the checking program or machine learning model determines whether predefined image content known to be unsuitable for the image processing program is contained in the microscope image and outputs a negative checking result when the predefined image content is detected in the microscope image.

19. The method according to claim 2 ,
wherein, in the event of a negative checking result, one or more of the fol

Assignees

• Zeiss Carl Microscopy Gmbh

Inventors

• Amthor Manuel
• Haase Daniel

Classifications

• G06T7/0002

  Inspection of images, e.g. flaw detection · CPC title

• G06F18/214

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06T2207/10056

  Microscopic image · CPC title

• G06T2207/30168

  Image quality inspection · CPC title

• G06T2207/20081

  Training; Learning · CPC title