Microscopy system and method for evaluating image processing results

Patent metadata

Field	Value
Publication number	US-12498556-B2
Application number	US-202217571677-A
Country	US
Kind code	B2
Filing date	Jan 10, 2022
Priority date	Jan 12, 2021
Publication date	Dec 16, 2025
Grant date	Dec 16, 2025

Abstract

In a method for evaluating image processing results, at least two microscope images that show the same structure are received. Each of the microscope images is processed by means of an image processing algorithm for calculating a respective image processing result. At least one confidence score of the image processing results is determined based on a degree of correspondence between the image processing results calculated from different microscope images.

First claim

What is claimed is:

1 . A method for evaluating image processing results, comprising
receiving, on a computer system including at least one computing device, at least two microscope images that show the same structure; the computer system processing each of the microscope images with a same image processing algorithm for calculating respective image processing results, wherein each of the microscope images is processed independently of each other with the same image processing algorithm such that each image processing result is calculated from only a respective one of the microscope images; the computer system determining at least one confidence score of the image processing results by determining differences between the image processing results calculated by the same image processing algorithm from different microscope images of the same structure.

2 . The method according to claim 1 ,
wherein the microscope images correspond to different captured raw images which show the same structure and differ in random image noise.

3 . The method according to claim 1 ,
wherein at least two of the microscope images are based on the same raw image and differ by a performed image manipulation.

4 . The method according to claim 3 ,
wherein the performed image manipulation comprises one or more of the following operations:
adding image noise;
mirroring or rotating image content;
excising or replacing image content;
condensing or stretching image content;
deforming and subsequently restoring image content;
modifying a brightness, tonal values or a contrast;
blurring or sharpening image content;
an image resolution modification.

5 . The method according to claim 1 ,
wherein the image processing algorithm comprises a trained machine learning model, which respectively calculates an image processing result from each of the microscope images, wherein each image processing result is a result image and the image processing algorithm for calculating the result images respectively processes the microscope images in at least one of the following ways: by means of a denoising, a deconvolution, a resolution enhancement, an artefact removal, a filling of image regions and holes, a mapping to another microscopy contrast method, a segmentation or an object detection; or wherein each of the image processing results is a classification result, a geometric variable or a count value of objects found in the microscope image.

6 . The method according to claim 1 ,
wherein the computer system uses a variance or a statistical measure of deviation between the image processing results for determining the differences between the image processing results.

7 . The method according to claim 1 ,
wherein the image processing results are result images, and wherein the computer system generates a confidence map by respectively determining the confidence score for different image pixels or image areas of the result images.

8 . The method according to claim 7 ,
wherein the computer system uses the confidence map to evaluate whether the image processing algorithm is suitable for the captured structure and whether a re-training of an image processing algorithm formed with a trained machine learning model should occur.

9 . The method according to claim 1 ,
wherein a confidence score machine learning model receives the image processing results as inputs and calculates the at least one confidence score as an output.

10 . The method according to claim 1 ,
wherein the image processing results are result images, and wherein a number, shape or size of certain objects or structures in the result images is taken into account for determining the differences between the result images.

11 . The method according to claim 1 ,
wherein image areas for which the confidence score exceeds a threshold value
are analyzed by capturing a new image,
are labelled with a warning,
are provided with artificial noise, or
are filled in with image information from at least one of the microscope images.

12 . The method according to claim 1 ,
wherein the computer system determines how at least one image processing parameter of the image processing algorithm is to be modified as a function of the confidence score, and wherein a new processing of the microscope images or one of the microscope images is carried out by the image processing algorithm with the at least one modified image processing parameter.

13 . The method according to claim 1 ,
wherein the computer system repeats the processing of microscope images in order to calculate image processing results and the determining of at least one confidence score of image processing results for different sets of microscope images, wherein the computer system uses the sets of microscope images as training data of a machine learning model and the computer system uses the associated confidence scores as target data in a training of the machine learning model so that the machine learning model is trained to calculate a mapping to at least one confidence score from a microscope image.

14 . A method for evaluating image processing results,
wherein an image processing algorithm calculates an image processing result for an entered microscope image, and wherein the computer system inputs the microscope image into a machine learning model which calculates at least one confidence score for the input microscope image, wherein the machine learning model is learned by carrying out the method of claim 13 .

15 . A computer program stored on a non-transitory computer-readable medium with commands that, when the computer program is executed by a computer system, cause the execution of the method according to claim 1 .

16 . A microscopy system comprising:
a microscope with a camera for capturing microscope images; and a computing device, which is configured to calculate an image processing result from a microscope image using a same image processing algorithm and to calculate at least one confidence score for the image processing result; wherein the computing device is configured to determine the at least one confidence score for an image processing result by:
respectively processing at least two microscope images showing the same structure by means of the image processing algorithm and calculating respective image processing results, wherein each of the microscope images is processed independently of each other with the same image processing algorithm such that each image processing result is calculated from only a respective one of the microscope images,
determining the at least one confidence score by determining differences between the image processing results calculated by the same image processing algorithm from different microscope images of the same structure.

17 . The microscopy system according to claim 16 ,
wherein the computing device is further configured to use the at least one confidence score to determine a reliability of the image processing results and/or evaluate a suitability of the image processing algorithm for the captured structure and/or determine whether a re-training of the image processing algorithm should occur.

18 . The method according to claim 1 , further comprising the computer system using the at least one confidence score in determining a reliability of the image processing results and/or in evaluating a suitability of the image processing algorithm for the captured structure and/or in determining whether a re-training of the image processing algorithm should occur.

Assignees

• Zeiss Carl Microscopy Gmbh

Inventors

• Amthor Manuel
• Haase Daniel

Classifications

• G06T5/60

  using machine learning, e.g. neural networks · CPC title

• G06T2207/30024

  Cell structures in vitro; Tissue sections in vitro · CPC title

• G06T2207/10056

  Microscopic image · CPC title

• G06T2207/30242

  Counting objects in image · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title