Microscopy system and method for processing a microscope image

Patent metadata

Field	Value
Publication number	US-12518548-B2
Application number	US-202217683133-A
Country	US
Kind code	B2
Filing date	Feb 28, 2022
Priority date	Mar 2, 2021
Publication date	Jan 6, 2026
Grant date	Jan 6, 2026

Abstract

Processing a microscope image includes forming an input image from a microscope image before the input image is input into an image processing program. The image processing program comprises a learned model for image processing which is trained with training images that show structures with certain image properties. The image processing program calculates an image processing result from the input image. The microscope image is converted into the input image by an image conversion program in such a manner that image properties of structures in the input image are modified with respect to image properties of the structures of the microscope image so that they are closer to the image properties of the structures of the training images.

First claim

What is claimed is:

1 . A method for processing a microscope image, comprising receiving a microscope image;
inputting an input image formed from the microscope image into an image processing program comprising a learned model for image processing which has been trained to calculate image processing results from input training images that show structures with certain image properties; calculating an image processing result from the input image by the image processing program; receiving a size of structures in the microscope image in pixels; wherein a target size of structures in pixels is given; calculating a scaling factor by which the size of structures is converted to the target size of structures in pixels; and converting the microscope image into the input image by an image conversion program in such a manner that image properties of structures in the input image are modified with respect to image properties of the structures in the microscope image so that they are closer to the image properties of the structures in the input training images, the converting including rescaling the microscope image with the scaling factor to generate the input image.

2 . The method according to claim 1 ,
wherein image properties of the structures are geometry properties and/or brightness properties of the structures.

3 . The method according to claim 2 ,
wherein the geometry properties of the structures comprise a size, orientation or image distortion; wherein the brightness properties relate to a brightness, a saturation or an image contrast of the structures.

4 . The method according to claim 1 ,
wherein the image conversion program determines the image properties of the structures in the microscope image, wherein target image properties are predetermined for image properties of structures of in microscope images, wherein the image conversion program changes the determined image properties to the target image properties, in order to calculate the input image from the microscope image.

5 . The method according to claim 1 , further including determining the size of structures of in the microscope image in pixels by image analysis of the microscope image.

6 . The method according to claim 1 ,
wherein the image conversion program comprises a learned model for determining image properties that is trained using training images to determine image properties of structures in microscope images.

7 . The method according to claim 1 ,
wherein the image conversion program is a learned model for image conversion that is trained using images for which it is specified in the form of annotations how these images are to be converted in order to form input images.

8 . The method according to claim 7 ,
wherein the annotations respectively indicate a scaling factor.

9 . The method according to claim 1 ,
wherein the image conversion program is a learned model for image conversion, and wherein the learned model for image conversion and the learned model for image processing are trained by a joint training with a joint optimization function.

10 . The method according to claim 1 ,
wherein the image conversion program, in order to determine a suitable conversion of the microscope image into the input image, tests a plurality of potential conversions, which are respectively applied to the microscope image in order to generate potential input images, wherein image properties of structures in the potential input images are subsequently evaluated according to a predetermined criterion and the potential input image with the best evaluation is selected as the input image.

11 . The method according to claim 1 ,
wherein the image conversion program takes into account contextual information regarding the microscope image when calculating the input image, the contextual information indicating one or more of the following:
microscope settings, a magnification of an employed objective, illumination properties or camera properties;
information regarding a measurement situation, an employed sample carrier type or an employed sample type.

12 . The method according to claim 1 ,
wherein the learned model for image processing calculates, for the input image, a classification, an image segmentation, a detection, an image enhancement, a reconstruction of image areas or an image-to-image mapping.

13 . The method according to claim 1 ,
wherein the image processing result is a result image and wherein a conversion of the result image is performed that is inverse to the conversion by which the image conversion program generates the input image from the microscope image.

14 . A non-transitory computer-readable medium storing a computer program with commands that, when executed by a computer, cause the execution of the method of claim 1 .

15 . A microscopy system comprising a microscope for capturing a microscope image; and
a computing device, which comprises an image processing program that is configured to process an input image formed from the microscope image into an image processing result, wherein the image processing program comprises a learned model for image processing which is trained to calculate image processing results from input training images that show structures with certain image properties; wherein the computing device comprises an image conversion program configured to convert the microscope image into the input image in such a manner that image properties of structures in the input image are modified with respect to image properties of the structures in the microscope image so that they are closer to the image properties of the structures in the input training images; wherein the image conversion program is a machine-learned model for image conversion that is trained using images as inputs and scaling factors as annotations specifying how these images shall be rescaled to form input images for the image processing program, wherein the machine-learned model for image conversion calculates a mapping of the microscope image to a scaling factor which is then used by the computing device to rescale the microscope image to form the input image for the image processing program.

16 . The microscopy system of claim 15 ,
wherein image properties of the structures are geometry properties and/or brightness properties of the structures.

17 . The microscopy system of claim 15 ,
wherein the image conversion program determines the image properties of the structures in the microscope image, wherein target image properties are predetermined for image properties of structures in microscope images, wherein the image conversion program changes the determined image properties to the target image properties, in order to calculate the input image from the microscope image.

18 . The microscopy system of claim 15 ,
wherein the image conversion program determines a size of structures of in the microscope image in pixels as an image property, wherein a target size of structures in pixels is predetermined as a target image property, wherein the image conversion program calculates a scaling factor by which the determined size of the structures in pixels is converted to the target size of structures in pixels, wherein the image conversion program processes the microscope image with the scaling factor in order to generate the input image.

19 . The microscopy system of claim 15 ,
wherein the image conversion program comprises a learned model for determining image properties that is trained using training image

Assignees

• Zeiss Carl Microscopy Gmbh

Inventors

• Amthor Manuel
• Haase Daniel

Classifications

• G06T2207/20016

  Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform · CPC title

• G06T7/62

  of area, perimeter, diameter or volume · CPC title

• G06T2207/30242

  Counting objects in image · CPC title

• G06T2207/30024

  Cell structures in vitro; Tissue sections in vitro · CPC title

• G06T2207/20081

  Training; Learning · CPC title