Microscopy system, method and computer program for processing microscope images

Patent metadata

Field	Value
Publication number	US-12056908-B2
Application number	US-202117376659-A
Country	US
Kind code	B2
Filing date	Jul 15, 2021
Priority date	Jul 17, 2020
Publication date	Aug 6, 2024
Grant date	Aug 6, 2024

Abstract

A method for processing microscope images comprises: acquiring a microscope image captured by a microscope; identifying at least one image section with sensitive information within the microscope image by means of an image processing program which uses provided reference information regarding sensitive information; rendering unrecognizable the at least one identified image section with sensitive information in order to generate an anonymized image; and outputting the anonymized image.

First claim

We claim:

1. A method for processing microscope images, comprising:
acquiring a microscope image captured by a microscope;
identifying at least one image section with sensitive information within the microscope image using an image processing program which uses provided reference information regarding sensitive information;
rendering unrecognizable the at least one identified image section with sensitive information in order to generate an anonymized image; and
outputting the anonymized image,
wherein the image processing program performs a classification of the microscope image by which the microscope image is assigned to an image class,
wherein a position of an image section with sensitive information is respectively stored as reference information for different image classes, and
wherein the identification of at least one image section with sensitive information occurs by using the position stored for the determined image class.

2. The method as defined in claim 1 ,
wherein the microscope image is an overview image of a sample carrier or a sample image of a microscopic sample.

3. The method as defined in claim 1 ,
wherein the reference information regarding sensitive information relates to one or more of the following: text, adhesive labels or markings on sample carriers, a slide cover area of a sample carrier; one or more sample chamber areas; one or more sample areas on a sample carrier; or a background area outside the sample carrier;
wherein the microscope image is an overview image and wherein the at least one image section with sensitive information comprises text, an adhesive label, a marking on the sample carrier, a slide cover area, one or more sample chamber areas, one or more sample areas on a sample carrier, or a background area outside a sample carrier.

4. The method as defined in claim 1 ,
wherein the predetermined reference information regarding sensitive information relates to one or more sample types, wherein the sample types are one or more of: biological cells, cell organelles, tissue sections, pathology data or manufacturer/product designations.

5. The method as defined in claim 1 ,
wherein a user is presented with a selection of options concerning the type of sensitive information for which corresponding image sections are to be rendered unrecognizable,
wherein the provided reference information relates to different types of sensitive information,
wherein the identification and rendering unrecognizable of image sections only includes image sections that relate to the types of sensitive information selected by the user.

6. The method as defined in claim 1 ,
wherein the image processing program selects the types of sensitive information for which corresponding image sections are rendered unrecognizable, based on an image content of the microscope image or a contextual datum relating to the microscope image.

7. The method as defined in claim 6 ,
wherein the image processing program performs the selection of the types of sensitive information depending on one or more of the following factors:
as a function of whether the microscope image is an overview image or a sample image;
as a function of parameters and settings on a microscope stand or of a microscope component;
as a function of a file name of the microscope image or a detail linked to a file of the microscope image;
as a function of a sample type or a sample carrier type used when the microscope image was captured;
as a function of a text content on a sample carrier.

8. The method as defined in claim 1 ,
wherein a shape is predetermined for the at least one image section, whereby the shape of the at least one image section is disassociated from a shape associated with the sensitive information.

9. The method as defined in claim 1 ,
wherein the image processing program comprises a trained reference model of a machine learning algorithm, which performs the identification of the at least one image section with sensitive information, wherein the reference information regarding sensitive information is provided in the form of model parameters in the trained reference model.

10. The method as defined in claim 9 , further comprising:
training the reference model of the machine learning algorithm in order to define the model parameters, wherein the training occurs with pairs each including a training image and an associated target image, wherein training images contain image sections with sensitive information and wherein areas corresponding to these image sections in terms of their position are tagged or rendered unrecognizable in the associated target images.

11. The method as defined in claim 9 ,
wherein the reference model of the machine learning algorithm also performs the operation of rendering the at least one image section unrecognizable.

12. The method as defined in claim 1 ,
wherein the image processing program identifies the at least one image section with sensitive information using segmentation or detection.

13. The method as defined in claim 1 ,
wherein the image processing program comprises at least one of:
a neural network trained using a machine learning algorithm, which generates the anonymized image directly from the microscope image, to which end the neural network is trained with pairs each including a training image and an associated target image, wherein training images contain image sections with sensitive information and areas corresponding to these image sections in terms of their position are rendered unrecognizable in the associated target images; and an autoencoder which is trained with a machine learning algorithm using training images that do not contain any image sections with sensitive information in order to generate the anonymized image directly from the microscope image.

14. The method as defined in claim 1 ,
wherein the operation of rendering an identified image section unrecognizable occurs by
pixelating, adding noise to or blurring the identified image section; or
replacing the identified image section or parts of the same using image content from a surrounding area; or
excising or truncating the identified image section; or
replacing the identified image section with a predetermined content, a predetermined pattern or a predetermined colour; or
deleting content of a certain colour channel of the microscope image at least in the identified image section.

15. The method as defined in claim 1 ,
wherein the operation of rendering an identified image section unrecognizable occurs by feeding the identified image section to:
a neural network trained to fill an image region based on contextual data from adjacent regions and by replacing the identified image section with an output of the neural network; or
an autoencoder, which generates an output therefrom that replaces the identified image section, wherein the autoencoder is trained with training images that do not contain any sensitive information.

16. The method as defined in claim 1 ,
wherein a type of operation for rendering an image section unrecognizable occurs as a function of the sensitive information, wherein, when the sensitive information contained in the image section is text, the operation of rendering the identified image section unrecognizable occurs by replacement with image content from a surrounding area, while, when the sensitive information contained in the image section is not text, the operation of rendering the identified image section unrecognizable occurs in a manner other than by replacement with image content from a surrounding area.

17.

Assignees

• Zeiss Carl Microscopy Gmbh

Inventors

• Amthor Manuel
• Haase Daniel
• Ohrt Thomas

Classifications

• G06N3/0455

  Auto-encoder networks; Encoder-decoder networks · CPC title

• G06N3/094

  Adversarial learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/0475

  Generative networks · CPC title

• G06N3/09

  Supervised learning · CPC title