Method of detecting presences of different antinuclear antibody fluorescence pattern types without counterstaining and apparatus therefor

Patent metadata

Field	Value
Publication number	US-12209965-B2
Application number	US-202217579578-A
Country	US
Kind code	B2
Filing date	Jan 19, 2022
Priority date	Jan 19, 2021
Publication date	Jan 28, 2025
Grant date	Jan 28, 2025

Abstract

A method and apparatus are provided for detecting respective potential presences of respective different cellular fluorescence pattern types on a biological cellular substrate including human epithelioma cells (HEp cells), wherein the fluorescence pattern types include different antinuclear antibody fluorescence pattern types. A method is also provided for detecting potential presences of different cellular fluorescence pattern types on a biological cellular substrate including human epithelioma cells by means of digital image processing, as well as a computing unit, a data network device, a computer program product and a data carrier signal therefor.

First claim

What is claimed is:

1. A method for detecting respective potential presences of respective different cellular fluorescence pattern types on a biological cellular substrate comprising human epithelioma cells, wherein the cellular fluorescence pattern types comprise a plurality of different antinuclear antibody fluorescence pattern types, the method comprising:
incubating the cellular substrate with a liquid patient sample which potentially comprises primary antibodies and, furthermore, with secondary antibodies which have been labelled with a fluorescent dye;
acquiring a total image which represents staining of the cellular substrate due to the fluorescent dye;
determining a segmented image by means of segmentation of the total image;
detecting, in the segmented image, respective image segments wherein a respective image segment represents a respective metaphase plate of a mitosis cell in a metaphase stage,
selecting sub-images of the total image which each comprise at least one mitotic cell and selecting corresponding sub-images of the segmented image on the basis of the detected image segments; and
detecting respective actual presences of the respective cellular fluorescence pattern types by means of a convolutional neural network on the basis of the selected sub-images of the total image and the selected sub-images of the segmented image,
wherein the convolutional neural network processes, in each case, a tuple of sub-images at the same time, which comprises at least one selected sub-image of the total image and a corresponding selected sub-image of the segmented image.

2. The method according to claim 1 , further comprising:
detecting in the total image, on the basis of the segmented image, respective image segments which each represent a mitotic cell of sufficient quality,
wherein a mitotic cell is of sufficient quality when it is present in a metaphase stage of mitosis; and
selecting sub-images of the total image and corresponding sub-images of the segmented image on the basis of the detected image segments which each represent at least one mitotic cell of sufficient quality.

3. The method according to claim 1 , further comprising:
determining respective measures of confidence for the respective actual presences of the respective fluorescence pattern types by means of the convolutional neural network on the basis of the selected sub-images of the total image and the selected sub-images of the segmented image.

4. The method according to claim 3 , wherein:
the convolutional neural network comprises an output layer which generates a respective feature map for a respective cellular fluorescence pattern type; and
the convolutional neural network determines a respective measure of confidence on the basis of a respective feature map.

5. The method according to claim 3 , further comprising:
segmenting the total image into image segments of different segment classes;
determining at least one brightness value for at least one fluorescence pattern type on the basis of one or more image segments of at least one particular segment class; and
verifying the measure of confidence of the at least one fluorescence pattern type on the basis of the brightness value of the at least one fluorescence pattern type.

6. The method according to claim 5 , wherein the verification of the measure of confidence is done on the basis of the brightness value and depending on a threshold value specifiable by a user.

7. The method according to claim 3 , further comprising, for a respective sub-image tuple which comprises a sub-image of the total image and a corresponding sub-image of the segmented image:
determining respective sub-image measures of confidence for respective actual sub-image presences of respective cellular fluorescence pattern types by means of the convolutional neural network; and
determining the respective measures of confidence for the respective actual presences of the respective fluorescence pattern types on the basis of the sub-image measures of confidence.

8. The method according to claim 1 , further comprising
dividing the total image into a set of sub-images according to a specified division scheme;
selecting sub-images of the total image on the basis of the detected image segments and selecting corresponding sub-images of the segmented image; and
detecting respective actual presences of the respective cellular fluorescence pattern types by means of the convolutional neural network on the basis of the selected sub-images of the total image and on the basis of the selected sub-images of the segmented image.

9. An apparatus for detecting respective potential presences of respective different cellular fluorescence pattern types on a biological cellular substrate comprising human epithelioma cells by means of digital image processing, the apparatus comprising:
a holding device for the cellular substrate, which was incubated with a liquid patient sample which potentially comprises primary antibodies and, furthermore, with secondary antibodies which have been labelled with a fluorescent dye;
at least one image acquisition unit for acquiring a total image which represents staining of the cellular substrate due to the fluorescent dye; and
at least one computing unit configured to:
determine a segmented image by means of segmentation of the total image;
detect in the segmented image respective image segments, wherein a respective image segment represents a respective metaphase plate of a mitosis cell in a metaphase stage,
select sub-images of the total image which each comprise at least one mitotic cell and corresponding sub-images of the segmented image on the basis of the detected image segments; and
detect respective actual presences of the respective cellular fluorescence pattern types by means of a convolutional neural network on the basis of the selected sub-images of the total image and the selected sub-images of the segmented image,
wherein the convolutional neural network processes, in each case, a tuple of sub-images at the same time, which comprises at least one selected sub-image of the total image and a corresponding selected sub-image of the segmented image.

10. A method for detecting respective potential presences of respective different cellular fluorescence pattern types on a biological cellular substrate comprising human epithelioma cells by means of digital image processing, the method comprising:
acquiring a total image which represents staining of the cellular substrate due to the fluorescent dye;
determining a segmented image by means of segmentation of the total image;
detecting in the segmented image respective image segments, wherein a respective image segment represents a respective metaphase plate of a mitosis cell in a metaphase stage,
selecting sub-images of the total image which each comprise at least one mitotic cell and corresponding sub-images of the segmented image on the basis of the detected image segments; and
detecting respective actual presences of the respective cellular fluorescence pattern types by means of a convolutional neural network on the basis of the selected sub-images of the total image and the selected sub-images of the segmented image,
wherein the convolutional neural network processes, in each case, a tuple of sub-images at the same time, which comprises at least one selected sub-image of the total image and a corresponding selected sub-image of the segmented image.

11. A method for digital image processing comprising:
receiving a total image which represents staining of a biological cellular substrate due to a fluorescent dye, wherein the biological cellular substrate comprises human epithelioma cells, the method

Assignees

• Euroimmun Medizinische Labordiagnostika Ag

Inventors

• Krause Christopher
• Krauth Jens
• Gerlach Stefan
• Marzahl Christian
• Hahn Melanie
• Voigt Joern

Classifications

• G06N3/09

  Supervised learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06T2207/30024

  Cell structures in vitro; Tissue sections in vitro · CPC title

• G06T2207/10064

  Fluorescence image · CPC title

• G06T7/0012

  Biomedical image inspection · CPC title