Distance determination of a sample plane in a microscope system

Patent metadata

Field	Value
Publication number	US-2020200531-A1
Application number	US-201916720745-A
Country	US
Kind code	A1
Filing date	Dec 19, 2019
Priority date	Dec 20, 2018
Publication date	Jun 25, 2020
Grant date	—

Abstract

A distance determination system for a microscope system for coarse focus setting includes a sample stage with a placement surface for holding a sample carrier which is displaceable along at least one direction of extent of a sample plane, an overview camera with a non-telecentric objective for producing digital images, directed at the sample stage, and an evaluation unit, which includes a storage system storing at least two recorded digital images of the sample stage at different viewing angles, a trained machine-learning-based system for identifying corresponding structures of a sample carrier that has been placed into the sample stage in the two recorded digital images and a distance determination unit, which is adapted for determining the distance of a reference point of the sample carrier from a reference point of the overview camera based on the different viewing angles onto the sample stage, a pixel distance of the two recorded digital images with respect to one another using the associated corresponding structures contained in the recorded images.

First claim

1 . A distance determination system for a microscope system, the distance determination system comprising:
a sample stage having a placement surface configured to hold a sample carrier, wherein the sample stage is displaceable along at least one direction of extent of a sample plane; an overview camera with a non-telecentric objective configured to produce digital images, wherein the overview camera is directed at the sample stage; an evaluation unit connected to the overview camera, the evaluation unit including
a storage system configured to store at least two recorded digital images of the sample stage at different viewing angles; and
a trained machine-learning-based system configured to identify corresponding structures of the sample carrier placed into the sample stage in the two recorded digital images, wherein the trained machine-learning-based system comprises a trained reference model that is trainable by way of a set of annotated training images of sample carriers in a manner such, and the trained machine-learning-based system is thus adapted such, that corresponding structures in the at least two recorded digital images are associable with one another; and
a distance determination unit configured to determine a distance of a reference point of the sample carrier from a reference point of the overview camera based on the different viewing angles onto the sample stage and a pixel distance of the two recorded digital images with respect to one another using the associated corresponding structures contained in the at least two recorded digital images.

2 . The distance determination system of claim 1 , wherein the trained machine-learning-based system is configured to classify individual pixels of the at least two recorded images.

3 . The distance determination system of claim 1 , wherein the trained machine-learning-based system is configured to output a reference coordinate of one of the corresponding structures.

4 . The distance determination system of claim 3 , wherein a class of the reference coordinate of one of the corresponding structures is also output.

5 . The distance determination system of claim 1 , wherein the sample stage is illuminated using an illumination means.

6 . The distance determination system of claim 1 , wherein the sample carrier is a multiwell plate, a slide, a Petri dish, or a chamber slide.

7 . The distance determination system of claim 1 , wherein the distance determination unit is further configured to determine a distance of the sample stage from a reference point of the overview camera.

8 . The distance determination system of claim 1 , further comprising:
a drive unit configured to change a distance of the sample stage relative to the overview camera.

9 . The distance determination system of claim 1 , wherein the overview camera or an optical element is positioned at an objective turret of the microscope system.

10 . The distance determination system of claim 1 , wherein the overview camera is a microscope objective and an image sensor.

11 . The distance determination system of claim 1 , wherein the machine-learning-based system is configured for supervised and/or unsupervised learning.

12 . The distance determination system of claim 1 , wherein the machine-learning-based system is a neural network.

13 . The distance determination system of claim 2 , wherein a pixel distance is determined by pixel classification of the segmented images or by way of reference coordinate of detected structures in the at least two recorded digital images using optimization of a target function having a plurality of corresponding structures of the at least two recorded digital images as input data, wherein a result of the target function indicates the pixel distance.

14 . The distance determination system of claim 1 , wherein the machine-learning-based system is further configured to associate corresponding structures in the at least two recorded digital images to identify corresponding structures of the sample stage and/or of a holding frame that has been placed into the sample stage in the two recorded digital images.

15 . The distance determination system of claim 1 , wherein the different viewing angles are caused by a relative movement between the sample stage and the overview camera.

16 . The distance determination system of claim 1 , wherein the different viewing angles are produced by recordings from two overview cameras located with an offset with respect to one another.

17 . A computer-implemented method for a microscope system, wherein the microscope system includes a sample stage with a placement surface for holding a sample carrier, wherein the sample stage is displaceable along at least one direction of extent of a sample plane, and wherein the microscope system comprises an overview camera with a non-telecentric objective for producing digital images, wherein the overview camera is directed at the sample stage, the method comprising:
storing at least two recorded digital images of the sample stage at different viewing angles; associating corresponding structures in the at least two recorded digital images using a trained machine-learning-based system to identify corresponding structures of the sample carrier placed into the sample stage in the two recorded digital images, wherein the trained machine-learning-based system comprises a trained reference model that is trainable by way of a set of annotated training images of sample carriers in a manner such, and the trained machine-learning-based system is thus adapted such, that corresponding structures in the at least two recorded digital images are associable with one another; and determining a distance of a reference point of the sample carrier from a reference point of the overview camera based on the different viewing angles onto the sample stage and a pixel distance of the at least two recorded digital images with respect to one another using the associated corresponding structures contained in the at least two recorded digital images.

18 . A computer program product for a microscope system, wherein the microscope system includes a sample stage with a placement surface for holding a sample carrier, wherein the sample stage is displaceable along at least one direction of extent of a sample plane, and wherein the microscope system includes an overview camera with a non-telecentric objective for producing digital images, wherein the overview camera is directed at the sample stage, wherein the computer program product has program instructions stored thereon, wherein the program instructions, when executed by one or more computers and/or control units and cause the one or the plurality of computers and/or control units to:
store at least two recorded digital images of the sample stage at different viewing angles; associate corresponding structures in the at least two recorded digital images by using a trained machine-learning-based system for identifying corresponding structures of the sample carrier that has been placed into the sample stage in the at least two recorded digital images, wherein the trained machine-learning-based system has a trained reference model that is trainable by way of a set of annotated training images of sample carriers in a manner such, and the trained machine-learning-based system is thus adapted such, that corresponding structures in the at least two recorded digital images are able to be associated with one another; and determine a distance of a reference point of the sample carrier from a reference point of the overview camer

Assignees

• Zeiss Carl Microscopy Gmbh

Inventors

• Amthor Manuel
• Haase Daniel

Classifications

• G02B21/26

  Stages; Adjusting means therefor · CPC title

• G02B21/006

  focusing arrangements; selection of the plane to be imaged · CPC title

• G02B21/367

  providing an output produced by processing a plurality of individual source images, e.g. image tiling, montage, composite images, depth sectioning, image comparison · CPC title

• G02B21/361 Primary

  Optical details, e.g. image relay to the camera or image sensor (G02B21/364 takes precedence; illumination details G02B21/06 and subgroups) · CPC title

• G01B11/14

  for measuring distance or clearance between spaced objects or spaced apertures (G01B11/26 takes precedence; rangefinders G01C3/00) · CPC title