Method and apparatus for analysis of a sample by light sheet microscopy

Patent metadata

Field	Value
Publication number	US-12313828-B2
Application number	US-202217680695-A
Country	US
Kind code	B2
Filing date	Feb 25, 2022
Priority date	Mar 1, 2021
Publication date	May 27, 2025
Grant date	May 27, 2025

Abstract

A method for light sheet microscope examination of a specimen, the specimen being illuminated with a light sheet by an illumination objective, and a light sheet microscope for performing the method. Light emitted by the specimen is imaged onto an area-type detector by a detection objective. An optical axis of the detection objective encloses with an optical axis of the illumination objective an angle that differs from 0° and from 180° and intersects the light sheet in a light sheet plane. The area-type detector records an image. A neural network analyzes the image to determine whether the light sheet plane is located in a focal plane of the detection objective, and/or in which direction along the optical axis the light sheet plane is located from the focal plane, and/or at what distance, measured along the optical axis of the detection objective, the light sheet plane is from the focal plane.

First claim

The invention claimed is:

1. A method for light sheet microscopic examination of a specimen, comprising:
illuminating the specimen with a light sheet in a light sheet plane via an illumination objective;
imaging light emitted by the specimen onto an area-type detector via a detection objective, wherein an optical axis of the detection objective encloses with an optical axis of the illumination objective an angle that differs from 0° and from 180° and intersects the light sheet plane;
recording at least one image with the area-type detector;
analyzing the at least one image using a neural network to determine:
i. whether or not the light sheet plane is located at least partially in a focal plane of the detection objective, and/or
ii. in which direction along the optical axis of the detection objective the light sheet plane is located from the focal plane, and/or
iii. at what distance, measured along the optical axis of the detection objective, the light sheet plane is located from the focal plane.

2. The method according to claim 1 , further wherein recording at least one image comprises recording a sequence of images either at a constant location of a focus of the detection objective and at different positions of the light sheet along the optical axis of the illumination objective or at different positions of the light sheet relative to the focal plane of the detection objective along the optical axis of the detection objective, transferring the sequence of images to the neural network for analysis and analyzing the sequence of images using the neural network.

3. The method according to claim 2 , further comprising combining the sequence of images to form an overall image before the analysis using the neural network when the sequence of images is recorded at different positions of the light sheet along the optical axis of the illumination objective, and to form a spatial image stack or an overall image when the sequence of images is recorded at different positions along the optical axis of the detection objective.

4. The method according to claim 3 , wherein the sequence of images is combined to form a spatial image stack and the neural network limits its analysis to only a part of the images of the spatial image stack.

5. The method according to claim 1 , further comprising determining a focus quality in order to determine the distance of the light sheet plane from the focal plane of the detection objective for the at least one image.

6. The method according to claim 1 , recording at least on image comprises recording a sequence of images at different positions of the light sheet in relation to the focal plane of the detection objective along the optical axis of the detection objective, and the method further comprises:
combining the sequence of images to form an overall image and transferring the overall image to the neural network for analysis;
determining a focus quality and calculating a score via the neural network for each image in the overall image in order to determine the distance of the light sheet plane from the focal plane of the detection objective; and
determining a position for the light sheet at which it corresponds to the focal plane of the detection objective in a region of an object field of the detection objective is determined on the basis of the score.

7. The method according to claim 1 , further comprising determining whether or not the light sheet plane is located at least partially in a focal plane of the detection objective, and iteratively determining a position for the light sheet plane at which it corresponds to the focal plane of the detection objective in a region of an object field of the detection objective.

8. The method according to claim 7 , further comprising:
a) recording a first sequence of images at different positions of the light sheet in relation to the focal plane of the detection objective along the optical axis of the detection objective;
b) determining the focus quality for each of the images using the neural network;
c) determining, via the neural network, the two images of the first sequence between which the focus quality is best;
d) recording, between the positions along the optical axis assigned to the two images of the first sequence, a further sequence of images at different positions of the light sheet in relation to the focal plane of the detection objective along the optical axis of the detection objective; and
e) repeating steps b) to d) until the distance between the two images between which the focused quality is best falls under a specified value and/or the focus quality of these two images exceeds a specified value.

9. The method according to claim 1 , wherein, for the at least one image, the distance of the light sheet plane from the focal plane is determined by regression via the neural network.

10. The method according to claim 1 , further comprising, after a determination of that position at which the light sheet plane is located at least partially in the focal plane of the detection objective, brining the light sheet plane into correspondence with the focal plane of the detection objective in a region of the object field of the detection objective.

11. The method according to claim 1 , further comprising dividing the at least one image into partial regions and separately determining for each partial region via the neural network as to in which direction and/or at what distance, measured parallel to the optical axis of the detection objective, the light sheet plane is located from the focal plane of the detection objective, and, when a partial region is selected and the distance has been determined, bringing the light sheet plane into correspondence with the focal plane of the detection objective in a region of an object field of the detection objective for this partial region.

12. The method according to claim 1 , further comprising determining a type of the specimen before the at least one image is recorded, specifying parameters for the light sheet in dependence on the type of the specimen, and selecting the neural network.

13. The method according to claim 1 , wherein the neural network used is a deep neural network (DNN).

14. The method of claim 13 , wherein the DNN is a convolutional neural network (CNN).

15. The method according to claim 13 , wherein the illumination takes place with a periodically structured light sheet and a convolutional neural network that is limited to a one-dimensional or two-dimensional convolution is used and/or a receptive field of the convolutional neural network is limited to a period of the light sheet.

16. The method according to claim 1 , further comprising, before the analysis of the at least one image using the neural network, pre-analyzing the at least one image is as to whether the specimen is located in the image and, when this is the case, whether the specimen is suitable for the stated determination.

17. The method according to claim 16 , further comprising dividing the at least one image into partial regions for the pre-analysis which are pre-analyzed individually and/or preanalyzing the at least one image via the neural network.

18. The method according to claim 1 , further comprising preparing the at least one image before the analysis or before a pre-analysis via image processing algorithms.

19. A light sheet microscope for carrying out the method according to claim 1 , comprising:
an illumination device for generating the light sheet, with an illumination objective for illuminating the specimen with the light sheet

Assignees

• Zeiss Carl Microscopy Gmbh

Inventors

• Amthor Manuel
• Haase Daniel
• Kalkbrenner Thomas
• Siebenmorgen Jörg

Classifications

• G02B21/244 Primary

  using image analysis techniques · CPC title

• G02B21/367

  providing an output produced by processing a plurality of individual source images, e.g. image tiling, montage, composite images, depth sectioning, image comparison · CPC title

• G06T7/0002 Primary

  Inspection of images, e.g. flaw detection · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title

• G06T2207/10056

  Microscopic image · CPC title