Volumetric imaging techniques

Patent metadata

Field	Value
Publication number	US-12418731-B2
Application number	US-202218556268-A
Country	US
Kind code	B2
Filing date	Apr 20, 2022
Priority date	Apr 20, 2021
Publication date	Sep 16, 2025
Grant date	Sep 16, 2025

Abstract

A volumetric imaging system implements obtaining raw image data of a sample volume of a sample material, the raw image data comprising light field data of the sample volume acquired using a microlens array disposed in front of a camera; and analyzing the raw image data using an image analysis pipeline configured to localize the objects of interest in the raw image data to obtain classified image data in which the objects of interest have been identified, the image analysis pipeline being configured to process the raw image data to improve signal extraction at depth in the scattering material to maximize localization accuracy.

First claim

What is claimed is:

1. A volumetric imaging system comprising:
a processor; and
a computer-readable medium storing executable instructions for causing the processor to perform operations comprising:
obtaining raw image data of a sample volume of a sample material, the raw image data comprising light field data of the sample volume acquired using a mesoscope or microscope having a microlens array disposed in front of a camera, the sample material comprising scattering or non-scattering tissue exhibiting time varying changes of light signals from objects of interest; and
analyzing the raw image data using an image analysis pipeline configured to localize the objects of interest in the raw image data to obtain classified image data in which the objects of interest have been identified, the image analysis pipeline being configured to process the raw image data to improve signal extraction at depth in scattering material to maximize localization accuracy.

2. The volumetric imaging system of claim 1 , wherein to obtain the raw image data of a sample volume of a scattering material, the volumetric imaging system is configured to receive the raw image data from an optical path comprising a custom tube lens configured to correct an output of an objective lens to provide at least a multi-millimeter field of view (FOV).

3. The volumetric imaging system of claim 2 , wherein the microlens array is placed in an image plane of the custom tube lens.

4. The volumetric imaging system of claim 2 , wherein the custom tube lens provides an optical resolution of at least 600 line pairs per millimeter across the entire FOV.

5. The volumetric imaging system of claim 2 , wherein the custom tube lens provides telecentric, homogenous illumination across both an image plane and focal plane of the sample material.

6. The volumetric imaging system of claim 2 , wherein the custom tube lens comprises three cemented doublet lenses in a configuration based on a Petzval objective design form, the custom tube lens being configured to correct aberrations such that diffraction-limited resolution is restored at an NA of 0.4 up to a field radios of at least 1.9 millimeters and reduces field curvature to approximately 60 μm.

7. The volumetric imaging system of claim 1 , wherein the microlens array is mounted on a kinematic mount affixed to a housing of the camera, and wherein the microlens array is aligned with the camera by:
illuminating the microlens array with a collimated green laser been;
adjusting kinematic mount until uniform and focus spots are observable behind all microlenses of the microlens array; and
aligning the microlens array parallel with a pixel grid of the camera.

8. The volumetric imaging system of claim 1 , wherein analyzing the raw image data using an image analysis pipeline further comprises:
analyzing the raw image data to provide non-rigid motion correction on the raw image data to produce motion-corrected image data.

9. The volumetric imaging system of claim 8 , wherein to analyze the raw image data to provide non-rigid motion correction on the raw image data to produce motion corrected image data, the executable instructions include instructions for causing the processor to perform operations of:
rearranging pixels in the raw image data into a plurality of sub-aperture images, wherein each sub-aperture image comprises pixels having a same relative position to a nearest microlens of the microlens array, such that each sub-aperture image corresponds to a perspective view onto the sample material from a respective angle; and
determining non-rigid motion information by analyzing the plurality of sub-aperture images.

10. The volumetric imaging system of claim 9 , wherein determining the non-rigid motion information for the plurality of sub-aperture images further comprises:
determining the non-rigid motion information by applying an FFT-based registration pipeline to the plurality of sub-aperture images.

11. The volumetric imaging system of claim 10 , wherein analyzing the raw image data using an image analysis pipeline further comprises:
removing background fluorescence contributions from the motion-corrected image data to produce background-corrected image data.

12. The volumetric imaging system of claim 11 , wherein analyzing the raw image data using an image analysis pipeline further comprises:
filtering the background-corrected image data using a shape-based filter to remove predetermined shapes from background-corrected image data to generate shape-filtered image data.

13. The volumetric imaging system of claim 12 , wherein analyzing the raw image data using an image analysis pipeline further comprises:
filtering the shape-filtered image data to emphasize objects of interest within the shape-filtered image data to generate summary image data.

14. The volumetric imaging system of claim 13 , wherein analyzing the raw image data using an image analysis pipeline further comprises:
performing background-rejecting phase-space reconstruction on the summary image data to remove background light from outside of the sample volume to generate reconstructed image data.

15. The volumetric imaging system of claim 14 , wherein analyzing the raw image data using an image analysis pipeline further comprises:
performing a morphological segmentation on the reconstructed image data to segment objects of interest from regions surrounding the objects of interest to generate morphologically segmented image data.

16. The volumetric imaging system of claim 15 , wherein the sample volume is a three-dimensional (3D) volume, wherein to perform the morphological segmentation on the reconstructed image data to segment objects of interest from regions surrounding the objects of interest to generate morphologically segmented image data, the executable instructions include instructions for causing the processor to perform operations of:
segmenting the 3D volume into a plurality of 2D planes; and
convolving each respective 2D plane of the plurality of 2D planes with a first kernel associated with an expected shape of an object of interest and a second kernel associated with an expected shape of a region surrounding the objects of interest to identify locations of the objects of interest within the 3D volume.

17. The volumetric imaging system of claim 15 , wherein analyzing the raw image data using an image analysis pipeline further comprises:
performing iterative spatiotemporal demixing on the morphologically segmented image data to demix signals associated with the objects of interest with signals associated with the regions surrounding the objects of interest and generate spatiotemporally demixed data.

18. The volumetric imaging system of claim 17 , wherein analyzing the raw image data using an image analysis pipeline further comprises:
performing a local demixing on the spatiotemporally demixed data to generate locally demixed data in which signals associated with the regions surrounding the objects of interest have been removed by solving an optimization problem that seeks to reduce crosstalk between each respective object of interest and a corresponding respective region surrounding each object of interest; and
analyzing the locally demixed data using a machine learning model trained to classify activity traces to provide classified image data.

19. A method implemented in a system for volumetric imaging, the method comprising:
obtaining raw image data of a sample volume of a sample material, the raw image data comprising light field data of the sam

Assignees

• Univ Rockefeller
• The Rockfeller Univ

Inventors

• Vaziri Alipasha
• Noebauer Tobias
• Zhang Yuanlong

Classifications

• G06T2207/30101

  Blood vessel; Artery; Vein; Vascular · CPC title

• G06T2207/30016

  Brain · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title

• G06T2207/20056

  Discrete and fast Fourier transform, [DFT, FFT] · CPC title

• G06T2207/20036

  Morphological image processing · CPC title