Method and apparatus for quantitative and depth resolved hyperspectral fluorescence and reflectance imaging for surgical guidance

Patent metadata

Field	Value
Publication number	US-11510600-B2
Application number	US-201615044097-A
Country	US
Kind code	B2
Filing date	Feb 15, 2016
Priority date	Jan 4, 2012
Publication date	Nov 29, 2022
Grant date	Nov 29, 2022

Abstract

An imaging system, such as a surgical microscope, laparoscope, or endoscope or integrated with these devices, includes an illuminator providing patterned white light and/or fluorescent stimulus light. The system receives and images light hyperspectrally, in embodiments using a hyperspectral imaging array, and/or using narrowband tunable filters for passing filtered received light to an imager. Embodiments may construct a 3-D surface model from stereo images, and will estimate optical properties of the target using images taken in patterned light or using other approximations obtained from white light exposures. Hyperspectral images taken under stimulus light are displayed as fluorescent images, and corrected for optical properties of tissue to provide quantitative maps of fluorophore concentration. Spectral information from hyperspectral images is processed to provide depth of fluorophore below the tissue surface. Quantitative images of fluorescence at depth are also prepared. The images are displayed to a surgeon for use in surgery.

First claim

We claim:

1. An optical and image processing system comprising:
a light source adapted to provide, in a first mode, light at a fluorescent stimulus wavelength;
a spatial modulator coupled to modulate light from the light source, forming spatially modulated fluorescent stimulus light, the spatial modulator configured to provide a plurality of different spatial light modulation patterns comprising light at the fluorescent stimulus wavelength, the spatial light modulation patterns each characterized by a spatial frequency;
projection apparatus configured to project onto tissue the spatially modulated fluorescent stimulus light, the tissue being illuminated with the plurality of different spatial light modulation patterns at the fluorescent stimulus wavelength, each spatial light pattern providing light varying in intensity across the tissue according to the spatial frequency;
a hyperspectral camera configured to receive light from the tissue and generate images therefrom; and
an image processing system coupled to the hyperspectral camera and configured with a memory containing machine readable instructions for performing depth resolved, fluorescent imaging by driving the spatial modulator to provide at least a first and a second predetermined spatial-light modulation pattern of the plurality of different spatial light modulation patterns, receiving first images generated by the hyperspectral camera at the fluorescent stimulus wavelength, at least one second image at a first fluorescent emissions wavelength while the tissue is illuminated with light of the fluorescent stimulus wavelength, and at least one third image at a second fluorescent emissions wavelength different from the first fluorescent emissions wavelength while the tissue is illuminated with the light of the fluorescent stimulus wavelength, for processing the received first, second, and third images generated by the hyperspectral camera to generate processed fluorescent images, and for displaying the processed fluorescent images;
wherein the first predetermined spatial light modulation pattern has a same spatial frequency as the second predetermined spatial light modulation pattern, and wherein the first predetermined spatial modulation pattern has spatial phase different from a spatial phase of the second predetermined spatial modulation pattern;
wherein the machine readable instructions for generating the processed fluorescent images comprise instructions for:
determining a relationship between depth and ratios of intensity in the second and third images for a fluorophore in tissue from images of the tissue as illuminated with the plurality of different spatial light modulation patterns;
applying stimulus wavelength light; and
determining a depth of the fluorophore at each pixel based upon the relationship between depth and the ratios of intensity in the second and third images;
where the hyperspectral camera is adapted to pass light received from the tissue through a filter device selected from the group consisting of a tiling pattern of optical filters on photosensors of an image sensor, the tiling pattern of optical filters comprising filters of at least 13 distinct wavelength bands between 400 nanometers and 1000 nanometers, and a tunable filter.

2. The system of claim 1 wherein the relationship between depth and ratios of intensity at the first and the second fluorescent emissions wavelength is determined on a per-pixel basis from the images of the tissue.

3. The system of claim 2 wherein the machine readable instructions further comprise instructions for determining optical properties of tissue, the optical properties of tissue including at least absorbance, and for correcting depth-resolved fluorescent images for the absorbance to provide images of absolute fluorophore concentration.

4. The system of claim 3 wherein the optical properties of tissue are determined at each voxel of a three-dimensional voxel-based model, and the three-dimensional voxel-based model is used in constructing the images of absolute fluorophore concentration.

5. The system of claim 1 wherein the light source is adapted to operate in a second mode providing light at fluorescent emissions wavelength and wherein the machine readable instructions for determining optical properties of tissue comprise instructions for:
determining reflectance and absorbance parameters at each pixel of an image at the fluorescent stimulus wavelength from the received first images, and determining absorbance parameters at each pixel of the image at the emissions wavelength from received images obtained while the light source is operated in the second mode and the tissue is illuminated with a plurality of spatial light patterns; and
using the reflectance and absorbance parameters at stimulus wavelength and the absorbance parameters at the emissions wavelength to correct the processed fluorescent images.

6. The system of claim 5 wherein the machine readable instructions include instructions for providing spatially modulated light when obtaining images from which the reflectance and absorbance parameters at stimulus and emissions wavelengths are determined.

7. An optical and image processing system of claim 6 wherein there are at least two cameras adapted to capture digital stereo images and the machine readable instructions further comprise instructions for extracting a surface profile from the stereo images.

8. An optical and image processing system of claim 7 , wherein the machine readable instructions further comprise instructions for determining an intraoperative location of structures located in preoperative medical images, and for displaying the determined intraoperative location.

9. An optical and image processing system of claim 8 wherein the machine readable instructions further comprise instructions for displaying the determined intraoperative location with the processed fluorescent images.

10. An optical and image processing system of claim 7 , wherein the machine readable instructions further comprise instructions for extracting a surface profile from depth-resolved fluorescent images.

11. An optical and image processing system of claim 7 , further comprising a tracking subsystem adapted to determine a location and viewing angle of a display and wherein the machine readable instructions further comprise instructions for displaying rendered information selected from the group consisting of depth-resolved fluorescent images and intraoperative locations of structures as viewed from the determined location and viewing angle.

12. The system of claim 1 where the light source further comprises a broadband illumination source coupled to provide light through a tunable filter having a passband at the fluorescent stimulus wavelength, light from the tunable filter being provided to the spatial modulator; and wherein the spatial modulator and projection apparatus is configurable to illuminate the tissue with the spatial light pattern in three different spatial phases.

13. The system of claim 1 where the hyperspectral camera is a wavelength-selective camera
where the light from the tissue passes through the tunable filter before reaching an imager.

14. The system of claim 1 wherein, during processing to generate the processed fluorescent images, the image processing system is configured to determine scattering and absorption parameters of the tissue at voxels of a three-dimensional model of the tissue and to use the scattering and absorption parameters to correct fluorescent emissions received from the hyperspectral camera at the fluorescent emissions wavelength.

15. An optical and image proc

Assignees

• Dartmouth College

Inventors

• Valdes Pablo
• Leblond Frederic
• Paulsen Keith D
• Wilson Brian Campbell
• Roberts David W
• Jermyn Michael

Classifications

• A61B5/14556 Primary

  by fluorescence (A61B5/14555 takes precedence) · CPC title

• A61B1/000094

  extracting biological structures · CPC title

• G01J3/0229

  using masks, aperture plates, spatial light modulators or spatial filters, e.g. reflective filters · CPC title

• A61B5/0071

  by measuring fluorescence emission · CPC title

• A61B1/043

  for fluorescence imaging · CPC title