Compact multifunctional system for imaging spectroscopy

Patent metadata

Field	Value
Publication number	US-10274420-B2
Application number	US-201515326511-A
Country	US
Kind code	B2
Filing date	Jul 24, 2015
Priority date	Jul 24, 2014
Publication date	Apr 30, 2019
Grant date	Apr 30, 2019

Abstract

A method for obtaining spectral imaging data comprises at least the steps of receiving a sample set of data generated by sampling a spectral property of an image of an object in a spatial basis, wherein the sampling of the spectral property of the image of the object comprises providing a Spectral Filter Array (SFA) by arranging a plurality of SFA elements together to form a surface; configuring each SFA element of the plurality of SFA elements to filter one or more spectral bandwidths centered each at specific wavelengths corresponding to that SFA element, whereby all of the plurality of SFA elements taken together cover a determined spectral range; and setting the specific wavelengths of each SFA element of the plurality of SFA elements on the surface such to obtain a uniform and aperiodic spatial distribution of all of the plurality of SFA elements across the surface. The sampling of the spectral property of the image of the object further comprises providing an image sensor configured to record at each pixel the light filtered by one of the plurality of SFA elements or a subset of the plurality of SFA elements thereby producing one intensity value of light filtered by the one of the plurality of elements or the subset of the plurality of SFA elements per pixel; forming the image of the object on the SFA through a lens or group of lenses; and recording for all of the pixels of the image sensor the spectrally filtered intensity values thereby obtaining a 2-dimensional array of the intensity values corresponding to the image of the object. The method for obtaining spectral imaging data further comprises the step of reconstructing a full 3 dimensional spectral data cube of the imaged object from the sampled 2-dimensional array.

First claim

The invention claimed is:

1. A method for obtaining spectral imaging data, comprising the steps of:
receiving a sample set of data generated by sampling a spectral property of an image of an object in a spatial basis, wherein the sampling of the spectral property of the image of the object comprises
providing a Spectral Filter Array (SFA) by
arranging a plurality of SFA elements together to form a surface,
configuring each SFA element of the plurality of SFA elements to filter one or more spectral bandwidths centered each at specific wavelengths corresponding to that SFA element, wherein all of the plurality of SFA elements taken together cover a determined spectral range, and
setting the specific wavelengths of each SFA element of the plurality of SFA elements on the surface such to obtain a statistically uniform and aperiodic spatial distribution of all of the plurality of SFA elements across the surface, optimizing for local uniformity and filling the surface with a repeating pattern of super-pixels, each super-pixel including a contiguous area of SFA elements including all unique SFA spectral transmissions, and continuously and randomly inter-changing the SFA elements inside of a respective super-pixel until a measured entropy of all the plurality of SFA elements central wavelengths converges to a value indicating a reaching of aperiodicity;
providing an image sensor configured to record at each pixel the light filtered by one of the plurality of SFA elements or a subset of the plurality of SFA elements thereby producing one intensity value of light filtered by the one of the plurality of elements or the subset of the plurality of SFA elements per pixel,
forming the image of the object on the SFA through a lens or group of lenses,
recording for all of the pixels of the image sensor the spectrally filtered intensity values thereby obtaining a 2-dimensional array of the intensity values corresponding to the image of the object,
reconstructing a full 3 dimensional spectral data cube of the imaged object from the sampled 2-dimensional array.

2. The method of claim 1 , wherein the step of reconstructing comprises:
creating a mathematical model as a simple or convolutional neural network;
training the convolutional neural network on a number of pairs of dense 3 dimensional spectral data cubes and synthetically generated sparsely and uniformly sampled spectral imaging data, such that the convolutional neural network learns an interpolation function; and
applying the trained convolutional neural network on the sparsely and uniformly sampled spectral imaging data.

3. The method of claim 2 , wherein the step of reconstructing further comprises:
a convex optimization method based on measurements of the spectral transmission of every pixel from the sparsely and uniformly sampled spectral imaging data.

4. The method of claim 1 , wherein the step of reconstructing further comprises:
creating the mathematical model as a system design matrix from measured transmissions of every pixel from the sparsely and uniformly sampled spectral imaging data; and
inferring non sampled spectral information using deconvolution or non-linear sparse reconstruction methods based on the system design matrix.

5. The method of claim 1 , wherein the step of setting the specific wavelengths of each SFA element of the plurality of SFA elements on the surface such to obtain the statistically uniform and aperiodic spatial distribution of the specific wavelengths of the SFA is deterministic, and includes an aperiodic tiling method.

6. The method of claim 1 , further comprising the steps of:
setting a specific spectral response of each SFA element of the plurality of SFA elements on the surface such to obtain the multiband spectral response distribution of the specific wavelengths of the SFA by
providing a plurality of multiband SFA elements such that neighboring SFA elements are spectrally orthogonal,
providing a plurality of multiband SFA elements such that each SFA element is sensitive in a number of spectral ranges covering a subset of the desired spectral range, and
providing a plurality of multiband SFA elements such that the collection of the sensitive ranges of the individual SFA elements cover the entire desired spectral range.

7. A method for obtaining a real-time monochromatic preview of an imaged object, the imaged object having been obtained using the method of obtaining spectral imaging data of claim 1 , the method for obtaining the real-time monochromatic preview comprising the steps of:
designing a uniform aperiodic Spectral Filter Array (SFA) by arranging a plurality of SFA elements together to form a surface, configuring each SFA element of the plurality of SFA elements to filter a spectral bandwidth centered at a central wavelength corresponding to that SFA element, whereby all of the plurality of SFA elements taken together cover a determined spectral range, and setting the central wavelength of each SFA element of the plurality of SFA elements on the surface such to obtain a distribution of all the central wavelengths to be uniform and aperiodic over the surface,
generating a subset of SFA elements by a 2-dimensional periodic selection,
replacing the SFA elements belonging to the above subset with SFA elements of identical transmission, and
from a 2-dimensional array of pixel intensity values, using a subset of values, corresponding to the subset of periodic SFA elements with identical transmission, to create a lower resolution monochromatic image.

8. A method for obtaining a real-time monochromatic preview of an imaged object, the imaged object having been obtained using the method of obtaining spectral imaging data of claim 1 , the method for obtaining the real-time monochromatic preview comprising the steps of:
designing a uniform aperiodic Spectral Filter Array (SFA) by arranging a plurality of SFA elements together to form a surface, configuring each SFA element of the plurality of SFA elements to filter multiple spectral bands centered at specific wavelengths corresponding to that SFA element, whereby all of the plurality of SFA elements taken together cover a determined spectral range, and setting the central wavelength of each SFA element of the plurality of SFA elements on the surface such that neighboring SFA elements are spectrally orthogonal,
from a 2-dimensional array of pixel intensity values, spatially interpolating each channel corresponding to an SFA element configuration independently at pixel locations where other SFA element configurations are present, and
averaging the independently interpolated channels, to create a high resolution monochromatic image.

9. A non-transitory computer readable medium having a computer-executable program stored thereon, the computer-executable program, when executed by a computing device, configured to perform a method according to claim 1 to reconstruct a 3D-hyperspectral image from a 2D-spatial-spectral dataset.

10. A method for Spectral Filter Array (SFA) element second-degree transmission cancellation through SFA design and subsequent response subtraction comprising at least:
designing a uniformly and aperiodically distributed Spectral Filter Array (SFA) by
arranging a plurality of SFA elements together to form a surface,
configuring each SFA element of the plurality of SFA elements to filter a spectral bandwidth centered at a central wavelength corresponding to that SFA element, whereby all of the plurality of elements taken together cover a determined spectral range, and
setting the central wavelength of each SFA element of the plurality of SFA elements on the surface such to obtain a distribution of all the central wavelengths to be uniform and ap

Assignees

• Ecole Polytechnique Fed Lausanne Epfl

Inventors

• Akhtman Yosef
• Constantin Dragos

Classifications

• G01J3/2823 Primary

  Imaging spectrometer · CPC title

• G01N21/31 Primary

  Investigating relative effect of material at wavelengths characteristic of specific elements or molecules, e.g. atomic absorption spectrometry {(G01N21/72 takes precedence)} · CPC title

• G06V20/13

  Satellite images · CPC title

• G06K9/00657

  Physics · mapped topic

• G01J3/0229

  using masks, aperture plates, spatial light modulators or spatial filters, e.g. reflective filters · CPC title