Image processing techniques using digital holographic microscopy

Patent metadata

Field	Value
Publication number	US-11977352-B2
Application number	US-202117213982-A
Country	US
Kind code	B2
Filing date	Mar 26, 2021
Priority date	Mar 27, 2020
Publication date	May 7, 2024
Grant date	May 7, 2024

Abstract

Digital holographic microscopy and related image processing techniques are described. A hologram captured in an image frame is split into different depths while a new hologram is being captured. Image slices of the hologram are determined and using free space impulse responses that are pre-calculated at a different precision than processing operations using the holographic data. Each computation is calculated in parallel based on the number of available processing cores and threads. The image slices are combined into a 2D array or 3D array to permit further processing of the combined array to count and size particles in the image frame. The reconstructed hologram is displayed at a subsequent image frame than that used to capture the hologram.

First claim

What is claimed is:

1. A computing device, comprising:
computer readable instructions;
processing circuitry to execute the computer readable instructions to:
calculate, for respective depths of a set of depths, respective free space impulse responses at an impulse response precision, the respective depths determined from a reference plane associated with an image sensor;
calculate, at the respective depths, respective image slices of a hologram from holographic data captured by the image sensor, the holographic data captured after calculation of the free space impulse responses, the image slices calculated at a reconstruction precision that is different from the impulse response precision; and
reconstruct the hologram from the image slices and the free space impulse responses; and

a memory device to store the free space impulse responses.

2. The computing device of claim 1 , wherein the impulse response precision is double precision and the reconstruction precision is single precision, and the processing circuitry is to calculate the free space impulse responses using twice as many bits as are used in reconstruction of the hologram.

3. The computing device of claim 1 , wherein the processing circuitry is to:
multiply forward Fast Fourier transforms (FFTs) of the respective free space impulse responses corresponding to respective ones of the depths with forward FFTs of the holographic data to form respective two-dimensional (2D) matrices corresponding to the respective ones of the depths; and
calculate inverse FFTs of the respective 2D matrices to represent the image slices.

4. The computing device of claim 3 , wherein the processing circuitry is to, for a respective one of the 2D matrices:
calculate a module of the respective one of the 2D matrices; and
scale the module to obtain an intensity of a respective one of the image slices.

5. The computing device of claim 3 , wherein the processing circuitry is to:
set a number of first parallel threads equal to a number of the depths;
set a number of second parallel threads to an integer ratio of a number of available hardware threads over the number of first parallel threads;
simultaneously compute the inverse FFTs of the respective 2D matrices on separate ones of the first parallel threads to form the image slices; and
combine the inverse FFTs to reconstruct the hologram.

6. The computing device of claim 1 , wherein the processing circuitry is to subtract a reference image from the respective image slices to correct for at least one of (a) one or more non-uniformities in illumination of an object forming the hologram or (b) one or more imperfections in a device providing the object for illumination.

7. The computing device of claim 6 , wherein the processing circuitry is to:
select a threshold to convert the respective image slices from respective grayscale image slices to respective binarized image slices having features limited to black and white;
combine the binarized image slices by at least one of:
a logical “or” operation to form a two-dimensional (2D) representation of the object, or
placement of the binarized image slices in a three-dimensional (3D) array that represents a scanned volume of the object to form a 3D representation of the object;

fill holes of detected toroids in, and perform segregation of, at least one of the 2D representation of the object to form a 2D segmented object or the 3D representation of the object to form a 3D segmented object; and
at least one of count or size particles in at least one of the 2D segmented object or the 3D segmented object.

8. The computing device of claim 7 , wherein the processing circuitry is to:
determine whether to limit processing in at least one successive image frame from a current image frame in which the hologram has been captured; and
determine whether to limit processing in the at least one successive image frame by:
evaluating a difference between the count and size of the particles in the 2D segmented object and the count and size of the particles in the 3D segmented object; and
in response to a determination that the difference is less than a threshold difference, limiting combination of second binarized image slices associated with the at least one successive image frame to formation of a 2D representation of an object in the at least one successive image frame without formation of a 3D representation of the object in the at least one successive image frame.

9. The computing device of claim 8 , wherein to determine whether to limit processing in the at least one successive image frame, the processing circuitry is to:
determine whether a threshold time interval has elapsed from a time when the combination of the second binarized image slices was limited;
in response to a determination that the threshold time interval has not elapsed, continue to limit processing in the at least one successive image frame; and
in response to a determination that the threshold time interval has elapsed, determine, based on a next image frame after the determination that the threshold time interval has elapsed, whether to again limit processing in at least one successive image frame after the next image frame.

10. The computing device of claim 1 , wherein the processing circuitry is to calculate the free space impulse responses prior to reconstruction of the hologram.

11. The computing device of claim 1 , wherein the processing circuitry is to calculate the free space impulse responses and reconstruct the hologram without using graphics processing unit (GPU) acceleration.

12. The computing device of claim 1 , wherein the processing circuitry is to calculate the free space impulse responses and reconstruct the hologram at a rate of at least 20 frames per second.

13. The computing device of claim 1 , wherein the processing circuitry is to at least one of perform particle detection, particle counting or particle sizing from a reconstruction of the hologram.

14. A non-transitory device-readable storage medium comprising instructions to cause processing circuitry to at least:
calculate, for respective depths of a set of depths, respective free space impulse responses at double precision, the depths determined from a reference plane;
capture, at an image sensor, a hologram after calculation of the free space impulse responses, the reference plane associated with the image sensor;
calculate, at single precision, respective forward Fast Fourier Transforms (FFTs) of the hologram at respective ones of the depths;
multiply, at respective ones of the depths, respective forward FFTs of the free space impulse response at the respective ones of the depths with the respective forward FFTs of the hologram at the respective ones of the depths to form respective two-dimensional (2D) matrices at the respective ones of the depths;
calculate, for the respective ones of the depths, respective inverse FFTs of the 2D matrices at the respective ones of the depths to form respective image slices at the respective ones of the depths; and
combine the respective image slices from the respective ones of the depths to reconstruct the hologram in at least one of a combined 2D array or a combined three-dimensional (3D) array.

15. The non-transitory device-readable storage medium of claim 14 , wherein the instructions are to cause the processing circuitry to:
set a number of first parallel threads equal to a number of the depths;
set a number of second parallel threads to an integer ratio of a number of available hardware threads over the number of first parallel threads; and
simultaneously compute the invers

Assignees

• Intel Corp

Inventors

• Wenus Jakub
• Cahill Niall
• Muniraj Inbarasan
• Deflumere Ashley
• Mcgrath Michael

Classifications

• G03H1/0443 Primary

  Digital holography, i.e. recording holograms with digital recording means (holobject computation G03H1/0866) · CPC title

• G03H1/0005 Primary

  Adaptation of holography to specific applications (holographic optical element G02B5/32; holographic scanner G02B26/106; recognition using holographic mask G06V10/88; holographic memories G11B7/0065, G11C13/042) · CPC title

• G03H1/0866

  Digital holographic imaging, i.e. synthesizing holobjects from holograms · CPC title

• G03H1/16

  using Fourier transform ({G03H1/10}, G03H1/12, G03H1/14 take precedence; analogue computers G06G, e.g. G06G7/19) · CPC title

• G03H1/2202

  Reconstruction geometries or arrangements · CPC title