Method and system for pixel super-resolution of multiplexed holographic color images

Patent metadata

Field	Value
Publication number	US-10795315-B2
Application number	US-201716300546-A
Country	US
Kind code	B2
Filing date	May 10, 2017
Priority date	May 11, 2016
Publication date	Oct 6, 2020
Grant date	Oct 6, 2020

Abstract

A method of generating a color image of a sample includes obtaining a plurality of low resolution holographic images of the sample using a color image sensor, the sample illuminated simultaneously by light from three or more distinct colors, wherein the illuminated sample casts sample holograms on the image sensor and wherein the plurality of low resolution holographic images are obtained by relative x, y, and z directional shifts between sample holograms and the image sensor. Pixel super-resolved holograms of the sample are generated at each of the three or more distinct colors. De-multiplexed holograms are generated from the pixel super-resolved holograms. Phase information is retrieved from the de-multiplexed holograms using a phase retrieval algorithm to obtain complex holograms. The complex hologram for the three or more distinct colors is digitally combined and back-propagated to a sample plane to generate the color image.

First claim

What is claimed is:

1. A method of generating a color image of a sample comprising:
obtaining a plurality of low resolution holographic images of the sample using a color image sensor, the sample illuminated simultaneously by light from three or more distinct colors, wherein the illuminated sample casts sample holograms on the color image sensor and wherein the plurality of low resolution holographic images are obtained by relative x, y, and z directional shifts between sample holograms and the color image sensor;
generating a pixel super-resolved hologram of the sample using the plurality of low resolution holographic images obtained by simultaneous illumination of the sample by light from three or more distinct colors;
generating de-multiplexed pixel super-resolved holograms at each of the three or more distinct colors using the pixel super-resolved hologram obtained from the simultaneous illumination of the sample by the light from three or more distinct colors;
retrieving phase information from the de-multiplexed holograms at each of the three or more distinct colors using a phase retrieval algorithm to obtain a complex hologram at each of the three or more distinct colors; and
digitally back-propagating and reconstructing the complex hologram for each of the three or more distinct colors to a sample plane to generate the color image of the sample by combining the reconstruction results of each of the three or more distinct colors.

2. The method of claim 1 , wherein the color image sensor comprises a color filter array (CFA).

3. The method of claim 2 , wherein the CFA is selected from the group consisting of a Bayer filter, RGBE filter, CYYM filter, CYGM filter, RGBW Bayer filter, and RGBW filter.

4. The method of claim 1 , further comprising performing saturation correction on the acquired holograms corresponding to the three or more distinct colors.

5. The method of claim 4 , wherein saturation correction comprises a Bayesian-estimation-based saturation correction.

6. The method of claim 1 , further comprising white-balancing the de-multiplexed holograms corresponding to the three or more distinct colors.

7. The method of claim 1 , wherein the low resolution holographic images of the sample are obtained without an image forming unit that includes a lens, mirror, or combinations of the same.

8. The method of claim 1 , wherein the low resolution holographic images of the sample are obtained with a lens-based imaging system.

9. The method of claim 1 , wherein the sample comprises a biological sample.

10. The method of claim 1 , wherein the phase retrieval algorithm comprises a multi-height (z) phase retrieval algorithm that uses one of the pixel super-resolved holograms at one of the colors and iteratively propagates it to a next (z) measurement plane where the amplitude of the field is partially or entirely replaced with the measured amplitude and continues until a last measurement plane (z) is reached.

11. The method of claim 1 , wherein the three or more distinct colors are selected from red (R), blue (B), and green (G) parts of the electromagnetic spectrum.

12. The method of claim 1 , wherein the three or more distinct colors are emitted simultaneously by a plurality of laser diodes or LEDs.

13. The method of claim 1 , wherein the color image sensor is located in a portable electronic device selected from the group consisting of a mobile phone, webcam, laptop, or tablet computing device.

14. A system for generating color images of a sample comprising:
an optically transparent sample holder configured to hold the sample thereon;
one or more light sources configured to simultaneously output at least three different colors at a distance z 1 from the sample on a first side of the sample holder;
a color image sensor having a color filter array (CFA), the color image sensor disposed on a second side of the sample holder and having an active surface thereof located at a distance z 2 from the sample, wherein z 2 is significantly smaller than z 1 ;
one or more processors configured to execute image processing software thereon, the image processing software:
obtaining a plurality of low resolution holographic images of the sample using the color image sensor, wherein the simultaneously illuminated sample casts sample holograms on the color image sensor and wherein the plurality of low resolution holographic images are obtained by relative x, y, and z directional shifts between sample holograms and the color image sensor;

generating a pixel super-resolved hologram of the sample using the plurality of low resolution holographic images obtained by simultaneous illumination of the sample by light from three or more distinct colors;
generating de-multiplexed pixel super-resolved holograms at each of the at least three different colors using the pixel super-resolved hologram obtained from the simultaneous illumination of the sample by the light from three or more distinct colors;
retrieving phase information from the de-multiplexed holograms at each of the at least three different colors using a phase retrieval algorithm to obtain a complex hologram at each of the three or more distinct colors; and
digitally back-propagating and reconstructing the complex hologram for each of the at least three different colors to a sample plane to generate the color image of the sample by combining the reconstruction results of each of the three or more distinct colors.

15. The system of claim 14 , wherein the relative x, y, and z directional shifts are performed by translation stage coupled to the color image sensor.

16. The system of claim 14 , wherein the relative x, y, and z directional shifts are performed by translation stage coupled to the sample holder.

17. The system of claim 14 , wherein the relative x, y directional shifts are performed by actuating different light sources.

18. The system of claim 14 , wherein the sample holder comprises a flow cell.

19. The system of claim 14 , wherein the one or more light sources comprise a plurality of laser diodes or LEDs.

20. The system of claim 19 , wherein the one or more light sources emit electromagnetic radiation from the red (R), blue (B), and green (G) parts of the electromagnetic spectrum.

21. The system of claim 19 , wherein the one or more light sources are contained in a modular attachment and wherein the color image sensor is contained in a portable electronic device to which the modular attachment is secured.

22. The system of claim 21 , wherein the portable electronic device comprises one of a mobile phone, webcam, tablet computer, or laptop computer.

Assignees

• Univ California

Inventors

• Ozcan Aydogan
• Wu Yichen
• Zhang Yibo
• Luo Wei

Classifications

• H04N23/951

  by using two or more images to influence resolution, frame rate or aspect ratio · CPC title

• H04N25/11

  Arrangement of colour filter arrays [CFA]; Filter mosaics · CPC title

• H04N23/88

  for colour balance, e.g. white-balance circuits or colour temperature control · CPC title

• H04N23/86

  for controlling the colour saturation of colour signals, e.g. automatic chroma control circuits · CPC title

• G03H1/04

  Processes or apparatus for producing holograms (G03H1/26 takes precedence) · CPC title