Spiking retina microscope

Patent metadata

Field	Value
Publication number	US-11501432-B2
Application number	US-202016913765-A
Country	US
Kind code	B2
Filing date	Jun 26, 2020
Priority date	Jun 26, 2020
Publication date	Nov 15, 2022
Grant date	Nov 15, 2022

Abstract

A spiking retina microscope comprising microscope optics and a neuromorphic imaging sensor. The microscope optics are configured to direct a magnified image of a specimen onto the neuromorphic imaging sensor. The neuromorphic imaging sensor comprises a plurality of sensor elements that are configured to generate spike signals in response to integrated light from the magnified image reaching a threshold. The spike signals may be processed by a processor unit to generate a result, such as tracking biological particles in a specimen comprising biological material.

First claim

What is claimed is:

1. A spiking retina microscope, comprising:
microscope optics; and
a neuromorphic imaging sensor, wherein the microscope optics are configured to direct a magnified image of a specimen onto the neuromorphic imaging sensor and the neuromorphic imaging sensor comprises a plurality of sensor elements that are configured to generate spike signals in response to integrated light from the magnified image reaching a threshold, wherein each of the spike signals identifies a location in the magnified image and a time of occurrence of the spike signal.

2. The spiking retina microscope of claim 1 , further comprising illumination of the specimen.

3. The spiking retina microscope of claim 1 , wherein the plurality of sensor elements are leaky sensor elements.

4. The spiking retina microscope of claim 1 , wherein the plurality of sensor elements are non-leaky sensor elements.

5. The spiking retina microscope of claim 1 , further comprising a processor unit configured to process the spike signals to generate a result.

6. The spiking retina microscope of claim 5 , wherein the processor unit is an embedded coprocessor on the neuromorphic imaging sensor.

7. The spiking retina microscope of claim 5 , wherein the processor unit is a computer or a neuromorphic processor.

8. The spiking retina microscope of claim 5 , wherein the processor unit is configured to process the spike signals to track biological particles in the specimen comprising biological material.

9. A method of examining a specimen, comprising:
placing the specimen in a spiking retina microscope, wherein the spiking retina microscope comprises microscope optics and a neuromorphic imaging sensor, such that a magnified image of the specimen is directed from the microscope optics onto the neuromorphic imaging sensor; and
processing spike signals from the neuromorphic imaging sensor to generate a result, wherein the spike signals are generated by a plurality of sensor elements in the neuromorphic imaging sensor in response to integrated light from the magnified image reaching a threshold, and wherein each of the spike signals identifies a location in the magnified image and a time of occurrence of the spike signal.

10. The method of claim 9 , further comprising illuminating the specimen.

11. The method of claim 9 , wherein the plurality of sensor elements are leaky sensor elements.

12. The method of claim 9 , wherein the plurality of sensor elements are non-leaky sensor elements.

13. The method of claim 9 , wherein processing the spike signals comprises processing the spike signals by an embedded coprocessor on the neuromorphic imaging sensor.

14. The method of claim 9 , wherein processing the spike signals comprises processing the spike signals by a processor unit comprising a computer or a neuromorphic processor.

15. The method of claim 9 , wherein processing the spike signals comprises processing the spike signals using a data cube or back projection.

16. The method of claim 9 , wherein processing the spike signals to generate the result comprises tracking an object in the specimen, noise reduction, background subtraction, or classification of an object in the specimen.

17. A method of tracking biological particles, comprising:
placing a specimen comprising biological material in a spiking retina microscope, wherein the spiking retina microscope comprises microscope optics and a neuromorphic imaging sensor, such that a magnified image of the specimen is directed from the microscope optics onto the neuromorphic imaging sensor; and
processing spike signals from the neuromorphic imaging sensor to track the biological particles in the biological material, wherein the spike signals are generated by a plurality of sensor elements in the neuromorphic imaging sensor in response to integrated light from the magnified image reaching a threshold, and wherein each of the spike signals identifies a location in the magnified image and a time of occurrence of the spike signal.

18. The method of claim 17 , wherein the biological particles comprise biological cells or biological particles within biological cells.

19. The spiking retina microscope of claim 1 , wherein the neuromorphic imaging sensor comprises a dynamic vision sensor (DVS) or a dynamic and active-pixel vision sensor (DAVIS).

20. The spiking retina microscope of claim 1 , wherein the neuromorphic imaging sensor comprises a spiking processing array (SPARR).

Assignees

• Nat Tech & Eng Solutions Sandia Llc

Inventors

• Severa William Mark
• Smith John Darby
• Cardwell Suma George

Classifications

• H04N7/188 Primary

  Capturing isolated or intermittent images triggered by the occurrence of a predetermined event, e.g. an object reaching a predetermined position (signal generation from motion picture films H04N5/253) · CPC title

• G06V20/693

  Acquisition · CPC title

• G06T7/0012 Primary

  Biomedical image inspection · CPC title

• H04N23/57

  Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices · CPC title

• H04N23/55

  Optical parts specially adapted for electronic image sensors; Mounting thereof · CPC title