Method and system for localisation microscopy

Patent metadata

Field	Value
Publication number	US-11169368-B2
Application number	US-201816959346-A
Country	US
Kind code	B2
Filing date	Dec 21, 2018
Priority date	Jan 2, 2018
Publication date	Nov 9, 2021
Grant date	Nov 9, 2021

Abstract

Embodiments of the present invention provide a method and system for processing microscopy images to enable localisation analysis of high density raw data, and thereby achieve higher spatial resolution than would otherwise be the case. This is achieved by exploiting temporal redundancies in the image data resulting from close-to emitters that would otherwise be resolved as a single emitter were they to emit or fluoresce at the same time, but which, by virtue of emitting or fluorescing at slightly different (yet potentially overlapping) times, can be subject to temporal filtering by different filters of different temporal bandwidth to resolve the two emitters. Effectively, the different temporal filters have different time constants which work together to effectively highlight the different emission or fluorescence times of the two emitters, to thereby allow the two close-to emitters to be separately resolved.

First claim

The invention claimed is:

1. A method for localisation microscopy, comprising:
receiving a temporal sequence of microscopy image frames captured by a microscope, the image frames being images containing emitters or fluorophores therein;
temporally filtering respective sets of pixel values for respective same pixel positions in the sequence of frames using a plurality of temporal filters having different filter characteristics in order to obtain respective sets of pluralities of temporally filtered pixel values for the respective same pixel positions in the frames; and
providing the respective sets of temporally filtered pixel values as input to a localisation algorithm to permit emitter or fluorophore localisation in dependence thereon.

2. A method according to claim 1 , wherein the temporal filtering comprises:
for a pixel position in the image frames, forming a pixel trace of corresponding pixel values at the same pixel position in the image frames;
applying a first temporal filter to the pixel values in the pixel trace to obtain a first filtered pixel trace; and
applying at least a second temporal filter different to the first temporal filter to the pixel values in the trace to obtain a second filtered pixel trace; and
combining the first filtered pixel trace and the second filtered pixel trace into a single filtered output trace for input to the localisation algorithm.

3. A method according to claim 2 , and further comprising:
applying at least a third temporal filter different to the first and second temporal filters to the pixel values in the trace to obtain a third filtered pixel trace; and
combining the first filtered pixel trace, the second filtered pixel trace, and the third filtered pixel trace into a single filtered output trace for input to the localisation algorithm.

4. A method according to claim 2 , wherein the first, second and third filtered pixel traces are concatenated together to form a combined output trace.

5. A method according to claim 1 , wherein the temporal filters are selected from the group comprising:
i) Harr-Wavelet Kernels;
ii) Butterworth; or
iii) Chebyshev.

6. A method according to claim 1 , wherein the plurality of temporal filters are temporal filters having different temporal characteristics, wherein the temporal characteristics relate to the number of sequential frames around a present frame to which a pixel for which a filtered value is being presently found that contribute to calculation of the filtered value.

7. A method according to claim 1 , and further comprising applying the localisation algorithm to the respective sets of temporally filtered pixel values to identify the positions or emitters or fluorophores in the input images at a higher resolution than would otherwise be possible using the microscope alone.

8. A localisation microscopy system, comprising:
a microscopy system arranged to generate computer readable microscopy image frames;
a processor; and
a computer-readable storage medium storing computer readable instructions that when executed by the processor cause the processor to undertake the following:
i) receive a temporal sequence of microscopy image frames captured by the microscopy system, the image frames being images containing emitter or fluorophores therein;
ii) temporally filter respective sets of pixel values for respective same pixel positions in the sequence of frames using a plurality of temporal filters having different filter characteristics in order to obtain respective sets of pluralities of temporally filtered pixel values for the respective same pixel positions in the frames; and
iii) provide the respective sets of temporally filtered pixel values as input to a localisation algorithm to permit emitter or fluorophore localisation in dependence thereon.

9. A system according to claim 8 , wherein the temporal filtering comprises:
for a pixel position in the image frames, forming a pixel trace of corresponding pixel values at the same pixel position in the image frames;
applying a first temporal filter to the pixel values in the pixel trace to obtain a first filtered pixel trace; and
applying at least a second temporal filter different to the first temporal filter to the pixel values in the trace to obtain a second filtered pixel trace; and
combining the first filtered pixel trace and the second filtered pixel trace into a single filtered output trace for input to the localisation algorithm.

10. A system according to claim 9 , and further comprising:
applying at least a third temporal filter different to the first and second temporal filters to the pixel values in the trace to obtain a third filtered pixel trace; and
combining the first filtered pixel trace, the second filtered pixel trace, and the third filtered pixel trace into a single filtered output trace for input to the localisation algorithm.

11. A system according to claim 9 , wherein the first, second and third filtered pixel traces are concatenated together to form a combined output trace.

12. A system according to claim 8 , wherein the temporal filters are selected from the group comprising:
i) Harr-Wavelet Kernels;
ii) Butterworth; or
iii) Chebyshev.

13. A system according to claim 8 , wherein the plurality of temporal filters are temporal filters having different temporal characteristics, wherein the temporal characteristics relate to the number of sequential frames around a present frame to which a pixel for which a filtered value is being presently found that contribute to calculation of the filtered value.

14. A system according to claim 8 , and further comprising applying the localisation algorithm to the respective sets of temporally filtered pixel values to identify the positions or emitters or fluorophores in the input images at a higher resolution than would otherwise be possible using the microscope alone.

15. One or more non-transitory computer readable media storing computer-readable instructions that when executed by a processor, configure a data processing system to:
receive a temporal sequence of microscopy image frames captured by a microscope, the image frames being images containing emitters or fluorophores therein;
temporally filter respective sets of pixel values for respective same pixel positions in the sequence of frames using a plurality of temporal filters having different filter characteristics in order to obtain respective sets of pluralities of temporally filtered pixel values for the respective same pixel positions in the frames; and
provide the respective sets of temporally filtered pixel values as input to a localisation algorithm to permit emitter or fluorophore localisation in dependence thereon.

16. The non-transitory computer readable media according to claim 15 , wherein the temporal filtering comprises:
for a pixel position in the image frames, forming a pixel trace of corresponding pixel values at the same pixel position in the image frames;
applying a first temporal filter to the pixel values in the pixel trace to obtain a first filtered pixel trace; and
applying at least a second temporal filter different to the first temporal filter to the pixel values in the trace to obtain a second filtered pixel trace; and
combining the first filtered pixel trace and the second filtered pixel trace into a single filtered output trace for input to the localisation algorithm.

17. The non-transitory computer readable media according to claim 16 , and further storing computer-readable instructions that when executed configure the data processing system to:
apply at least a t

Assignees

• King S College London

Inventors

• Marsh Richard
• Cox Susan

Classifications

• G02B21/16 Primary

  adapted for ultraviolet illumination {; Fluorescence microscopes (G02B21/0076 takes precedence)} · CPC title

• G02B27/58

  Optics for apodization or superresolution; Optical synthetic aperture systems · CPC title

• G01N21/6408

  with measurement of decay time, time resolved fluorescence · CPC title

• G01N21/6458

  Fluorescence microscopy (fluorescence microscopes per se G02B21/0076 and G02B21/16) · CPC title

• G02B21/367 Primary

  providing an output produced by processing a plurality of individual source images, e.g. image tiling, montage, composite images, depth sectioning, image comparison · CPC title