Fully automatic, template-free particle picking for electron microscopy

Patent metadata

Field	Value
Publication number	US-2020167913-A1
Application number	US-201816621306-A
Country	US
Kind code	A1
Filing date	Jun 13, 2018
Priority date	Jun 13, 2017
Publication date	May 28, 2020
Grant date	—

Abstract

Systems and methods are described for the fully automatic, template-free locating and extracting of a plurality of two-dimensional projections of particles in a micrograph image. A set of reference images is automatically assembled from a micrograph image by analyzing the image data in each of a plurality of partially overlapping windows and identifying a subset of windows with image data satisfying at least one statistic criterion compared to other windows. A normalized cross-correlation is then calculated between the image data in each reference image and the image data in each of a plurality of query image windows. Based on this cross-correlation analysis, a plurality of locations in the micrograph is automatically identified as containing a two-dimensional projection of a different instance of the particle of the first type. The two-dimensional projections identified in the micrograph are then used to determine the three-dimensional structure of the particle.

First claim

What is claimed is:

1 . A method for automatically locating and extracting a plurality of two-dimensional projections of particles in a micrograph image, the method comprising:
receiving, by an electronic processor, a micrograph image of a sample including a plurality of particles of a first type; automatically assembling, by the electronic processor, a set of reference images from the micrograph image by
analyzing image data from the micrograph image in each of a plurality of partially overlapping windows,
identifying a subset of windows with image data satisfying at least one statistic criterion compared to other windows in the plurality of partially overlapping windows, and
including the identified subset of windows in the set of reference images;
analyzing image data from the micrograph image in each of a plurality of query image windows by calculating a cross-correlation between the image data in each of the plurality of query image windows and each reference image of the set of reference images; and automatically identifying, based at least in part on the cross-correlation, a plurality of locations in the micrograph image each containing a two-dimensional projection of a different instance of a particle of the first type.

2 . The method of claim 1 , further comprising:
extracting a plurality of images of two-dimensional projections of the particle based on the plurality of identified locations in the micrograph image; and generating a three-dimensional model of the particle based on the plurality of images of the two-dimensional projections of the particle extracted from the micrograph image.

3 . The method of claim 1 , wherein automatically assembling, by the electronic processor, the set of reference images from the micrograph image further includes dividing an image area of the micrograph image into a plurality of container areas,
wherein analyzing the image data from the micrograph image in each of the plurality of partially overlapping windows includes calculating a mean intensity and a variance for the image data in each of the plurality of overlapping windows, and wherein identifying the subset of windows with image data satisfying the at least one statistic criterion includes identifying, in each containing area of the plurality of container areas,
a window with a highest mean intensity as compared to other windows in the same container area,
a window with a lowest mean intensity as compared to the other windows in the same container area,
a window with a highest variance as compared to the other windows in the same container area, and
a window with a lowest variance as compared to the other windows in the same container area.

4 . The method of claim 1 , wherein analyzing the image data from the micrograph image in each of the plurality of query image windows by calculating a cross-correlation between the image data in each of the plurality of query image windows and each reference image of the set of reference images includes
calculating a normalized cross-correlation between each query image window and each reference image by calculating the cross-correlation between the query image window and the reference image at each of a plurality of offsets.

5 . The method of claim 1 , wherein analyzing the image data from the micrograph image in each of the plurality of query image windows includes calculating a value indicative of a response signal for each query image window based on the calculated cross-correlation.

6 . The method of claim 5 , wherein calculating the value indicative of the response signal for each query image window based on the calculated cross-correlation includes determining, for each query image window of the plurality of query image windows, a number of reference images in the set of reference images for which the cross-correlation between the reference image and the query image window exceeds a threshold.

7 . The method of claim 5 , wherein calculating the value indicative of the response signal for each query image window based on the calculated cross-correlation includes determining, for each query image window of the plurality of query image windows, a maximum normalized cross-correlation value between the query image window and each of the reference windows.

8 . The method of claim 5 , wherein automatically identifying the plurality of locations in the micrograph image each containing a two-dimensional projection of a different instance of the particle of the first type includes
determining that a query image window of the plurality of query image windows includes a two-dimensional projection of an instance of the particle of the first type when the value indicative of the response signal for the query image window exceeds a first threshold; and determining that the query image window does not include a two-dimensional projection of the instance of the particle of the first type when the value indicative of the response signal for the query image window is less than a second threshold.

9 . The method of claim 8 , wherein the first threshold and the second threshold are calculated based at least in part on an analysis of the image data from the micrograph image.

10 . The method of claim 5 , further comprising:
automatically assembling a training set of image data for a classifier based at least in part on the value indicative of the response signal for each of the plurality of query image windows; and training a classifier based on the training set of image data from the micrograph image, wherein automatically identifying a plurality of locations in the micrograph image each containing a two-dimensional projection of a different instance of the particle of the first type includes
analyzing the micrograph image using the trained classifier.

11 . The method of claim 10 , wherein training the classifier based on the training set of image data from the micrograph image includes training a support vector machine classifier.

12 . The method of claim 10 , wherein automatically assembling the training set of image data for the classifier includes
identifying a first subset of query image windows, wherein the first subset of query image windows includes a first defined number of query image windows with the highest values indicative of the response signal, identifying a second subset of query image windows, wherein the second subset of query image windows includes a second defined number of query image windows with the highest values indicative of the response signal, assembling a training set of image data for a particle model based on the first subset of query image windows, and assembling a training set of image data for a noise model based on query image windows for the micrograph image that are not included in the second subset of query image windows.

13 . The method of claim 12 , wherein the second defined number is at least as large as the first defined number and wherein the second subset of query image windows includes all of the query image windows from the first subset of query image windows.

14 . The method of claim 10 , further comprising generating a binary segmentation image of the micrograph based on an output of the classifier, wherein the binary segmentation image classifies each individual pixel in the micrograph image as either a particle pixel or a noise pixel, wherein particle pixels contain image data indicative of at least part of a two-dimensional projection of a particle of the first type.

15 . The method of claim 14 , further comprising identifying a plurality of clusters of particle pixels in the micro

Assignees

• Univ Princeton

Inventors

• Singer Amit
• Heimowitz Ayelet
• Anden Joakim
• Khoo Yuehaw
• Kileel Joseph

Classifications

• G06T2207/10056

  Microscopic image · CPC title

• G06T7/143

  involving probabilistic approaches, e.g. Markov random field [MRF] modelling · CPC title

• G06T2207/30024

  Cell structures in vitro; Tissue sections in vitro · CPC title

• G06T7/174

  involving the use of two or more images · CPC title

• G06T2207/20081

  Training; Learning · CPC title