Systems and methods of metadata and image management for reviewing data from transmission electron microscope (TEM) sessions

Patent metadata

Field	Value
Publication number	US-11755639-B2
Application number	US-202217931466-A
Country	US
Kind code	B2
Filing date	Sep 12, 2022
Priority date	Apr 7, 2021
Publication date	Sep 12, 2023
Grant date	Sep 12, 2023

Abstract

Disclosed herein are methods and systems of metadata management for reviewing data from microscopy experimental sessions. Image data from an experimental session is stored in an archive at one or more filepath locations, either locally or on a network. Metadata associated with the image data is stored in a database with a reference to the filepath where the raw image is stored, such that the metadata is associated in the database with the image data. A user can perform post-experimental filtering, sorting, and searching of the underlying image data using the metadata, which allows the image data to be analyzed without duplication of the image data and without manual review of each individual image. The filtered data is presented in an interactive timeline format.

First claim

What is claimed is:

1. A system for generating a visual representation of experimental data from an experimental session on a transmission electron microscope using metadata from the experimental session, the system comprising:
a transmission electron microscope having a detector for capturing images of a sample during the experimental session; and
a computer system communicatively coupled to the transmission electron microscope,
the computer system comprising:
a memory; and
at least one processor configured for:
receiving, from the transmission electron microscope, a plurality of images of the sample captured by the detector of the transmission electron microscope during the experimental session;
storing, in the memory, the plurality of images of the sample received from the transmission electron microscope;
storing, in the memory, metadata associated with the plurality of images, wherein each image of the plurality of images is associated with specific metadata for said image, and wherein the specific metadata for said image includes a reference to the stored version of said image; and
generating a subset of the plurality of images, wherein the images in the subset of the plurality of images are selected based on the metadata associated with each selected image;
wherein the subset of the plurality of images is generated without storing in the memory a separate copy of each image in the subset of the plurality of images.

2. The system of claim 1 , wherein the at least one processor is further configured for generating a preview of each image in the subset of the plurality of images based on the metadata.

3. The system of claim 1 , wherein the at least one processor is further configured for displaying at least some images in the subset of the plurality of images, wherein each displayed image is displayed using a reduced size image from the stored version of each displayed image, with each displayed image being displayed in response to user input.

4. The system of claim 3 , wherein each displayed image is displayed with information related to the displayed image overlaid onto the displayed image, wherein the overlaid information is based on the metadata.

5. The system of claim 4 , wherein the at least one processor is further configured for storing the subset of the plurality of images for further review without storing a separate copy of each image in the subset of the plurality of images, wherein the subset of the plurality of images is stored using references in the metadata to the stored versions of the images in the subset of the plurality of images.

6. The system of claim 1 , wherein the at least one processor is further configured for filtering images in the plurality of images based on the metadata.

7. The system of claim 6 , wherein the images in the plurality of images are filtered to generate the subset of the plurality of images such that the subset of the plurality of images represents images from the plurality of images that have at least one similar property, wherein the at least one similar property is determined based on the metadata.

8. The system of claim 6 , wherein the filtered images reduce an active dataset of the plurality of images.

9. The system of claim 1 , wherein access to the stored versions of the images in the plurality of images occurs in approximately real-time using the metadata.

10. The system of claim 1 , wherein the images of the plurality of images are indexed based on the metadata.

11. A non-transitory computer-readable storage medium, the non-transitory computer-readable storage medium storing instructions to be implemented on at least one computing device including at least one processor, the instructions when executed by the at least one processor cause the at least one computing device to perform a method of generating a visual representation of experimental data from an experimental session on a transmission electron microscope using metadata from the experimental session, the method comprising:
receiving, from the transmission electron microscope, a plurality of images of a sample captured by a detector of the transmission electron microscope during the experimental session;
storing the plurality of images of the sample received from the transmission electron microscope;
storing metadata associated with the plurality of images, wherein each image of the plurality of images is associated with specific metadata for said image, and wherein the specific metadata for said image includes a reference to the stored version of said image; and
generating a subset of the plurality of images, wherein the images in the subset of the plurality of images are selected based on the metadata associated with each selected image;
wherein the subset of the plurality of images is generated without storing a separate copy of each image in the subset of the plurality of images.

12. The non-transitory computer-readable storage medium of claim 11 , wherein the method performed by the at least one computing device further comprises generating a preview of each image in the subset of the plurality of images based on the metadata.

13. The non-transitory computer-readable storage medium of claim 11 , wherein the method performed by the at least one computing device further comprises displaying at least some images in the subset of the plurality of images, wherein each displayed image is displayed using a reduced size image from the stored version of each displayed image, with each displayed image being displayed in response to user input.

14. The non-transitory computer-readable storage medium of claim 13 , wherein each displayed image is displayed with information related to the displayed image overlaid onto the displayed image, wherein the overlaid information is based on the metadata.

15. The non-transitory computer-readable storage medium of claim 14 , wherein the method performed by the at least one computing device further comprises storing the subset of the plurality of images for further review without storing a separate copy of each image in the subset of the plurality of images, wherein the subset of the plurality of images is stored using references in the metadata to the stored versions of the images in the subset of the plurality of images.

16. The non-transitory computer-readable storage medium of claim 11 , wherein the method performed by the at least one computing device further comprises filtering images in the plurality of images based on the metadata.

17. The non-transitory computer-readable storage medium of claim 16 , wherein the images in the plurality of images are filtered to generate the subset of the plurality of images such that the subset of the plurality of images represents images from the plurality of images that have at least one similar property, wherein the at least one similar property is determined based on the metadata.

18. The non-transitory computer-readable storage medium of claim 16 , wherein the filtered images reduce an active dataset of the plurality of images.

19. The non-transitory computer-readable storage medium of claim 11 , wherein access to the stored versions of the images in the plurality of images occurs in approximately real-time using the metadata.

20. The non-transitory computer-readable storage medium of claim 11 , wherein the images of the plurality of images are indexed based on the metadata.

Assignees

• Protochips Inc

Inventors

• Walden Ii Franklin Stampley
• Damiano Jr John
• Nackashi David P
• Gardiner Daniel Stephen
• Uebel Mark
• Franks Alan Philip
• Friend Joshua
• Marusak Katherine Elizabeth

Classifications

• G06T11/26

  Drawing of charts or graphs · CPC title

• G06F16/51 Primary

  Indexing; Data structures therefor; Storage structures · CPC title

• G06F16/535 Primary

  Filtering based on additional data, e.g. user or group profiles · CPC title

• G06F16/168

  Details of user interfaces specifically adapted to file systems, e.g. browsing and visualisation, 2d or 3d GUIs (query results presentation G06F16/156) · CPC title

• G16H30/20

  for handling medical images, e.g. DICOM, HL7 or PACS · CPC title