Systems and methods to process electronic images to provide localized semantic analysis of whole slide images

Patent metadata

Field	Value
Publication number	US-12417530-B2
Application number	US-202217813651-A
Country	US
Kind code	B2
Filing date	Jul 20, 2022
Priority date	Oct 1, 2020
Publication date	Sep 16, 2025
Grant date	Sep 16, 2025

Abstract

Systems and methods are disclosed for identifying formerly conjoined pieces of tissue in a specimen, comprising receiving one or more digital images associated with a pathology specimen, identifying a plurality of pieces of tissue by applying an instance segmentation system to the one or more digital images, the instance segmentation system having been generated by processing a plurality of training images, determining, using the instance segmentation system, a prediction of whether any of the plurality of pieces of tissue were formerly conjoined, and outputting at least one instance segmentation to a digital storage device and/or display, the instance segmentation comprising an indication of whether any of the plurality of pieces of tissue were formerly conjoined.

First claim

What is claimed is:

1. A computer-implemented method of training a machine learning model to identify previously conjoined pieces of tissue in a specimen using a synthetic dataset, comprising:
receiving a plurality of digital images associated with a plurality of pathology specimens, at least a portion of the plurality of digital images comprising images of previously conjoined pieces of tissue;
generating a synthetic dataset based on the plurality of digital images;
training a panoptic or instance segmentation model based on the plurality of digital images and the synthetic dataset; and
outputting the panoptic or instance segmentation model to at least one digital storage device, wherein generating the synthetic dataset further comprises:
generating a bank of tissue cores based on the plurality of digital images, the plurality of digital images including one or more annotated digital images;
generating an empty synthetic slide by sampling background or selecting a fixed background color from the one or more annotated digital images;
randomly placing and/or rotating one or more tissue cores and corresponding tissue mask from the bank of tissue cores onto the empty synthetic slide; and
placing the empty synthetic slide with one or more tissue cores and corresponding tissue mask on an array.

2. The computer-implemented method of claim 1 , wherein generating the synthetic dataset further comprises:
generating at least one slide background for images associated with the synthetic dataset;
embedding one or more artifact and/or floater for at least one of the images associated with the synthetic dataset; and
embedding one or more levels and/or cores for at least one of the images associated with the synthetic dataset.

3. The computer-implemented method of claim 2 , wherein the synthetic dataset comprises at least one annotation.

4. The computer-implemented method of claim 1 , wherein each annotated digital image comprises at least one annotation, and each annotation takes a form of a polygon that segments a distinct region of tissue of the pathology specimen, and
the method further comprising:
determining a tissue mask based on each polygon, each tissue mask segmenting tissue from a background;
converting the corresponding tissue mask into a single annotation mask for an entirety of the empty synthetic slide to generate a synthetic digital image;
outputting the synthetic digital image and the at least one annotation; and
saving the synthetic digital image and the at least one annotation to a digital storage device.

5. The computer-implemented method of claim 1 , further comprising adding random noise to the array.

6. The computer-implemented method of claim 1 , further comprising:
running the panoptic or instance segmentation model on a region of interest in a pathology specimen to generate a report; and
outputting the report to a digital storage device and/or display.

7. The computer-implemented method of claim 6 , wherein the report comprises at least one of an identification of a specimen type, an association of two or more tissue fragments that belong together, and/or a correspondence of two or more sectioned levels of the pathology specimen.

8. The computer-implemented method of claim 1 , further comprising:
running the panoptic or instance segmentation model on one or more digital pathology slides to generate at least one of one or more labeled tissue cores and/or levels, one or more artifacts, and/or a background; and
determining whether any tissue regions of the digital pathology slides belong to a similar core at a different level.

9. The computer-implemented method of claim 1 , further comprising:
running the panoptic or instance segmentation model on one or more digital pathology slides to identify one or more tissue regions across the one or more digital pathology slides;
matching the one or more tissue regions as belonging to a similar tissue block at a different level; and
outputting at least one matching tissue regions to a digital storage.

10. The computer-implemented method of claim 9 , where matching the one or more tissues regions comprises using a correlation-based method.

11. The computer-implemented method of claim 9 , where matching the one or more tissue regions comprises using a feature-based method.

12. A system for training a machine learning model to identify previously conjoined pieces of tissue in a specimen using a synthetic dataset, the system comprising:
at least one memory storing instructions; and
at least one processor configured to execute the instructions to perform operations comprising:
receiving a plurality of digital images associated with a plurality of pathology specimens, at least a portion of the plurality of digital images comprising images of previously conjoined pieces of tissue;
generating a synthetic dataset based on the plurality of digital images;
training a panoptic or instance segmentation model based on the plurality of digital images and the synthetic dataset; and
outputting the panoptic or instance segmentation model to at least one digital storage device, wherein generating the synthetic dataset further comprises:
generating a bank of tissue cores based on the plurality of digital images, the plurality of digital images including one or more annotated digital images;
generating an empty synthetic slide by sampling background or selecting a fixed background color from the one or more annotated digital images;
randomly placing and/or rotating one or more tissue cores and corresponding tissue mask from the bank of tissue cores onto the empty synthetic slide; and
placing the empty synthetic slide with one or more tissue cores and corresponding tissue mask on an array.

13. The system of claim 12 , wherein generating the synthetic dataset further comprises:
generating at least one slide background for images associated with the synthetic dataset;
embedding one or more artifact and/or floater for at least one of the images associated with the synthetic dataset; and
embedding one or more levels and/or cores for at least one of the images associated with the synthetic dataset.

14. The system of claim 12 , wherein each annotated digital image comprises at least one annotation, and each annotation takes a form of a polygon that segments a distinct region of tissue of the pathology specimen, and
wherein the operations further comprise:
determining a tissue mask based on each polygon, each tissue mask segmenting tissue from a background;
converting the corresponding tissue mask into a single annotation mask for an entirety of the empty synthetic slide to generate a synthetic digital image;
outputting the synthetic digital image and the at least one annotation; and
saving the synthetic digital image and the at least one annotation to a digital storage device.

15. The system of claim 12 , wherein the operations further comprise:
running the panoptic or instance segmentation model on a region of interest in a pathology specimen to generate a report; and
outputting the report to a digital storage device and/or display.

16. The system of claim 15 , wherein the report comprises at least one of an identification of a specimen type, an association of two or more tissue fragments that belong together, and/or a correspondence of two or more sectioned levels of the pathology specimen.

17. The system of claim 12 , wherein the operations further comprise:
running the panoptic or instance segmentation model on one or more digital pathology slides to identify one or more tissue regi

Assignees

• Paige Ai Inc

Inventors

• Sainson Antoine
• Rothrock Brandon
• Yousfi Razik
• Raciti Patricia
• Hanna Matthew
• Kanan Christopher

Classifications

• G06T7/00

  Image analysis · CPC title

• G06T2207/30024

  Cell structures in vitro; Tissue sections in vitro · CPC title

• G06T2207/20182

  Noise reduction or smoothing in the temporal domain; Spatio-temporal filtering · CPC title

• G06T2207/20081

  Training; Learning · CPC title

• G06T2207/10024

  Color image · CPC title