Multi-model medical scan analysis system using fine-tuned models

Patent metadata

Field	Value
Publication number	US-11748677-B2
Application number	US-202117393726-A
Country	US
Kind code	B2
Filing date	Aug 4, 2021
Priority date	Nov 21, 2018
Publication date	Sep 5, 2023
Grant date	Sep 5, 2023

Abstract

A multi-model medical scan analysis system is operable to generate a generic model by performing a training step on image data of a plurality of medical scans and corresponding labeling data. A plurality of fine-tuned models are generated by performing a fine-tuning step on the generic model. Abnormality detection data is generated for a new medical scan by utilizing the generic model. A first one of the plurality of abnormality types that is detected in the new medical scan is determined based on a corresponding one of the plurality of probability values. Additional abnormality data is generated by performing a fine-tuned inference function on the image data of the new medical scan that utilizes one of the plurality of fine-tuned models that corresponds to the first one of the plurality of abnormality types. The additional abnormality data is transmitted for display.

First claim

What is claimed is:

1. A multi-model medical scan analysis system, comprising:
at least one processor; and
a memory that stores operational instructions that, when executed by the at least one processor, cause the multi-model medical scan analysis system to:
generate a generic model by performing a training step on image data of a plurality of medical scans and corresponding labeling data, wherein the corresponding labeling data of each of the plurality of medical scans includes indicates whether each of a plurality of abnormality types are present in the each of the plurality of medical scans;
generate a plurality of fine-tuned models by performing a fine-tuning step on the generic model, wherein each of the plurality of fine-tuned models corresponds to one of the plurality of abnormality types;
generate abnormality detection data for a new medical scan by utilizing the generic model, wherein the abnormality detection data indicates a plurality of probability values that each indicate a probability that a corresponding one of the plurality of abnormality types is present in the new medical scan;
determine a first one of the plurality of abnormality types that is detected in the new medical scan based on a corresponding one of the plurality of probability values;
generate additional abnormality data for the new medical scan by performing a fine-tuned inference function on the image data of the new medical scan, wherein the fine-tuned inference function utilizes one of the plurality of fine-tuned models that corresponds to the first one of the plurality of abnormality types; and
transmit the additional abnormality data for display.

2. The multi-model medical scan analysis system of claim 1 , wherein the additional abnormality data includes a new probability value indicating whether the first one of the plurality of abnormality types is present.

3. The multi-model medical scan analysis system of claim 2 , wherein the additional abnormality data indicates an increased confidence that first one of the plurality of abnormality types is present in the new medical scan in response to the new probability value indicating a higher probability value than the corresponding one of the plurality of probability values of the abnormality detection data.

4. The multi-model medical scan analysis system of claim 2 , wherein the additional abnormality data indicates that the first one of the plurality of abnormality types is not present in the new medical scan in response to the new probability value comparing unfavorably to a detection threshold.

5. The multi-model medical scan analysis system of claim 1 , wherein performing the fine-tuning step includes modifying a plurality of pre-tuned weights of the generic model.

6. The multi-model medical scan analysis system of claim 5 , wherein performing the fine-tuning step includes performing a training step on a subset of the plurality of medical scans that have corresponding labeling data indicating the first one of the plurality of abnormality types is present in the each of the subset of the plurality of medical scans, and wherein performing the training step includes utilizing a set of initial weights corresponding to the plurality of pre-tuned weights of the generic model.

7. The multi-model medical scan analysis system of claim 1 , wherein performing the fine-tuning step to generate one of the plurality of fine-tuned models corresponding to the first one of the plurality of abnormality types includes utilizing additional labeling data of a subset of the plurality of medical scans, wherein each of the subset of the plurality of medical scans have corresponding labeling data indicating the first one of the plurality of abnormality types is present in the each of the subset of the plurality of medical scans, wherein the additional labeling data indicates characterization labels characterizing the first one of the plurality of abnormality types in the each of the subset of the plurality of medical scans, and wherein the additional abnormality data indicates output data for the characterization labels.

8. The multi-model medical scan analysis system of claim 7 , wherein the output data indicates a plurality of probability values indicating whether each of a plurality of characterization types characterize the first one of the plurality of abnormality types.

9. The multi-model medical scan analysis system of claim 8 , wherein the output data indicates a subset of the plurality of characterization types determined to characterize the first one of the plurality of abnormality types in response to corresponding ones of the plurality of probability values comparing favorably to a characterization threshold.

10. The multi-model medical scan analysis system of claim 1 , wherein the operational instructions, when executed by the at least one processor, further cause the multi-model medical scan analysis system to:
determine a second one of the plurality of abnormality types by determining a second corresponding one of the plurality of probability values compares favorably to a detection threshold;
select a second one of the plurality of fine-tuned models that corresponds to the second one of the plurality of abnormality types;
generate second additional abnormality data for the new medical scan by performing a second fine-tuned inference function on the image data of the new medical scan, wherein the second fine-tuned inference function utilizes the second one of the plurality of fine-tuned models; and
transmit the second additional abnormality data to a client device for display via a display device in conjunction with the additional abnormality data.

11. The multi-model medical scan analysis system of claim 1 , wherein the abnormality detection data indicates region of interest data indicating a location of the first one of the plurality of abnormality types in the image data of the new medical scan, and wherein the operational instructions, when executed by the at least one processor, further cause the multi-model medical scan analysis system to:
select a sub-region of the image data of the new medical scan based on the region of interest data, wherein the fine-tuned inference function is performed only on the sub-region of the image data of the new medical scan.

12. The multi-model medical scan analysis system of claim 11 , wherein performing the fine-tuning step to generate the one of the plurality of fine-tuned models includes:
determining a subset of the plurality of medical scans that have corresponding labeling data indicating the first one of the plurality of abnormality types is present in the each of the subset of the plurality of medical scans;
selecting a plurality of sub-regions by selecting a sub-region of the image data of each of the subset of the plurality of medical scans based on region interest data of the corresponding labeling data indicating a location of the first one of the plurality of abnormality types in the image data of the each of the subset of the plurality of medical scans; and
performing a training step on the plurality of sub-regions to generate the one of the plurality of fine-tuned models.

13. The multi-model medical scan analysis system of claim 1 , wherein the plurality of medical scans correspond to chest x-rays.

14. A method for execution by a multi-model medical scan analysis system, the method comprising:
generating a generic model by performing a training step on image data of a plurality of medical scans and corresponding labeling data, wherein the corresponding labeling data of each of the plurality of medical scans includes indicates whether each of a plurality

Assignees

• Enlitic Inc

Inventors

• Prosky Jordan
• Yao Li
• Poblenz Eric C
• Lyman Kevin
• Covington Ben
• Upton Anthony

Classifications

• G06T12/20

  Inverse problem, i.e. transformations from projection space into object space · CPC title

• G06T11/26

  Drawing of charts or graphs · CPC title

• G06T11/10

  Texturing; Colouring; Generation of textures or colours (retouching, inpainting or scratch removal G06T5/77) · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06N3/09

  Supervised learning · CPC title