Optimization of a deep learning model for performing a medical imaging analysis task

Patent metadata

Field	Value
Publication number	US-12561807-B2
Application number	US-202318166128-A
Country	US
Kind code	B2
Filing date	Feb 8, 2023
Priority date	Jun 13, 2018
Publication date	Feb 24, 2026
Grant date	Feb 24, 2026

Abstract

Systems and methods are provided for optimizing a deep learning model. A multi-site dataset associated with different clinical sites and a deployment dataset associated with a deployment clinical site are received. A deep learning model is trained based on the multi-site dataset. The trained deep learning model is optimized based on the deployment dataset. The optimized trained deep learning model is output.

First claim

The invention claimed is:

1 . A method for optimizing a deep learning model comprising:
receiving 1) a multi-site dataset associated with different clinical sites and 2) a deployment dataset associated with a deployment clinical site, wherein the multi-site dataset comprises a first annotated dataset and a second annotated dataset, the second annotated dataset comprising a plurality of sub-datasets each associated with a respective clinical site of the different clinical sites; training a deep learning model for performing a medical image analysis task based on the multi-site dataset, wherein training a deep learning model for performing a medical image analysis task based on the multi-site dataset comprises:
reordering the plurality of sub-datasets of the second annotated dataset based on a similarity between the first annotated dataset and each of the plurality of the sub-datasets; and
updating the deep learning model based on the reordered plurality of sub-datasets, wherein the deep learning model was pretrained based on the first annotated dataset;
optimizing the trained deep learning model for performing the medical image analysis task based on uncertainties associated with the deployment dataset; and outputting the optimized trained deep learning model.

2 . The method of claim 1 , wherein updating the deep learning model based on the reordered plurality of sub-datasets comprises:
for each respective sub-dataset of the reordered plurality of sub-datasets: performing a first comparison and a second comparison, wherein the first comparison is between 1) a performance the deep learning model updated with the respective sub-dataset and any prior sub-datasets from prior iterations and 2) a performance of the deep learning model pretrained based on the first annotated dataset, and wherein the second comparison is between a) a performance of the deep learning model updated with the respective sub-dataset and any prior sub-datasets from prior iterations and b) a performance of the deep learning model trained on the first annotated dataset and the second annotated dataset; and determining whether the deep learning model should be updated with the respective sub-dataset based on the first comparison and the second comparison and, in response to determining that the deep learning model should be updated with the respective sub-dataset, updating the deep learning model with the respective sub-dataset.

3 . The method of claim 2 , wherein determining whether the deep learning model should be updated with the respective sub-dataset based on the first comparison and the second comparison comprises:
determining that the deep learning model should be updated with the respective sub-dataset in response to determining that A) the performance of the deep learning model updated with the respective sub-dataset and any prior sub-datasets from prior iterations is greater than or equal to the performance of the deep learning model pretrained based on the first annotated dataset in the first comparison and B) the performance the deep learning model updated with the respective sub-dataset and any prior sub-datasets from prior iterations is greater than or equal to the performance of the deep learning model trained on the first annotated dataset and the second annotated dataset in the second comparison.

4 . The method of claim 1 , wherein the deployment dataset comprises an annotated deployment dataset and wherein optimizing the trained deep learning model for performing the medical image analysis task based on uncertainties associated with the deployment dataset comprises:
reordering the annotated deployment dataset based on the uncertainties; and optimizing the trained deep learning model based on the reordered annotated deployment dataset.

5 . The method of claim 4 , wherein reordering the annotated deployment dataset based on the uncertainties comprises:
applying the trained deep learning model to each sample in the annotated deployment dataset to generate a respective pseudo-label; and comparing an annotation associated with each sample in the annotated deployment dataset with its respective pseudo-label to determine an uncertainty associated with each sample in the annotated deployment dataset.

6 . The method of claim 4 , wherein optimizing the trained deep learning model based on the reordered annotated deployment dataset comprises:
for each respective sample in the reordered annotated deployment dataset: performing a first comparison and a second comparison, wherein the first comparison is between 1) a performance of the trained deep learning model updated with the respective sample and any prior sample from prior iterations and 2) a performance of the trained deep learning mode, and wherein the second comparison is between a) a performance of the trained deep learning model updated with the respective sample and any prior sample from prior iterations and b) a performance of a deep learning model trained only with the annotated deployment dataset, and determining whether to update the trained deep learning model with the respective sample based on the first comparison and the second comparison and, in response to determining that the trained deep learning model should be updated with the respective sample, updating the trained deep learning model with the respective sample.

7 . The method of claim 6 , wherein determining whether to update the trained deep learning model with the respective sample based on the first comparison and the second comparison comprises:
determining that the trained deep learning model should be updated with the respective sample in response to determining that A) the performance of the trained deep learning model updated with the respective sample and any prior sample from prior iterations is greater than or equal to the performance of the trained deep learning mode minus an error in the first comparison and B) the performance of the trained deep learning model updated with the respective sample and any prior sample from prior iterations is greater than or equal to the performance of a deep learning model trained only with the annotated deployment dataset in the second comparison.

8 . An apparatus for optimizing a deep learning model comprising:
means for receiving 1) a multi-site dataset associated with different clinical sites and 2) a deployment dataset associated with a deployment clinical site, wherein the multi-site dataset comprises a first annotated dataset and a second annotated dataset, the second annotated dataset comprising a plurality of sub-datasets each associated with a respective clinical site of the different clinical sites; means for training a deep learning model for performing a medical image analysis task based on the multi-site dataset, wherein the means for training a deep learning model for performing a medical image analysis task based on the multi-site dataset comprises:
means for reordering the plurality of sub-datasets of the second annotated dataset based on a similarity between the first annotated dataset and each of the plurality of the sub-datasets; and
means for updating the deep learning model based on the reordered plurality of sub-datasets, wherein the deep learning model was pretrained based on the first annotated dataset;
means for optimizing the trained deep learning model for performing the medical image analysis task based on uncertainties associated with the deployment dataset; and means for outputting the optimized trained deep learning model.

9 . The apparatus of claim 8 , wherein the means for updating the deep learning model based on the reordered plurality of sub-datasets comprises:
for each respective sub-dataset of the reordered plurali

Assignees

• Siemens Healthineers Ag

Inventors

• Shi Bibo
• Kamen Ali
• Meng Tongbai
• Diallo Mamadou
• Lou Bin
• GRIMM Robert
• Von Busch Heinrich
• Kiefer Berthold

Classifications

• G06V10/26

  Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion · CPC title

• G06V10/454

  Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN] · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06F18/2411

  based on the proximity to a decision surface, e.g. support vector machines · CPC title