Localization and classification of abnormalities in medical images

Patent metadata

Field	Value
Publication number	US-12444054-B2
Application number	US-202519058065-A
Country	US
Kind code	B2
Filing date	Feb 20, 2025
Priority date	Jun 13, 2018
Publication date	Oct 14, 2025
Grant date	Oct 14, 2025

Abstract

Systems and methods are provided for optimizing a deep learning model. A multi-site dataset associated with different clinical sites and a deployment dataset associated with a deployment clinical site are received. A deep learning model is trained based on the multi-site dataset. The trained deep learning model is optimized based on the deployment dataset. The optimized trained deep learning model is output.

First claim

The invention claimed is:

1. A method comprising:
receiving a plurality of input medical images;
localizing a lesion in each of the plurality of input medical images using a trained localization network;
classifying the lesion in each of the plurality of input medical images based on the localizing, using a trained classification network, to generate a classification result for the lesion in each of the plurality of input medical images;
combining the classification results; and
outputting the combined classification results.

2. The method of claim 1 , wherein the plurality of input medical images comprises images acquired with different acquisition protocols.

3. The method of claim 1 , wherein the plurality of input medical images comprises x-ray images.

4. The method of claim 1 , wherein the plurality of input medical images is preprocessed before the receiving.

5. The method of claim 4 , wherein the plurality of input medical images is preprocessed by at least one of removing geometric variability in the plurality of input medical images or normalizing intensity variability in the plurality of input medical images.

6. The method of claim 1 , wherein localizing a lesion in each of the plurality of input medical images using a trained localization network comprises:
generating a localization map for each of the plurality of input medical images.

7. The method of claim 1 , wherein the trained classification network comprises a neural network.

8. The method of claim 7 , wherein inputs into an input layer of the neural network are based on outputs of the localization network.

9. The method of claim 1 , wherein the combined classification results comprise an indication of malignancy.

10. The method of claim 1 , further comprising:
determining a lesion score.

11. The method of claim 10 , wherein the lesion score comprises a PI-RADS (prostate imaging-reporting and data system) score.

12. The method of claim 1 , wherein the trained localization network and the trained classification network are jointly trained.

13. The method of claim 1 , wherein outputting the combined classification results comprises:
displaying the combined classification results.

14. An apparatus comprising:
means for receiving a plurality of input medical images;
means for localizing a lesion in each of the plurality of input medical images;
means for classifying the lesion in each of the plurality of input medical images based on the localizing to generate a classification result for the lesion in each of the plurality of input medical images;
means for combining the classification results; and
means for outputting the combined classification results.

15. The apparatus of claim 14 , wherein the plurality of input medical images comprises images acquired with different acquisition protocols.

16. The apparatus of claim 14 , wherein the plurality of input medical images comprises x-ray images.

17. The apparatus of claim 14 , wherein localizing a lesion in each of the plurality of input medical images using a trained localization network comprises:
generating a localization map for each of the plurality of input medical images.

18. The apparatus of claim 14 , wherein the combined classification results comprise an indication of malignancy.

19. The apparatus of claim 14 , further comprising:
means for determining a lesion score.

20. The apparatus of claim 14 , wherein:
the means for localizing a lesion in each of the plurality of input medical images comprises a trained localization network; and
the means for classifying the lesion in each of the plurality of input medical images comprises a trained classification network.

21. The apparatus of claim 20 , wherein the trained localization network and the trained classification network are jointly trained.

22. The apparatus of claim 14 , wherein the means for outputting the combined classification results comprises:
means for displaying the combined classification results.

23. A non-transitory computer readable medium storing computer program instructions, the computer program instructions when executed by a processor cause the processor to perform operations comprising:
receiving a plurality of input medical images;
localizing a lesion in each of the plurality of input medical images using a trained localization network;
classifying the lesion in each of the plurality of input medical images based on the localizing, using a trained classification network, to generate a classification result for the lesion in each of the plurality of input medical images;
combining the classification results; and
outputting the combined classification results.

24. The non-transitory computer readable medium of claim 23 , wherein the plurality of input medical images comprises images acquired with different acquisition protocols.

25. The non-transitory computer readable medium of claim 23 , wherein the plurality of input medical images comprises x-ray images.

26. The non-transitory computer readable medium of claim 23 , wherein localizing a lesion in each of the plurality of input medical images using a trained localization network comprises:
generating a localization map for each of the plurality of input medical images.

27. The non-transitory computer readable medium of claim 23 , wherein the combined classification results comprise an indication of malignancy.

28. The non-transitory computer readable medium of claim 23 , the operations further comprising:
determining a lesion score.

29. The non-transitory computer readable medium of claim 23 , wherein the trained localization network and the trained classification network are jointly trained.

30. The non-transitory computer readable medium of claim 23 , wherein outputting the combined classification results comprises:
displaying the combined classification results.

Assignees

• Siemens Healthineers Ag

Inventors

• Kamen Ali
• Lou Bin
• Shi Bibo
• Von Roden Nicolas
• Kiefer Berthold
• GRIMM Robert
• Von Busch Heinrich
• Diallo Mamadou
• Meng Tongbai
• Comaniciu Dorin
• Winkel David Jean
• Yu Xin

Classifications

• G06V10/26

  Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion · CPC title

• G06V10/454

  Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN] · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06F18/2411

  based on the proximity to a decision surface, e.g. support vector machines · CPC title