Method and system for disease quantification of anatomical structures

Patent metadata

Field	Value
Publication number	US-12119117-B2
Application number	US-202217726307-A
Country	US
Kind code	B2
Filing date	Apr 21, 2022
Priority date	Apr 23, 2021
Publication date	Oct 15, 2024
Grant date	Oct 15, 2024

Abstract

This disclosure discloses a method and system for predicting disease quantification parameters for an anatomical structure. The method includes extracting a centerline structure based on a medical image. The method further includes predicting the disease quantification parameter for each sampling point on the extracted centerline structure by using a GNN, with each node corresponds to a sampling point on the extracted centerline structure and each edge corresponds to a spatial constraint relationship between the sampling points. For each node, a local feature is extracted based on the image patch for the corresponding sampling point by using a local feature encoder, and a global feature is extracted by using a global feature encoder based on a set of image patches for a set of sampling points, which include the corresponding sampling point and have a spatial constraint relationship defined by the centerline structure. Then, an embed feature is obtained based on both the local feature and the global feature and input into to the node. The method is able to integrate local and global consideration factors of the sampling points into the GNN to improve the prediction accuracy.

First claim

What is claimed is:

1. A computer-implemented method of predicting disease quantification parameters for an anatomical structure, comprising:
receiving a medical image containing the anatomical structure;
extracting, by at least one processor, a centerline structure based on the medical image; and
predicting the disease quantification parameter for each sampling point the extracted centerline structure by using a graph neural network (GNN), wherein each node of the GNN corresponds to a sampling point on the extracted centerline structure and each edge of the GNN corresponds to a spatial constraint relationship between two sampling points, wherein predicting the disease quantification parameter for each sampling point further comprises:
extracting a local feature based on an image patch for the sampling point by using a local feature encoder;
extracting a global feature by using a global feature encoder based on a set of image patches for a set of sampling points, which include the sampling point and have a spatial constraint relationship defined by the centerline structure; and
obtaining an embed feature based on both the local feature and the global feature and inputting the embed feature to the node of the GNN corresponding to the sampling point.

2. The computer-implemented method of claim 1 , wherein each of the local feature encoder and the global feature encoder adopts a dual-level attention mechanism comprising a first level attention module and a second level attention module, wherein the first level attention module is configured to extract a diseased location-aware attention map and the second level attention module is configured to extract a pixel-to-pixel inter-dependence attention map.

3. The computer-implemented method of claim 1 , wherein the GNN includes a high-order GNN.

4. The computer-implemented method of claim 1 , wherein the anatomical structure includes an anatomical tree structure or an anatomical graph structure.

5. The computer-implemented method of claim 1 , wherein the local feature encoder includes a first CNN combined with an attention mechanism for localization, followed by a first self-attention block combined with an attention mechanism for pixel-to-pixel correlation.

6. The computer-implemented method of claim 1 , wherein the global feature encoder includes a second CNN combined with an attention mechanism for localization, followed by a RNN, which is then followed by a second self-attention block combined with an attention mechanism for pixel-to-pixel correlation, wherein the RNN is configured to accumulate information from a root of the anatomical structure to the current sampling point.

7. The computer-implemented method of claim 5 , wherein the attention mechanism for localization includes a residual attention, and the first self-attention block includes a nonlocal network.

8. The computer-implemented method of claim 1 , wherein obtaining an embed feature based on both the local feature and the global feature further comprises concatenating the local feature and the global feature to obtain the embed feature.

9. The computer-implemented method of claim 1 , wherein the GNN includes a graph convolution based neural network or a graph gated based neural network.

10. The computer-implemented method of claim 1 , wherein the anatomical structure includes a vessel structure or an airway structure.

11. A system for predicting disease quantification parameters for an anatomical structure, which comprises:
an interface configured to receive a medical image containing the anatomical structure; and
a processor configured to:
extract a centerline structure based on the medical image; and
predict the disease quantification parameter for each sampling point on the extracted centerline structure by using a graph neural network (GNN), wherein each node of the GNN corresponds to a sampling point on the extracted centerline structure and each edge of the GNN corresponds to a spatial constraint relationship between two sampling points, wherein to predict the disease quantification parameter for each sampling point, the processor is further configured to:
extract a local feature based on an image patch for the sampling point by using a local feature encoder;
extract a global feature by using a global feature encoder based on a set of image patches for a set of sampling points, which include the sampling point and have a spatial constraint relationship defined by the centerline structure; and
obtain an embed feature based on both the local feature and the global feature and inputting the embed feature to the node of the GNN corresponding to the sampling point.

12. The system of claim 11 , wherein each of the local feature encoder and the global feature encoder adopts a dual-level attention mechanism comprising a first level attention module and a second level attention module, wherein the first level attention module is configured to extract diseased location-aware attention map and the second level attention module is configured to extract pixel-to-pixel inter-dependence attention map.

13. The system of claim 11 , wherein the GNN includes a high-order GNN.

14. The system of claim 11 , wherein the anatomical structure includes an anatomical tree structure or an anatomical graph structure.

15. The system of claim 11 , wherein the local feature encoder includes a first CNN combined with an attention mechanism for localization, followed by a first self-attention block combined with an attention mechanism for pixel-to-pixel correlation.

16. The system of claim 11 , wherein the global feature encoder includes a second CNN combined with an attention mechanism for localization, followed by a RNN, which is then followed by a second self-attention block combined with an attention mechanism for pixel-to-pixel correlation, wherein the RNN is configured to accumulate the information from a root of the anatomical structure to the current sampling point.

17. The system of claim 11 , wherein to obtain an embed feature based on both the local feature and the global feature, the processor is further configured to concatenate the local feature and the global feature to obtain the embed feature.

18. The system of claim 11 , wherein the GNN includes a graph convolution based neural network or a graph gated based neural network.

19. The system of claim 11 , wherein the anatomical structure includes a vessel structure or an airway structure.

20. A non-transitory computer-readable storage medium, with computer executable instructions stored thereon, which, when being performed by a processor, perform a method of predicting disease quantification parameters for an anatomical structure, the method comprising:
receiving a medical image containing the anatomical structure;
extracting a centerline structure based on the medical image; and
predicting the disease quantification parameter for each sampling point on the extracted centerline structure by using a graph neural network (GNN) where each node of the GNN corresponds to a sampling point on the extracted centerline structure and each edge of the GNN corresponds to a spatial constraint relationship between two sampling points, wherein predicting the disease quantification parameter for each sampling point further comprises:
extracting a local feature based on an image patch for the sampling point by using a local feature encoder;
extracting a global feature by using a global feature encoder based on a set of image patches for a set of sampling points, which include th

Assignees

• Shenzhen Keya Medical Tech Corporation

Inventors

• Wang Xin
• Yin Youbing
• Kong Bin
• Lu Yi
• Yang Hao-Yu
• Guo Xinyu
• Song Qi

Classifications

• G06N3/0895

  Weakly supervised learning, e.g. semi-supervised or self-supervised learning · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/0442

  characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] · CPC title

• G06N3/045

  Combinations of networks · CPC title