Medical image analysis using navigation processing

Patent metadata

Field	Value
Publication number	US-11494908-B2
Application number	US-202117408321-A
Country	US
Kind code	B2
Filing date	Aug 20, 2021
Priority date	Mar 22, 2021
Publication date	Nov 8, 2022
Grant date	Nov 8, 2022

Abstract

The present disclosure relates to a medical image analysis method, a medical image analysis device, and a computer-readable storage medium. The medical image analysis method includes receiving a medical image acquired by a medical imaging device; determining a navigation trajectory by performing navigation processing on the medical image based on an analysis requirement, the analysis requirement indicating a disease to be analyzed; extracting an image block set along the navigation trajectory; extracting image features using a first learning network based on the image block set; and determining an analysis result using a second learning network based on the image features and the navigation trajectory.

First claim

What is claimed is:

1. A medical image analysis method, comprising:
receiving a medical image acquired by a medical imaging device;
determining a navigation trajectory by performing a navigation processing on the medical image based on an analysis requirement, the analysis requirement indicating a disease to be analyzed;
extracting an image block set along the navigation trajectory;
extracting image features using a first learning network based on the image block set extracted along the navigation trajectory;
receiving a medical record of a subject of the medical image;
screening out texts relevant to the analysis requirement from the medical record;
extracting text features based on the relevant texts;
fusing the text features and the image features to obtain fused features; and
determining an analysis result using a second learning network based on the fused features and the navigation trajectory.

2. The medical image analysis method according to claim 1 , wherein the navigation trajectory is directional, the image features include a sequence of image block features in the image block set, the first learning network comprises a convolutional neural network, and the second learning network comprises one or a combination of a recurrent neural network and a recursive neural network.

3. The medical image analysis method according to claim 1 , wherein the analysis requirement further comprises at least one of a target region to be analyzed, a manual setting, or an automatic setting for navigation.

4. The medical image analysis method according to claim 1 , wherein the navigation trajectory includes a path or pattern of key points, or a template of structural or functional regions that are associated with each other for the analysis requirement.

5. The medical image analysis method according to claim 1 , wherein for a classification of fatty liver, the navigation trajectory is a path or pattern formed by more than one key points of a liver and a kidney that are manually labelled or automatically detected.

6. The medical image analysis method according to claim 1 , wherein for a vascular lesion detection, the navigation trajectory is a directional path of blood vessel segments formed by key points that are manually labelled or automatically detected, or a complete or partial blood vessel centerline that is automatically computed.

7. The medical image analysis method according to claim 1 , wherein for a cerebral ischemia detection, the navigation trajectory is a template of structural or functional regions corresponding to left and right brains that are manually sketched or automatically detected.

8. The medical image analysis method according to claim 1 , wherein the image features include a sequence of image block features in the image block set,
wherein the second learning network is configured to:
learn at least one of a spatial relationship or a functional relationship based on the image features extracted by the first learning network and the navigation trajectory, and
determine the analysis result based on the at least one of the spatial relationship or the functional relationship.

9. The medical image analysis method according to claim 1 , wherein the text features are extracted based on the relevant texts using a natural language processing model.

10. The medical image analysis method according to claim 1 , wherein fusing the text features and the image features to obtain the fused features further comprises:
merging the text features and the image features to generate merged features, feeding the merged features into a multilayer perceptron, and optimizing the merged features through back propagation to obtain the fused features.

11. The medical image analysis method according to claim 1 , wherein the navigation processing uses a classification learning network established based on a deep learning training process.

12. A medical image analysis device, comprising:
an input/output device configured to receive a medical image acquired by a medical imaging device and a medical record of a subject of the medical image; and
a processor configured to:
determine a navigation trajectory by performing a navigation processing on the medical image based on an analysis requirement, wherein the analysis requirement indicates a disease to be analyzed;
extract an image block set along the navigation trajectory;
extract image features using a first learning network based on the image block set extracted along the navigation trajectory;
screen out texts relevant to the analysis requirement from the medical record;
extract text features based on the relevant texts;
fuse the text features and the image features to obtain fused features; and
determine an analysis result using a second learning network based on the fused features and the navigation trajectory.

13. The medical image analysis device according to claim 12 , wherein the navigation trajectory is directional, the image features include a sequence of image block features in the image block set, the first learning network comprises a convolutional neural network, and the second learning network comprises one or a combination of a recurrent neural network and a recursive neural network.

14. The medical image analysis device according to claim 12 , wherein the analysis requirement further comprises at least one of a target region to be analyzed, a manual setting, or an automatic setting for navigation.

15. The medical image analysis device according to claim 12 , wherein the navigation trajectory includes a path or pattern of key points, or a template of structural or functional regions that are associated with each other for the analysis requirement.

16. The medical image analysis device according to claim 12 , wherein the image features are a sequence of image block features in the image block set,
wherein the second learning network is configured to learn at least one of a spatial relationship or a functional relationship based on the image features extracted by the first learning network and the navigation trajectory, and determine the analysis result based on the at least one of the spatial relationship or the functional relationship.

17. The medical image analysis device according to claim 12 , wherein the navigation processing uses a classification learning network established based on a deep learning training process.

18. A non-transitory computer-readable storage medium on which computer program instructions are stored, wherein when executed by a processor the computer program instructions cause the processor to perform a medical image analysis method, the medical image analysis method comprising:
receiving a medical image acquired by a medical imaging device;
determining a navigation trajectory by performing a navigation processing on the medical image based on an analysis requirement, the analysis requirement indicating a disease to be analyzed;
extracting an image block set along the navigation trajectory;
extracting image features using a first learning network based on the image block set extracted along the navigation trajectory;
receiving a medical record of a subject of the medical image;
screening out texts relevant to the analysis requirement from the medical record;
extracting text features based on the relevant texts;
fusing the text features and the image features to obtain fused features; and
determining an analysis result using a second learning network based on the fused features and the navigation trajectory.

Assignees

• Shenzhen Keya Medical Tech Corporation

Inventors

• Li Ruoping
• Zhao Pengfei
• Li Junhuan
• Ouyang Bin
• Li Yuwei
• Cao Kunlin
• Song Qi

Classifications

• G06V10/454

  Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN] · CPC title

• G06V10/82

  using neural networks · CPC title

• G06T2207/30084

  Kidney; Renal · CPC title

• G06T2207/30056

  Liver; Hepatic · CPC title

• G06T2207/20021

  Dividing image into blocks, subimages or windows · CPC title