Dual domain tracking of target structures

Patent metadata

Field	Value
Publication number	US-2024061908-A1
Application number	US-202318384794-A
Country	US
Kind code	A1
Filing date	Oct 27, 2023
Priority date	Mar 26, 2021
Publication date	Feb 22, 2024
Grant date	—

Abstract

Embodiments described herein provide for determining a probability distribution of a three-dimensional point in a template feature map matching a three-dimensional point in space. A dual-domain target structure tracking end-to-end system receives projection data in one dimension or two dimensions and a three-dimensional simulation image. The end-to-end system extracts a template feature map from the simulation image using segmentation. The end-to-end system extracts features from the projection data, transforms the features of the projection data into three-dimensional space, and sequences the three-dimensional space to generate a three-dimensional feature map. The end-to-end system compares the template feature map to the generated three-dimensional feature map, determining an instantaneous probability distribution of the template feature map occurring in the three-dimensional feature map.

First claim

What we claim is:

1 . A computer-implemented method of location prediction using an end-to-end target structure tracking system comprising:
executing, by a computer, a machine learning model to extract a set of features from imaging projection data associated with a target structure of a patient's anatomy; executing, by the computer, a recurrent neural network to obtain three-dimensional feature map data associated with the target structure, the recurrent neural network configured to sequence the imaging projection data using the set of features in three-dimensional space; and indicating, by the computer, a probability of a three-dimensional point in a template feature map matching a location of a three-dimensional point in the three-dimensional feature map data, the template feature map comprising the target structure.

2 . The computer-implemented method according to claim 1 , further comprising:
receiving, by the computer, segment information associated with the target structure; and extracting, by the computer, the template feature map from the three-dimensional simulation image using the segment information and the three-dimensional simulation image.

3 . The computer-implemented method according to claim 1 , further comprising:
executing, by the computer, a forward projection algorithm to transform the set of features in three-dimensional space into a set of features in two-dimensional space; and feeding, by the computer, the set of features in two-dimensional space into the machine learning model.

4 . The computer-implemented method according to claim 1 , wherein the computer executes a second machine learning model to transform the extracted set of features into a set of features in three-dimensional space.

5 . The computer-implemented method according to claim 1 , further comprising:
determining, by the computer, a deformation field associated with the target structure; and indicating, by the computer, the probability of a three-dimensional point in the template feature map, using the deformation field, matching the location of a three-dimensional point in the three-dimensional feature map data.

6 . The computer-implemented method according to claim 1 , further comprising:
determining, by the computer, a confidence value based on peaks and sidelobes associated with a probability distribution corresponding to the probability of a three-dimensional point in the template feature map matching a location of a three-dimensional point in the three-dimensional feature map data.

7 . The computer-implemented method according to claim 1 , further comprising:
determining, by the computer, a classification for a point in the three-dimensional feature map data based on the probability of a three-dimensional point in the template feature map matching the location of a three-dimensional point in the three-dimensional feature map data satisfying a threshold.

8 . The computer-implemented method according to claim 1 , wherein the imaging projection data is based on at least one of a stereo projection pair or a projection set generated using a multi-view imaging system.

9 . The computer-implemented method according to claim 1 , further comprising:
extracting, by the computer, an additional template feature map from the three-dimensional simulation image associated with an additional target structure; comparing, by the computer, the additional template feature map to the three-dimensional feature map data; and generating, by the computer, a multi-channel probability distribution indicating the probability of a three-dimensional point in the template feature map matching the location of a three-dimensional point in the three-dimensional feature map data and the probability of a three-dimensional point in the additional template feature map matching the location of a three-dimensional point in the three-dimensional feature map data.

10 . The computer-implemented method according to claim 1 , wherein comparing the template feature map to the three-dimensional feature map data comprises convolving each point of the template feature map with each point of the three-dimensional feature map data.

11 . A system comprising:
a server comprising a processor and a non-transitory computer-readable medium containing instructions that when executed by the processor causes the processor to perform operations comprising:
executing a machine learning model to extract a set of features from imaging projection data associated with a target structure of a patient's anatomy;
executing a recurrent neural network to obtain three-dimensional feature map data associated with the target structure, the recurrent neural network configured to sequence the imaging projection data using the set of features in three-dimensional space; and
indicating a probability of a three-dimensional point in a template feature map matching a location of a three-dimensional point in the three-dimensional feature map data, the template feature map comprising the target structure.

12 . The system according to claim 11 , wherein the instructions further cause the server to:
receive segment information associated with the target structure; and extract the template feature map from the three-dimensional simulation image using the segment information and the three-dimensional simulation image.

13 . The system according to claim 11 , wherein the instructions further cause the server to:
execute a forward projection algorithm to transform the set of features in three-dimensional space into a set of features in two-dimensional space; and feed the set of features in two-dimensional space into the machine learning model.

14 . The system according to claim 11 , wherein the computer executes a second machine learning model to transform the extracted set of features into a set of features in three-dimensional space.

15 . The system according to claim 11 , wherein the instructions further cause the server to:
determine a deformation field associated with the target structure; and indicate the probability of a three-dimensional point in the template feature map, using the deformation field, matching the location of a three-dimensional point in the three-dimensional feature map data.

16 . The system according to claim 11 , wherein the instructions further cause the server to:
determine a confidence value based on peaks and sidelobes associated with a probability distribution corresponding to the probability of a three-dimensional point in the template feature map matching a location of a three-dimensional point in the three-dimensional feature map data.

17 . The system according to claim 11 , wherein the instructions further cause the server to:
determine a classification for a point in the three-dimensional feature map data based on the probability of a three-dimensional point in the template feature map matching the location of a three-dimensional point in the three-dimensional feature map data satisfying a threshold.

18 . The system according to claim 11 , wherein the imaging projection data is based on at least one of a stereo projection pair or a projection set generated using a multi-view imaging system.

19 . The system according to claim 11 , wherein the instructions further cause the server to:
extract an additional template feature map from the three-dimensional simulation image associated with an additional target structure; compare the additional template feature map to the three-dimensional feature map data; and generate a

Assignees

• Siemens Healthineers Int Ag

Inventors

• Paysan Pascal
• Walczak Michal
• Zhu Liangjia
• Roggen Toon
• Scheib Stefan

Classifications

• G06T12/20

  Inverse problem, i.e. transformations from projection space into object space · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/0442

  characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] · CPC title

• G06F18/2415 Primary

  based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate · CPC title