Systems and methods for locating patient features

Patent metadata

Field	Value
Publication number	US-2021121244-A1
Application number	US-201916665804-A
Country	US
Kind code	A1
Filing date	Oct 28, 2019
Priority date	Oct 28, 2019
Publication date	Apr 29, 2021
Grant date	—

Abstract

Methods and systems for locating one or more target features of a patient. For example, a computer-implemented method includes receiving a first input image; receiving a second input image; generating a first patient representation corresponding to the first input image; generating a second patient representation corresponding to the second input image; determining one or more first features corresponding to the first patient representation in a feature space; determining one or more second features corresponding to the second patient representation in the feature space; joining the one or more first features and the one or more second features into one or more joined features; determining one or more landmarks based at least in part on the one or more joined features; and providing a visual guidance for a medical procedure based at least in part on the information associated with the one or more landmarks.

First claim

What is claimed is:

1 . A computer-implemented method for locating one or more target features of a patient, the method comprising:
receiving a first input image; receiving a second input image; generating a first patient representation corresponding to the first input image; generating a second patient representation corresponding to the second input image; determining one or more first features corresponding to the first patient representation in a feature space; determining one or more second features corresponding to the second patient representation in the feature space; joining the one or more first features and the one or more second features into one or more joined features; determining one or more landmarks based at least in part on the one or more joined features; and providing a visual guidance for a medical procedure based at least in part on the information associated with the one or more landmarks; wherein the computer-implemented method is performed by one or more processors.

2 . The computer-implemented method of claim 1 , further comprising:
acquiring the first input image using a visual sensor; and acquiring the second input image using a medical scanner.

3 . The computer-implemented method of claim 2 , wherein the visual sensor includes at least one of a RGB sensor, a RGBD sensor, a laser sensor, a FIR sensor, a NIR sensor, an X-ray sensor, and a lidar sensor.

4 . The computer-implemented method of claim 2 , wherein the medical scanner includes at least one of an ultrasound scanner, an X-ray scanner, a MR scanner, a CT scanner, a PET scanner, a SPECT scanner, and a RGBD scanner.

5 . The computer-implemented method of claim 1 , wherein:
the first input image is two-dimensional; and the second input image is three-dimensional.

6 . The computer-implemented method of claim 1 , wherein:
the first patient representation includes one selected from an anatomical image, a kinematic model, a skeleton model, a surface model, a mesh model, and a point cloud; and the second patient representation includes one selected from an anatomical image, a kinematic model, a skeleton model, a surface model, a mesh model, a point cloud, and a three-dimensional volume.

7 . The computer-implemented method of claim 1 , wherein:
the one or more first features includes one selected from a pose, a surface, and an anatomical landmark; and the one or more second features includes one selected from a pose, a surface, and an anatomical landmark.

8 . The computer-implemented method of claim 1 , wherein the joining the one or more first features and the one or more second features into one or more joined features includes:
matching the one or more first features to the one or more second features; and aligning the one or more first features to the one or more second features.

9 . The computer-implemented method of claim 8 , wherein the matching the one or more first features to the one or more second features includes pairing each first feature of the one or more first features to a second feature of the one or more second features.

10 . The computer-implemented method of claim 8 , wherein:
determining one or more first features corresponding to the first patient representation in a feature space includes determining one or more first coordinates corresponding to the one or more first features; determining one or more second features corresponding to the second patient representation in the feature space includes determining one or more second coordinates corresponding to the one or more second features; and aligning the one or more first features to the one or more second features includes aligning the one or more first coordinates to the one or more second coordinates.

11 . The computer-implemented method of claim 1 , wherein the information associated with the one or more landmarks includes one of landmark name, landmark coordinate, landmark size, and landmark property.

12 . The computer-implemented method of claim 1 , wherein the providing a visual guidance for a medical procedure includes localizing a display region onto a target region based at least in part on a selected target landmark.

13 . The computer-implemented method of claim 1 , wherein the providing a visual guidance for a medical procedure includes mapping and interpolating the one or more landmarks onto a patient coordinate system.

14 . The computer-implemented method of claim 1 , wherein:
the medical procedure is an interventional procedure; and the providing a visual guidance for a medical procedure includes providing information associated with one or more targets of interest, the information includes a number of targets, one or more target coordinates, one or more target sizes, or one or more target shapes.

15 . The computer-implemented method of claim 1 , wherein:
the medical procedure is a radiation therapy; and the providing a visual guidance for a medical procedure includes providing information associated with a region of interest; the information includes a region size or a region shape.

16 . The computer-implemented method of claim 1 , wherein the computer-implemented method is performed by one or more processors using a machine learning model.

17 . The computer-implemented method of claim 16 , further comprising training the machine learning model by at least:
determining one or more losses between the one or more first features and the one or more second features; and modifying one or more parameters of the machine learning model based at least in part on the one or more losses.

18 . The computer-implemented method of claim 17 , wherein modifying one or more parameters of the machine learning model based at least in part on the one or more losses includes:
modifying one or more parameters of the machine learning model to reduce the one or more losses.

19 . A system for locating one or more target features of a patient, the system comprising:
an image receiving module configured to:
receive a first input image; and
receive a second input image;
a representation generating module configured to:
generate a first patient representation corresponding to the first input image; and
generate a second patient representation corresponding to the second input image;
a feature determining module configured to:
determine one or more first features corresponding to the first patient representation in a feature space; and
determine one or more second features corresponding to the second patient representation in the feature space; a feature joining module configured to join the one or more first features and the one or more second features into one or more joined features; a landmark determining module configured to determine one or more landmarks based at least in part on the one or more joined features; and a guidance providing module configured to provide a visual guidance based at least in part on the information associated with the one or more landmarks.

20 . A non-transitory computer-readable medium with instructions stored thereon, that when executed by a processor, causes the processor to perform one or more processes including:
receiving a first input image; receiving a second input image; generating a first patient representation corresponding to the first medical image; generating a second patient representation corresponding to the second medical image; determining one or more first features corresponding to the first patient representation

Assignees

• Shanghai United Imaging Intelligence Co Ltd

Inventors

• Innanje Arun
• Wu Ziyan
• Karanam Srikrishna

Classifications

• G06F18/213

  Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title

• G06T2207/30004

  Biomedical image processing · CPC title