Multi-task progressive networks for patient modeling for medical scans

Patent metadata

Field	Value
Publication number	US-11559221-B2
Application number	US-201916361553-A
Country	US
Kind code	B2
Filing date	Mar 22, 2019
Priority date	Mar 22, 2019
Publication date	Jan 24, 2023
Grant date	Jan 24, 2023

Abstract

For training for and performance of patient modeling from surface data in a medical system, a progressive multi-task model is used. Different tasks for scanning are provided, such as landmark estimation and patient pose estimation. One or more features learned for one task are used as fixed or constant features in the other task. This progressive approach based on shared features increases efficiency while avoiding reductions in accuracy for any given task.

First claim

What is claimed is:

1. A method for patient modeling from surface data in a medical system, the method comprising:
capturing, with a sensor, an outer surface of a patient, the surface data being from the capturing of the outer surface of the patient;
estimating, by a processor, a first value of a first patient characteristic from the surface data as a first task, the first patient characteristic estimated by a first machine-learned model of a progressive multi-task network;
estimating, by the processor, a second value of a second patient characteristic from the surface data as a second task, the second patient characteristic estimated by a second machine-learned model of the progressive multi-task network, the second machine-learned model including features learned in training the first machine-learned model, the second machine-learned model having been trained after the first machine-learned model using the features learned in training the first machine-learned model as unchanging in the training of the second machine-learned model; and
controlling, by the processor, scanning by a medical scanner of the patient based on the first and second values of the first and second patient characteristics.

2. The method of claim 1 wherein capturing comprises capturing with the sensor being a depth sensor.

3. The method of claim 1 wherein capturing comprises capturing with the sensor being a camera where the surface data is based on optical measurements.

4. The method of claim 1 wherein the first characteristic is a different type of characteristic than the second characteristic, the first and second characteristics each being one of landmarks, pose, body shape, weight, height, or internal body markers.

5. The method of claim 1 wherein estimating the first value comprises estimating with the features being learned convolution kernels from within first machine-learned model, the first machine-learned model comprising a first image-to-image network.

6. The method of claim 5 wherein estimating the second value comprises estimating with the second machine-learned model comprising a second image-to-image network, the features being from an encoder of the first image-to-image network and being used in an encoder of the second image-to-image network.

7. The method of claim 5 wherein estimating the second value comprises estimating with the second machine-learned model comprising a neural network, the features being at a bottleneck of the first image-to-image network and being used as inputs to the neural network.

8. The method of claim 1 wherein estimating the first value comprises estimating with the first machine-learned model comprising a first encoder-decoder trained to output upper body landmarks as the first characteristic, wherein estimating the second value comprises estimating with the second machine-learned model comprises a second encoder-decoder trained to output lower body landmarks as the second characteristic, further comprising estimating a third value for a third characteristic as a third task by a third machine-learned model, the third machine-learned model having been trained after the first machine-learned models using the features learned in training the first machine-learned models as unchanging in the training of the third machine- learned model.

9. The method of claim 1 wherein the second machine-learned model was trained using the features learned in training the first machine-learned model as constants such that the features do not change in the training for estimating by the first machine-learned model.

10. The method of claim 1 wherein the second characteristic comprises body shape, and wherein controlling comprises setting an iso-center using the body shape.

11. The method of claim 1 wherein the first characteristic comprises one or more landmarks, and wherein controlling comprises setting a scan range using the one or more landmarks.

12. The method of claim 1 wherein the second characteristic comprises a patient pose, and wherein controlling comprises re-orienting the patient on a bed or correcting a pose entered into a medical scanner.

13. The method of claim 1 wherein the second characteristic comprises body shape, and wherein controlling comprises performing a magnetic resonance scan with specific absorption rate settings based on the body shape.

14. The method of claim 1 wherein the second characteristic comprises a patient weight, height, or weight and height, and wherein controlling comprises configuring a scan based on the weight, height, or weight and height.

15. The method of claim 1 wherein the second characteristic estimated by the second machine-learned model comprises an internal body marker, and wherein controlling comprises controlling based on a simulated topogram or image from the internal body marker.

16. The method of claim 1 wherein controlling comprises configuring the medical scanner comprising a medical diagnostic imaging scanner or therapeutic scanner to scan based on the first and second values.

17. A medical scanner system using patient modeling, the medical scanner system comprising:
a depth camera configured to measure depths to a patient while the patient is on a patient bed in a medical scanner;
an image processor configured to determine two or more of patient pose, patient height, patient weight, patient shape, and patient landmark by application of a progressive multi-task machine-learned model; and
a controller configured to operate the medical scanner based on the patient pose, patient height, patient weight, and patient landmark.

18. The medical scanner system of claim 17 wherein the progressive multi-task machine-learned model comprises a neural network for each of the two or more of the patient pose, patient height, patient weight, and patient landmark, features learned from one of the neural networks being used in another one of the neural networks.

Assignees

• Siemens Healthcare Gmbh

Inventors

• Tamersoy Birgi
• Singh Vivek Kumar
• Ma Kai
• Chen Terrence
• Wimmer Andreas

Classifications

• G16H50/20 Primary

  for computer-aided diagnosis, e.g. based on medical expert systems · CPC title

• A61B5/1121

  Determining geometric values, e.g. centre of rotation or angular range of movement · CPC title

• A61B5/055 Primary

  involving electronic [EMR] or nuclear [NMR] magnetic resonance, e.g. magnetic resonance imaging · CPC title

• A61B90/39

  Markers, e.g. radio-opaque or breast lesions markers · CPC title

• G06N20/20

  Ensemble learning · CPC title