Systems and methods for generating patient models based on ultrasound images

Patent metadata

Field	Value
Publication number	US-2024164758-A1
Application number	US-202217989251-A
Country	US
Kind code	A1
Filing date	Nov 17, 2022
Priority date	Nov 17, 2022
Publication date	May 23, 2024
Grant date	—

Abstract

Sensing device(s) may be installed in a medical environment to captures images of the medical environment, which may include an ultrasound probe and a patient. The images may be processed to determine, automatically, the position of the ultrasound probe relative to the patient's body. Based on the determined position, ultrasound image(s) taken by the ultrasound probe may be aligned with a 3D patient model and displayed with the 3D patient model, for example, to track the movements of the ultrasound probe and/or provide a visual representation of the anatomical structure(s) captured in the ultrasound image(s) against the 3D patient model. The ultrasound images may also be used to reconstruct a 3D ultrasound model of the anatomical structure(s).

First claim

What is claimed is:

1 . A system, comprising:
at least one sensing device configured to capture images of a medical environment, wherein the medical environment includes a patient and an ultrasound machine, and the ultrasound machine comprises an ultrasound probe; and one or more processors configured to:
obtain a three-dimensional (3D) human model of the patient, wherein the 3D human model indicates at least a pose and a shape of the patient's body;
receive a first ultrasound image of the patient captured using the ultrasound probe;
determine, based on the images of the medical environment captured by the at least one sensing device, a position of the ultrasound probe;
align the first ultrasound image with the 3D human model based on at least the position of the ultrasound probe; and
generate a visual representation that shows the alignment of the first ultrasound image and the 3D human model.

2 . The system of claim 1 , wherein the visual representation includes a 3D body contour of the patient, and wherein the one or more processors are further configured to fill a first inside portion of the 3D body contour with the first ultrasound image based on the alignment of the first ultrasound image and the 3D human model.

3 . The system of claim 2 , wherein the one or more processors are further configured to:
receive a second ultrasound image of the patient captured using the ultrasound probe; align the second ultrasound image with the 3D human model based on at least the position of the ultrasound probe; and add the second ultrasound image to the visual representation by filling a second inside portion of the 3D body contour with the second ultrasound image based on the alignment of the second ultrasound image and the 3D human model.

4 . The system of claim 3 , wherein the first and second ultrasound images of the patient are associated with an anatomical structure of the patient, and the one or more processors are further configured to reconstruct a 3D ultrasound model of the anatomical structure based on at least the first ultrasound image and the second ultrasound image.

5 . The system of claim 1 , wherein the 3D human model of the patient is obtained from another source or generated by the one or more processors based on the images captured by the at least one sensing device.

6 . The system of claim 1 , wherein the one or more processors are further configured to determine an orientation of the ultrasound probe, and align the first ultrasound image with the 3D human model further based on the determined orientation of the ultrasound probe.

7 . The system of claim 1 , wherein the one or more processors being configured to determine the position of the ultrasound probe comprises the one or more processors being configured to detect, in the images of the medical environment, a marker associated with the ultrasound probe and determine the position of the ultrasound probe based on the detected marker.

8 . The system of claim 1 , wherein the one or more processors being configured to determine the position of the ultrasound probe comprises the one or more processors being configured to detect, based on a machine learning model, visual features associated with the ultrasound probe in the images of the medical environment, and determine the position of the ultrasound probe based on the detected visual features.

9 . The system of claim 1 , wherein the one or more processors are further configured to:
receive a second ultrasound image of the patient captured using the ultrasound probe; determine, based on respective visual features of the first ultrasound image and the second ultrasound image detected by a machine learning model, that the first ultrasound image is substantially similar to the second ultrasound image; and provide an indication that the first ultrasound image and the second ultrasound image are duplicative of each other.

10 . The system of claim 1 , wherein the one or more processors are further configured to detect, based on a machine learning model, a medical abnormality in the first ultrasound image, and provide an indication of the detected medical abnormality.

11 . The system of claim 1 , wherein the at least one sensing device is configured to be installed on the ultrasound machine or from a ceiling of the medical environment.

12 . The system of claim 1 , wherein the one or more processors being configured to determine the position of the ultrasound probe comprises the one or more processors being configured to determine the position of the ultrasound probe relative to the patient's body.

13 . A method, comprising:
capturing images of a medical environment, wherein the medical environment includes a patient and an ultrasound machine, and wherein the ultrasound machine includes an ultrasound probe; obtaining a three-dimensional (3D) human model of the patient, wherein the 3D human model indicates at least a pose and a shape of the patient's body; receiving a first ultrasound image of the patient captured using the ultrasound probe; determining, based on the images of the medical environment, a position of the ultrasound probe; aligning the first ultrasound image with the 3D human model based on at least the position of the ultrasound probe; and generating a visual representation that shows the alignment of the first ultrasound image and the 3D human model.

14 . The method of claim 13 , wherein the visual representation includes a 3D body contour of the patient, and wherein the method further comprises filling a first inside portion of the 3D body contour with the first ultrasound image based on the alignment of the first ultrasound image and the 3D human model.

15 . The method of claim 14 , further comprising:
receiving a second ultrasound image of the patient captured using the ultrasound probe; aligning the second ultrasound image with the 3D human model based on at least the position of the ultrasound probe; and adding the second ultrasound image to the visual representation by filling a second inside portion of the 3D body contour with the second ultrasound image based on the alignment of the second ultrasound image and the 3D human model.

16 . The method of claim 15 , wherein the first and second ultrasound images of the patient are associated with an anatomical structure of the patient, and wherein the method further comprises reconstructing a 3D ultrasound model of the anatomical structure based on at least the first ultrasound image and the second ultrasound image.

17 . The method of claim 13 , further comprising determining an orientation of the ultrasound probe, wherein the first ultrasound image is aligned with the 3D human model further based on the determined orientation of the ultrasound probe.

18 . The method of claim 13 , wherein determining the position of the ultrasound probe comprises detecting, in the images of the medical environment, a marker associated with the ultrasound probe and determining the position of the ultrasound probe based on the detected marker.

19 . The method of claim 13 , determining the position of the ultrasound probe comprises detecting, using a machine learning model, visual features associated with the ultrasound probe in the images of the medical environment, and determining the position of the ultrasound probe based on the detected visual features.

20 . The method of claim 13 , further comprising:
receiving a second ultrasound image of the patient captured using the ultrasound probe; determining

Assignees

• Shanghai United Imaging Intelligence Co Ltd

Inventors

• Wu Ziyan
• Sun Shanhui
• Innanje Arun
• Planche Benjamin
• Sharma Abhishek
• Zheng Meng

Classifications

• A61B8/5261 Primary

  combining images from different diagnostic modalities, e.g. ultrasound and X-ray · CPC title

• A61B8/5223

  for extracting a diagnostic or physiological parameter from medical diagnostic data (for algorithms to analyse biomedical images G06T7/0012) · CPC title

• A61B8/4444 Primary

  related to the probe · CPC title

• A61B8/4254

  using sensors mounted on the probe · CPC title

• A61B8/483

  involving the acquisition of a 3D volume of data · CPC title