Systems and methods for fetus monitoring

Patent metadata

Field	Value
Publication number	US-12178641-B2
Application number	US-202217814223-A
Country	US
Kind code	B2
Filing date	Jul 21, 2022
Priority date	Jul 21, 2022
Publication date	Dec 31, 2024
Grant date	Dec 31, 2024

Abstract

The present disclosure provides a system and method for fetus monitoring. The method may include obtaining ultrasound data relating to a fetus collected by an ultrasound imaging device; generating a 4D image of the fetus based on the ultrasound data; directing a display component of a virtual reality (VR) device to display the 4D image to an operator; detecting motion of the fetus based on the ultrasound data; and directing a haptic component of the VR device to provide haptic feedback with respect to the motion to the operator.

First claim

What is claimed is:

1. A system, comprising:
at least one storage device storing a set of instructions for fetus monitoring; and
at least one processor configured to communicate with the at least one storage device, wherein when executing the set of instructions, the at least one processor is configured to direct the system to perform operations including:
obtaining ultrasound data relating to a fetus collected by an ultrasound imaging device;
generating at least one 4D image of the fetus based on the ultrasound data;
directing a display component of a virtual reality (VR) device to display the at least one 4D image to an operator;
detecting motion of at least part of the fetus based on the ultrasound data; and
directing a haptic component of the VR device to provide a feedback force with respect to the motion to the operator, wherein the at least one 4D image includes a first 3D fetus image and a second 3D fetus image captured prior to the first 3D fetus image, and the detecting the motion of the at least part of the fetus based on the ultrasound data includes:
determining a vertex correspondence between a plurality of first vertexes of a first mesh surface and a plurality of second vertexes of a second mesh surface, the first mesh surface representing the first 3D fetus image, and the second mesh surface representing the second 3D fetus image; and
for each of the plurality of first vertexes, determining motion information from its corresponding second vertex to the first vertex based on the vertex correspondence, wherein the at least one processor is configured to direct the system to determine a magnitude of the feedback force by:
determining a target area of the at least one 4D image;
determining, among the plurality of first vertexes, a plurality of target first vertexes in a portion of the first mesh surface that corresponds to the target area;
for each of the plurality of target first vertexes, determining a magnitude of a vertex force based on motion information of the target first vertex, the motion information of the target first vertex relating to the motion of the at least part of the fetus; and
determining the magnitude of the feedback force based on the magnitudes of the vertex forces of the plurality of target first vertexes.

2. The system of claim 1 , wherein the generating at least one 4D image of the fetus based on the ultrasound data includes:
generating multiple initial 3D images based on the ultrasound data relating to the fetus;
generating multiple 3D fetus images by segmenting a portion representing the fetus from each initial 3D image; and
generating the at least one 4D image based on the multiple 3D fetus images.

3. The system of claim 2 , wherein the generating the at least one 4D image based on the multiple 3D fetus images includes:
extracting a mesh surface from each 3D fetus image; and
rendering at least one 4D mesh surface including the multiple mesh surfaces to generate the at least one 4D image.

4. The system of claim 1 , wherein the determining
the vertex correspondence between the plurality of first vertexes of
the first mesh surface and the plurality of second vertexes of the second mesh surface includes:
determining a motion field between the first 3D fetus image and the second 3D fetus image; and
determining the vertex correspondence based on the motion field.

5. The system of claim 4 , wherein the motion field is determined based on an optical flow-based technique or a motion field determination model.

6. The system of claim 1 , wherein the determining
the vertex correspondence between the plurality of first vertexes of
the first mesh surface and the plurality of second vertexes of the second mesh surface includes:
generating a first point cloud representing the first 3D fetus image and a second point cloud representing the second 3D fetus image, wherein the first point cloud includes a plurality of first points, and the second point cloud includes a plurality of second points; and
determining the vertex correspondence by registering the first point cloud to the second point cloud.

7. The system of claim 1 , wherein the at least one processor is further configured to direct the system to perform operations including:
directing a speaker to play a sound relating to the fetus.

8. The system of claim 1 , wherein the ultrasound imaging device is a 4D ultrasound imaging device.

9. The system of claim 1 , wherein the motion information of the target first vertex includes a displacement of the target first vertex, and the determining the magnitude of the vertex force based on the motion information of the target first vertex includes:
determining the magnitude of the vertex force based on the displacement of the target first vertex.

10. The system of claim 1 , wherein, for each of the plurality of target first vertexes, the determining a magnitude of the vertex force based on the motion information of the target first vertex includes:
determining the magnitude of the vertex force based on Newton's second law according to Equation as follows:
F=ma,

wherein F denotes the vertex force, m denotes a vertex mass of the target first vertex, and a denotes an acceleration of the target first vertex.

11. The system of claim 10 , wherein the vertex mass is determined based on a size or a volume of the fetus.

12. The system of claim 1 , wherein the display component of the VR device includes a first display component corresponding to a left eye of the operator and a second display component corresponding to a right eye of the operator, and the at least one processor is configured to direct the system to perform operations including:
rendering a first image corresponding to a first eye view and a second image corresponding to a second eye view based on the at least one 4D image; and
directing the first display component to display the first image and the second display component to display the second image to the operator.

13. A method for fetus monitoring, implemented on a computing device having at least one processor and at least one storage device, the method comprising:
obtaining ultrasound data relating to a fetus collected by an ultrasound imaging device;
generating at least one 4D image of the fetus based on the ultrasound data;
directing a display component of a virtual reality (VR) device to display the at least one 4D image to an operator;
detecting motion of at least part of the fetus based on the ultrasound data; and
directing a haptic component of the VR device to provide a feedback force with respect to the motion to the operator, wherein the at least one 4D image includes a first 3D fetus image and a second 3D fetus image captured prior to the first 3D fetus image, and the detecting the motion of the at least part of the fetus based on the ultrasound data includes:
determining a vertex correspondence between a plurality of first vertexes of a first mesh surface and a plurality of second vertexes of a second mesh surface, the first mesh surface representing the first 3D fetus image, and the second mesh surface representing the second 3D fetus image; and
for each of the plurality of first vertexes, determining motion information from its corresponding second vertex to the first vertex based on the vertex correspondence, wherein the at least one processor is configured to direct the system to determine a magnitude of the feedback force by:
determining a target area of the at least one 4D image;
determining, among the plurality of first vertexes, a plurality of target first vertexes in a portion of the first mesh surface that corresponds to the target area;

Assignees

• Shanghai United Imaging Intelligence Co Ltd

Inventors

• Sun Shanhui
• Wu Ziyan
• Chen Xiao
• Chen Zhang
• Liu Yikang
• Innanje Arun
• Chen Terrence

Classifications

• A61B8/483

  involving the acquisition of a 3D volume of data · CPC title

• A61B8/466

  adapted to display 3D data · CPC title

• A61B8/5207

  involving processing of raw data to produce diagnostic data, e.g. for generating an image · CPC title

• G06F3/011

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• A61B8/5246

  combining images from the same or different imaging techniques, e.g. color Doppler and B-mode · CPC title