System and method for tracking a deformation

Patent metadata

Field	Value
Publication number	US-11500086-B2
Application number	US-202017034139-A
Country	US
Kind code	B2
Filing date	Sep 28, 2020
Priority date	Sep 28, 2020
Publication date	Nov 15, 2022
Grant date	Nov 15, 2022

Abstract

An imaging system to reconstruct a reflectivity image of a scene including an object moving with the scene. A tracking system to track a deforming object to estimate an object deformation for each time step. Sensors acquire snapshots of the scene, each acquired snapshot of the object includes measurements in the object deformation for that time step, to produce a set of object measurements with deformed shapes over the time steps. Compute a correction to estimates of object deformation for each time step, with matching measurements of the corrected object deformation for each time step to measurements in the acquired snapshot of object for that time step. Select a corrected deformation over other corrected deformations for each time step, according to a distance between the corrected deformation and the estimate of the deformation, to obtain a final estimate of the deformation of the deformable object moving in the scene.

First claim

What is claimed:

1. An imaging system comprising:
a tracking system to track a deforming object within a scene over multiple time steps for a period of time to produce an initial estimate of a deformation of the object for each time step;
a measurement sensor captures measurement data by capturing snapshots of the object deforming in the scene over the multiple time steps for the time period; and
a processor that calculates, for the measurement data, deformation information of the deforming object, based on using each acquired snapshot of the object having measurements of the object in a deformation for that time step, to produce a set of measurements of the object with deformed shapes over the multiple time steps, and that for each time step of the multiple time steps, the processor sequentially calculates deformation information of object, by computing a correction to the estimates of the deformation of the object,
wherein the correction includes matching measurements of the corrected deformation of the object for each time step to measurements in the acquired snapshot of the object for that time step, and for each time step, select a corrected deformation over other corrected deformations for that time step, according to a distance between the corrected deformation and the initial estimate of the deformation, to obtain a final estimate of the deformation of the deformable object moving in the scene and a final image of the object moving within the scene.

2. The imaging system according to claim 1 , wherein the measurement sensor captures measurements of the object deforming in the scene over the multiple time steps for the time period, by continuously capturing snapshots of the object for the multiple steps for the period of time, and sequentially transmits the measurement data to the processor, where at each time step, the object exhibits a different deformation for the multiple time steps.

3. The imaging system according to claim 1 , wherein the tracking system tracks the deformable object during the same time period or a different time period, as that of the measurement sensor capturing snapshots of the object deforming.

4. The imaging system according to claim 1 wherein the deformation is wholly or partly caused by the object moving in the scene, or wherein the deformation is wholly or partly caused by the measurement sensor moving while capturing the scene.

5. The imaging system of claim 1 , wherein the system is a coherent imaging system, such as a radar imaging system, a magnetic resonance imaging system or an ultrasound imaging system.

6. The imaging system of claim 1 , wherein the tracking system includes at least one tracking sensor that is one or combination of an optical camera, a depth camera and an infrared camera, and wherein the measurement sensor is at least one electromagnetic sensor that includes one or combination of a mmWave radar, a Thz imaging sensor, and a backscatter X-ray sensor.

7. The imaging system of claim 1 , wherein the correction to the estimates of the deformation of the object for each time step is computed using an optimization that minimizes a cost function that includes an amount of a distance of how far the estimated deformation moves elements of the object, and a level of a measurement of how the deformed object matches to the measurements of the tracking system.

8. The imaging system of claim 7 , wherein the matching the measurements of the corrected deformation of the object for each time step to measurements in the acquired snapshot of the object for that time step is based on using a cost function that penalizes an amount of a distance between measurements of the corrected deformations of the object and measurements in the acquired snapshot of the object for that time step.

9. The imaging system of claim 7 , wherein the estimating of the corrected deformation over other corrected deformations for that time step, is according to the distance between the corrected deformation and the initial estimate of the deformation, and based on using a cost function that penalizes more the corrections to the deformations, in which elements of the object move an amount of a distance farther, when compared to their deformed location.

10. The imaging system of claim 1 , further comprising:
an optimal transport problem, which includes a cost that penalizes deformations according to an amount of a distance of how far these deformations move elements of the object image from their position, and a cost that penalizes deformations according to a level of a matching score of how well the measurements of the corrected deformations of the object match to the measurements of the tracking system.

11. The imaging system of claim 1 , wherein the object deforming in the scene is one of, a mammal including a human, an amphibian, a bird, a fish, an invertebrate or a reptile, wherein the object deforming in the scene is an organ inside a body of the human, an organ inside of the amphibian, an organ inside of the bird, an organ inside of the fish, an organ inside of the invertebrate or an organ inside of the reptile.

12. The imaging system of claim 1 , wherein the final estimate of the deformation of the deformable object, the final image of the object, or both, are labeled as an object report, and outputted to, and received by, a communication network associated with an entity such as an operator of the system, the operator generates at least one action command that is sent to, and received by a controller associated with the system which implements the generated at least one action command, resulting in changing a property of the object based upon the object report.

13. The imaging system of claim 12 , wherein the property of the object includes one or a combination of, a defect in the object, a medical condition of the object, a presence of a weapon on the object or a presence of an undesirable artifact on the object.

14. The imaging system of claim 12 , wherein the at least one action command includes one or a combination of, a level of an object defect inspection from a set of different levels of object defect inspections, a level of an object medical testing from a set of different levels of object medical testing, a level of an object security and safety inspection from a set of different levels of object security and safety inspections.

15. An image processing method, comprising:
tracking a deforming object within the scene over multiple time steps for a period of time via a tracking system to estimate a deformation of the object for each time step;
acquiring measurement data by continuously capturing snapshots of the object deforming in the scene over the multiple time steps for the period of time;
calculating, for the measurement data, deformation information of the deforming object, such that each acquired snapshot of the object includes measurements of the object in a deformation for that time step, to produce a set of measurements of the object with deformed shapes over the multiple time steps;
calculating deformation information of the object, by computing a correction to the estimates of the deformation of the object for each time step for the multiple time steps, such that the correction includes
matching measurements of the corrected deformation of the object for each time step to measurements in the acquired snapshot of the object for that time step, and for each time step, select a corrected deformation over other corrected deformations for that time step, according to a distance between the corrected deformation and the initial estimate of the deformation, to obtain a final estimate of the def

Assignees

• Mitsubishi Electric Res Laboratories Inc

Inventors

• Aeron Shuchin
• Ma Yanting
• Boufounos Petros
• Mansour Hassan

Classifications

• G01S15/66

  Sonar tracking systems · CPC title

• G06T2207/10088

  Magnetic resonance imaging [MRI] · CPC title

• G01S13/862

  Combination of radar systems with sonar systems · CPC title

• G06T2207/10044

  Radar image · CPC title

• G01S13/9021

  SAR image post-processing techniques · CPC title