Robotic tracking navigation with data fusion

Patent metadata

Field	Value
Publication number	US-11429111-B2
Application number	US-201816484389-A
Country	US
Kind code	B2
Filing date	Feb 6, 2018
Priority date	Feb 8, 2017
Publication date	Aug 30, 2022
Grant date	Aug 30, 2022

Abstract

Disclosed are systems and methods of sensor fusion for exemplary use with robotic navigation control. Systems and methods include providing local estimates of a target location from a plurality of expert modules that process sensor data. The local estimates are weighted based upon a Mahalanobis distance from an expected estimated value and based upon a Euclidean distance between the local estimates. The local estimates are fused in a Bayesian fusion center based upon the weight given to each of the local estimates.

First claim

The invention claimed is:

1. A navigable robot comprising:
at least one sensor configured to produce tracking data of a target object, wherein the at least one sensor comprises a camera and the tracking data comprises at least image data;
a plurality of detector modules, each detector module in the plurality comprising at least one processing algorithm to independently provide a local estimate of a location of the target object when applied to the tracking data;
a controller connected to the at least one sensor and the plurality of detector modules, the controller receives tracking data acquired by the at least one sensor, the controller calculates a Mahalanobis distance (MD) between an estimated value and the local estimate for each of the local estimates, wherein the MD is a first weighting factor for each of the local estimates, calculates a Euclidean distance between each of the local estimates, and calculates a second weighting factor for each of the local estimates based upon the calculated Euclidean distance, and the controller combines the local estimates in an adaptive Bayesian fusion based upon each of the local estimates weighted by the respective first weighting factor and the second weighting factor for each of the local estimates to produce a target location; and
at least one navigation motor and the controller produces at least one control command based upon the target location and provides the at least one control command to the navigation motor, and in response to the at least one control command, the navigation motor operates to move the navigable robot relative to the target location;
wherein the controller uses an autoencoder trained with acquired data to learn a model of the detector modules of the plurality of detector modules under normal operation and analyzes local estimates from each detector module of the plurality of detector modules to determine an event of detector failure and decreases a weighting of a local estimate from a detector module based upon the determined event of detector failure.

2. The navigable robot of claim 1 , wherein the controller calculates the MD for each of the local estimates with at least one first Kalman filter and further combines the local estimates weighted by the respective first weighting factor and the second weighting factor for each of the local estimates with at least one second Kalman filter.

3. The navigable robot of claim 1 , comprising a plurality of sensors including the at least one sensor and wherein the plurality of sensors comprise at least one non-camera sensor.

4. The navigable robot of claim 3 , wherein the non-camera sensor comprises a range sensor, depth sensor, radar, or GPS.

5. The navigable robot of any of claim 1 , wherein the navigable robot is a pan-tilt robot, an unmanned aerial vehicle (UAV), or a mobile ground based robot.

6. The navigable robot of claim 1 , wherein the controller applies a Jaccard Index to the outputs of each detector module of the plurality of detector modules to segment the data into datasets for analysis with the autoencoder.

7. A method of robotic navigation of a navigable robot, wherein the navigable robot comprises:
at least one sensor configured to produce tracking data of a target object;
a plurality of detector modules, each detector module in the plurality comprising at least one processing algorithm to independently provide a local estimate of a location of the target object when applied to the tracking data;
a controller connected to the at least one sensor and the plurality of detector modules, the controller receives tracking data acquired by the at least one sensor and applies the algorithms from the plurality of detector modules to the tracking data to produce a plurality of separate estimates of the location of the target object in the tracking data, the controller weights the separate estimates and combines the local estimates in an adaptive Bayesian fusion based upon the weighted local estimates to produce a target location; and
at least one navigation motor and the controller produces at least one control command based upon the target location and provides the at least one control command to the navigation motor to move the navigable robot relative to the target location; and
the method of robotic navigation comprises:
obtaining tracking data from the at least one sensor;
separately estimating with the plurality of detector modules local estimates of a target location from the tracking data from the at least one sensor;
calculating a Mahalanobis distance between an estimated value and the local estimate for each of the local estimates, wherein the Mahalanobis distance is a first weighting factor for each of the local estimates;
applying a sigmoid function to the MD for the output of each detector module before producing the first weighting factor for each of the local estimates
calculating a Euclidean distance between each of the local estimates, and calculating a second weighting factor for each of the local estimates based upon the calculated Euclidean distance; and
combining the local estimates in an adaptive Bayesian fusion based upon each of the local estimates weighted by the respective first weighting factor and the second weighting factor for each of the local estimates to produce the target location.

8. The method of claim 7 , further comprising providing an instruction to the navigable robot to move relative to the target location.

9. The method of claim 8 wherein the tracking data is obtained from a plurality of sensors comprising at least one camera, the plurality of sensors comprising the at least one sensor.

10. The device of claim 9 , wherein the plurality of sensors comprises a plurality of cameras.

11. The method of claim 9 , wherein the plurality of detector modules comprises a plurality of visual tracking algorithms applied to image data from the at least one camera.

12. The method of claim 11 , wherein the plurality of image recognition algorithms comprise one or more of TLD, CMT, STRUCK, and GOTURN.

13. The method of claim 12 , wherein the local estimates of the target location are each estimated using a Kalman Filter.

14. The method of claim 7 , further comprising generating robotic control commands from the target location.

Assignees

• Univ Marquette

Inventors

• Medeiros Henry Ponti
• Guevara Andres Echeverri
• Bernal Juan Tapiero
• O'Rourke James

Classifications

• G06T7/277

  involving stochastic approaches, e.g. using Kalman filters · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title

• G05D1/00

  Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots (drive control systems specially adapted for autonomous road vehicles B60W60/00) · CPC title

• G01C21/005

  with correlation of navigation data from several sources, e.g. map or contour matching (G01C21/30 takes precedence) · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title