Autonomously-controlled inspection platform with model-based active adaptive data collection

Patent metadata

Field	Value
Publication number	US-10739770-B2
Application number	US-201815872582-A
Country	US
Kind code	B2
Filing date	Jan 16, 2018
Priority date	Jan 16, 2018
Publication date	Aug 11, 2020
Grant date	Aug 11, 2020

Abstract

Modifying a motion plan for an autonomously-operated inspection platform (AIP) includes obtaining sensor data for an industrial asset area of interest, analyzing the obtained sensor data during execution of an initial motion plan to determine if modification of the initial motion plan is required. If modification is required then performing a pose estimation on a first group of potential targets and a second group of potential targets, optimizing the results of the pose estimation to determine a modification to the initial motion plan, performing reactive planning to the initial motion plan to include the modification, the reactive planning providing a modified motion plan that includes a series of waypoints defining a modified path, and autonomously controlling motion of the AIP along the modified path. The analysis, pose estimation, optimization, and reactive planning occurring during movement of the AIP along a motion plan. A system and computer-readable medium are disclosed.

First claim

We claim:

1. A method of in-motion modification of an initial motion plan for an autonomously-operated inspection platform (AIP), the method comprising:
receiving, at a processor of the AIP, the initial motion plan, wherein the initial motion plan includes a series of initial waypoints defining an initial path about an industrial asset;
autonomously controlling, via the processor, movement of the AIP along the initial path;
obtaining, via a sensor of the AIP, sensor data for one or more areas of interest of the industrial asset;
determining, based on the sensor data and during execution of the initial motion plan, whether to modify the initial motion plan to include additional waypoints along which the AIP is to inspect one or more additional areas of interest of the industrial asset;
upon determining to modify the initial motion plan:
performing a pose estimation on a first group of potential targets and a second group of potential targets on the industrial asset;
optimizing results of the pose estimation to determine a modification to the initial motion plan; and
performing reactive planning to incorporate the modification into the initial motion plan, wherein incorporating the modification into the initial motion plan comprises adding the additional waypoints to the series of initial waypoints to define a modified path having an updated series of waypoints that includes the series of initial waypoints and the additional waypoints, wherein the additional waypoints are interspersed with the series of initial waypoints such that the modified path minimizes a travel time of the AIP through the updated series of waypoints; and

autonomously controlling movement of the AIP along the modified path.

2. The method of claim 1 , wherein performing the pose estimation includes:
identifying at least one object of the industrial asset in at least one of the first group of potential targets and the second group of potential targets; and
identifying both a position and an orientation for the at least one object relative to a location of the AIP.

3. The method of claim 1 , wherein the modified motion plan includes a position and an orientation of an object identified by the pose estimation, and wherein the position and the orientation determine a relative position of the AIP that enables the AIP to obtain additional sensor data for the object.

4. The method of claim 1 , comprising extracting features from the sensor data collected while the AIP travels along at least one of the initial path and the modified path.

5. The method of claim 4 , further comprising:
performing feature matching between a cluster and the features to obtain the first group of potential targets; and
applying a classifier model to a machine learning model and the features to obtain the second group of potential targets.

6. The method of claim 5 , wherein the cluster represents a two-dimensional model of at least a portion of the industrial asset.

7. The method of claim 5 , wherein the machine learning model represents a support vector machine model of at least a portion of the industrial asset.

8. A non-transitory computer readable medium having instructions stored thereon, wherein the instructions, when executed by a processor, cause the processor to:
receive an initial motion plan for an autonomously-operated inspection platform (AIP), wherein the initial motion plan includes a series of initial waypoints defining an initial path about an industrial asset;
autonomously control movement of the AIP along the initial path;
obtain, via a sensor of the AIP, sensor data for one or more areas of interest of the industrial asset;
determine, based on the sensor data and during execution of the initial motion plan, whether to modify the initial motion plan to include additional waypoints along which the AIP is to travel; and
upon determining to modify the initial motion plan:
perform a pose estimation on a group of potential targets on the industrial asset to determine both a position and an orientation of the group of potential targets relative to the AIP;
determine, based on both the position and the orientation of the group of potential targets, a modification to the initial motion plan, wherein the modification includes the additional waypoints, and wherein the additional waypoints define poses of the AIP, relative to the industrial asset, that enable the AIP to acquire sensor data of the group of potential targets; and
perform reactive planning to incorporate the modification into the initial motion plan to generate a modified motion plan that includes an updated series of waypoints defining a modified path; and

autonomously control movement of the AIP along the modified path.

9. The non-transitory computer readable medium of claim 8 , wherein the instructions, when executed by the processor, cause the processor to extract features from the sensor data collected while the AIP travels along at least one of the initial path and the modified path.

10. The non-transitory computer readable medium of claim 9 , wherein the instructions, when executed by the processor, cause the processor to:
perform feature matching between a cluster and the features to obtain the group of potential targets; and
apply a classifier model to a machine learning model and the features to obtain a second group of potential targets.

11. The non-transitory computer readable medium of claim 10 , wherein the cluster represents a two-dimensional model of at least a portion of the industrial asset.

12. The non-transitory computer readable medium of claim 10 , wherein the machine learning model represents a support vector machine model of at least a portion of the industrial asset.

13. The non-transitory computer readable medium of claim 8 , wherein the poses of the AIP define six degrees of freedom of the AIP relative to the industrial asset at each of the additional waypoints, wherein the six degrees of freedom include x-plane coordinates of the AIP, y-plane coordinates of the AIP, z-plane coordinates of the AIP, roll angle of the AIP, pitch angle of the AIP, and yaw angle of the AIP.

14. A system for in-motion modification of an initial motion plan of an autonomously-operated inspection platform (AIP), the system comprising:
an AIP having a processor in communication with a memory unit, the memory unit including executable instructions stored thereon that, when executed by the processor, cause the processor to:
receive an initial motion plan for the AIP, wherein the initial motion plan includes a series of initial waypoints defining an initial path about an industrial asset;
autonomously control movement of the AIP along the initial path;
obtain, via a sensor of the AIP, sensor data for one or more areas of interest of the industrial asset;
determine, based on the sensor data and during execution of the initial motion plan, whether to modify the initial motion plan to include additional waypoints along which the AIP is to inspect one or more additional areas of interest of the industrial asset;
upon determining to modify the initial motion plan:
perform a pose estimation on a first group of potential targets and a second group of potential targets on the industrial asset;
optimize results of the pose estimation to determine a modification to the initial motion plan, wherein the modification includes the additional waypoints, and wherein the additional waypoints define poses of the AIP, with respect to the industrial asset, that maximize inspection coverage of the one or more additional areas of interest by the AIP; and
perform reactive planning to incorporate the modificat

Assignees

• Gen Electric

Inventors

• Tan Huan
• Dasilva Ana
• Gros Eric
• Patrick Romano
• Theurer Charles
• Castillo-Effen Mauricio
• Lizzi John

Classifications

• B64U2101/30

  for imaging, photography or videography · CPC title

• B64U2201/10

  autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS] · CPC title

• B64U2101/26

  for manufacturing, inspections or repairs · CPC title

• G05D1/0094 Primary

  involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target · CPC title

• G05D1/0202 Primary

  specially adapted to aircraft · CPC title