System and method providing situational awareness for autonomous asset inspection robot monitor

Patent metadata

Field	Value
Publication number	US-10682677-B2
Application number	US-201715591851-A
Country	US
Kind code	B2
Filing date	May 10, 2017
Priority date	May 10, 2017
Publication date	Jun 16, 2020
Grant date	Jun 16, 2020

Abstract

A three-dimensional model data store may contain a three-dimensional model of an industrial asset, including points of interest associated with the industrial asset. An inspection plan data store may contain an inspection plan for the industrial asset, including a path of movement for an autonomous inspection robot. An industrial asset inspection platform may receive sensor data from an autonomous inspection robot indicating characteristics of the industrial asset and determine a current location of the autonomous inspection robot along the path of movement in the inspection plan along with current context information. A forward simulation of movement for the autonomous inspection robot may be executed from the current location, through a pre-determined time window, to determine a difference between the path of movement in the inspection plan and the forward simulation of movement along with future context information.

First claim

The invention claimed is:

1. A system for inspecting an industrial asset, comprising:
a three-dimensional (3D) model data store comprising a 3D model of the industrial asset, the 3D model including a plurality of points of interest associated with the industrial asset;
an inspection plan data store comprising an inspection plan for inspecting the industrial asset, the inspection plan including a path of movement for execution by an autonomous inspection robot; and
an industrial asset inspection platform communicatively coupled to the 3D model data store and the inspection plan data store, wherein the industrial asset inspection platform includes at least one processor operatively coupled to at least one memory, wherein the at least one processor is configured to:
receive sensor data from the autonomous inspection robot indicative of one or more characteristics of the industrial asset;
determine a current location of the autonomous inspection robot along the path of movement of the inspection plan;
execute and update a forward simulation of movement for the autonomous inspection robot from the current location of the autonomous inspection robot to generate a predicted path of movement of the autonomous inspection robot that extends from the current location over a duration of a pre-determined time window; and
overlay, via an interactive display, the predicted path of movement onto a representation of the 3D model in real-time, wherein the interactive display is configured to receive a user input and to transmit the user input to the at least one processor to cause the at least one processor to adjust the path of movement of the autonomous inspection robot.

2. The system of claim 1 , wherein the at least one processor is further configured to determine current context information including at least one of a sensor projection onto the 3D model of the industrial asset, an estimated remaining battery life, an estimated time of arrival at a point of interest, a data link strength, a storage media operational status, a number of available satellites, and a strength of a Differential Global Positioning System (“DGPS”) base station link.

3. The system of claim 2 , wherein the at least one processor of the industrial asset inspection platform is further configured to automatically display an alarm via the interactive display based on the current context information.

4. The system of claim 1 , comprising the autonomous inspection robot, wherein the autonomous inspection robot includes at least one sensor configured to collect the sensor data.

5. The system of claim 4 , wherein the autonomous inspection robot is a drone.

6. The system of claim 4 , wherein the at least one sensor includes at least one of a camera, a video camera, an infra-red camera, a microphone, a chemical detector, a Light Detection and Ranging (“LIDAR”) sensor, and a radiation detector.

7. The system of claim 4 , wherein the inspection plan data store comprises inspection plans for a plurality of autonomous inspection robots configured to acquire additional sensor data indicative of the one or more characteristics of the asset.

8. The system of claim 1 , wherein the interactive display is configured to receive the user input and to transmit the user input to the at least one processor to cause the at least one processor to pause inspection operations of the industrial asset, resume inspection operations of the industrial asset, or abort inspection operations of the industrial asset.

9. The system of claim 1 , wherein the inspection plan includes instructions specifying a type of sensor, of the autonomous inspection robot, to be used for acquisition of data at a particular point of interest of the plurality of points of interest.

10. The system of claim 1 , wherein the inspection plan data store includes information indicative of an environment surrounding the industrial asset.

11. The system of claim 1 , wherein the inspection plan data store includes information indicative of prior inspections of the industrial asset.

12. The system of claim 1 , wherein the predicted path of movement is indicative of a first portion of the path of movement, wherein a second portion of the path of movement includes a past path of movement previously traveled by the autonomous inspection robot to arrive at the current location, and wherein a third portion of the path of movement includes a future path of movement to be traveled by the autonomous inspection robot subsequent to completion of the predicted path of movement.

13. The system of claim 1 , wherein the at least one processor is further configured to overlay indications of the plurality of points of interest onto the representation of the 3D model.

14. The system of claim 13 , wherein the at least one processor is further configured to display, via the interactive display, the sensor data indicating the one or more characteristics of the industrial asset.

15. The system of claim 1 , wherein the at least one processor is further configured to determine whether a sensor of the autonomous inspection robot is targeting one of the plurality of points of interest on the industrial asset by performing a sensor projection onto the 3D model of the industrial asset.

16. The system of claim 1 , wherein the interactive display is configured to receive the user input and to transmit the user input to the at least one processor to enable user-controlled piloting of the autonomous inspection robot via the interactive display.

17. The system of claim 1 , wherein the inspection plan includes instructions specifying an acquisition time during which the autonomous inspection robot is to collect data at a particular point of interest of the plurality of points of interest.

18. A method for inspecting an industrial asset, comprising:
receiving, at a processor of an industrial asset inspection platform, sensor data from a sensor of an autonomous inspection robot indicative of one or more characteristics of the industrial asset;
identifying, in an inspection plan data store, an inspection plan for inspecting the industrial asset, the inspection plan including a path of movement for the autonomous inspection robot;
determining a current location of the autonomous inspection robot along the path of movement of the inspection plan;
executing and updating a forward simulation of movement for the autonomous inspection robot from the current location of the autonomous inspection robot to generate a predicted path of movement of the autonomous inspection robot that extends from the current location over a duration of a pre-determined time window;
acquiring, from a three-dimensional (3D) model data store, a 3D model of the industrial asset, the 3D model including a plurality of points of interest associated with the industrial asset; and
overlaying, via an interactive display, the predicted path of movement of the autonomous inspection robot, from the current location of the onto a representation of the 3D model in real-time, wherein the interactive display is configured to receive a user input and to transmit the user input to the processor to cause the processor to adjust the path of movement of the autonomous inspection robot.

19. The method of claim 18 , further comprising determining current context information during inspection of the asset, wherein the current context information includes at least one of a sensor projection onto the three-dimensional model of the industrial asset, an estimated remaining battery life, an estimated time of arrival at a point of int

Assignees

• Gen Electric

Inventors

• Sen Shiraj
• Gray Steven
• Abate Nicholas
• Silva Filho Roberto
• Huang Ching-Ling
• Castillo-Effen Mauricio
• Baloch Ghulam Ali
• Venkataramana Raju
• Forman Douglas

Classifications

• B64U2201/10

  autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS] · CPC title

• B25J5/007

  mounted on wheels · CPC title

• B08B13/00 Primary

  Accessories or details of general applicability for machines or apparatus for cleaning · CPC title

• B64C39/024

  of the remote controlled vehicle type, i.e. RPV · CPC title