Automated and adaptive three-dimensional robotic site surveying

Patent metadata

Field	Value
Publication number	US-10607406-B2
Application number	US-201815879743-A
Country	US
Kind code	B2
Filing date	Jan 25, 2018
Priority date	Jan 25, 2018
Publication date	Mar 31, 2020
Grant date	Mar 31, 2020

Abstract

A method for generating a three-dimensional model of an asset includes receiving input parameters corresponding to constraints of a mission plan for operating an unmanned vehicle around an asset, generating the mission plan based on the input parameters including information of a representative asset type, wherein the mission plan includes waypoints identifying locations and orientations of one or more image sensors of the unmanned vehicle, generating a flight path for the unmanned vehicle connecting the waypoints that satisfy one or more predefined criteria, monitoring a vehicle state of the unmanned vehicle during execution of the flight path from one waypoint to the next waypoint, determining, at each waypoint, a local geometry of the asset sensed by the one or more image sensors, changing the mission plan on-the-fly based on the local geometry, and capturing images of the asset along waypoints of the changed mission plan.

First claim

What is claimed is:

1. A system for generating a three-dimensional model of an asset, comprising:
a memory storing instructions; and
a processor configured to execute the instructions, wherein the instructions, when executed by the processor, cause the processor to:
receive input parameters corresponding to constraints of a mission plan for operating an unmanned vehicle around the asset, wherein the constraints comprise an inner geofence that corresponds to a representative shape of the asset and an outer geofence that defines an exclusion zone for the unmanned vehicle;
generate the mission plan based on the input parameters, wherein the mission plan defines a plurality of waypoints indicative of locations and orientations of one or more image sensors of the unmanned vehicle with respect to the asset;
generate a flight path for the unmanned vehicle based on the mission plan that extends between the inner geofence and the outer geofence by connecting the waypoints;
determine, via the one or more image sensors, a local geometry of the asset at each of the waypoints during execution of the mission plan;
update the mission plan to an updated mission plan having updated waypoints based on a determination that the local geometry differs from the representative shape; and
capture images of the asset at the updated waypoints corresponding to the updated mission plan.

2. The system of claim 1 , wherein the processor is further configured to monitor a vehicle state of the unmanned vehicle during execution of the flight path.

3. The system of claim 1 , wherein the input parameters further comprise asset constraints, three-dimensional reconstruction constraints, mission constraints, or a combination thereof.

4. The system of claim 1 , wherein the one or more image sensors comprise a camera, a video camera, an infra-red camera, a microphone, a chemical detector, a Light Detection and Ranging (“LIDAR”) sensor, a radiation detector, or a combination thereof.

5. The system of claim 1 , wherein the representative shape is expressed in terms of primitive shapes and parameters that are derived without access to a three-dimensional reference model of the asset.

6. The system of claim 1 , wherein updating the mission plan comprises modifying the orientation of the one or more image sensors, adding additional waypoints to the flight path, modifying respective positions of the waypoints to adjust the flight path, or a combination thereof.

7. The system of claim 1 , wherein the constraints further comprise environmental constraints, vehicle constraints, task constraints, or a combination thereof.

8. A method for generating a three-dimensional model of an asset, comprising:
receiving input parameters corresponding to constraints of a mission plan for operating an unmanned vehicle around the asset, wherein the constraints comprise an inner geofence that corresponds to a representative shape of the asset and an outer geofence that defines an exclusion zone for the unmanned vehicle;
generating the mission plan based on the input parameters, wherein the mission plan defines a plurality of waypoints indicative of locations and orientations of one or more image sensors of the unmanned vehicle with respect to the asset;
generating a flight path for the unmanned vehicle based on the mission plan that extends between the inner geofence and the outer geofence by connecting the waypoints;
determining, via the one or more image sensors, a local geometry of the asset at each of the waypoints during execution of the mission plan;
updating the mission plan to an updated mission plan having updated waypoints based on a determination that the local geometry differs from the representative shape; and
capturing images of the asset at the updated waypoints corresponding to the updated mission plan.

9. The method of claim 8 , further comprising monitoring a vehicle state of the unmanned vehicle during execution of the flight path.

10. The method of claim 8 , wherein the input parameters further comprise asset constraints, three-dimensional reconstruction constraints, mission constraints, or a combination thereof.

11. The method of claim 8 , wherein the one or more image sensors comprise a camera, a video camera, an infra-red camera, a microphone, a chemical detector, a Light Detection and Ranging (“LIDAR”) sensor, a radiation detector, or a combination thereof.

12. The method of claim 8 , wherein the representative shape is expressed in terms of primitive shapes and parameters.

13. The method of claim 8 , wherein updating the mission plan comprises modifying the orientation of the one or more image sensors, adding additional waypoints to the flight path, modifying respective positions of the waypoints to adjust the flight path, or a combination thereof.

14. The method of claim 8 , wherein the constraints further comprise environmental constraints, vehicle constraints, task constraints, or a combination thereof.

15. A non-transitory computer readable medium having instructions stored thereon, wherein the instructions, when executed by a processor, cause the processor to:
receive input parameters corresponding to constraints of a mission plan for operating an unmanned vehicle around an asset, wherein the constraints comprise an inner geofence that corresponds to a representative shape of the asset and an outer geofence that defines an exclusion zone for the unmanned vehicle;
generate the mission plan based on the input parameters, wherein the mission plan defines a plurality of waypoints indicative of locations and orientations of one or more image sensors of the unmanned vehicle with respect to the asset;
generate a flight path for the unmanned vehicle based on the mission plan that extends between the inner geofence and the outer geofence by connecting the waypoints;
determine, via the one or more image sensors, a local geometry of the asset at each of the waypoints during execution of the mission plan;
update the mission plan to an updated mission plan having updated waypoints based on a determination that the local geometry differs from the representative shape; and
capture images of the asset at the updated waypoints corresponding to the updated mission plan.

16. The non-transitory computer readable medium of claim 15 , wherein the instructions, when executed by the processor, cause the processor to monitor a vehicle state of the unmanned vehicle during execution of the flight path.

17. The non-transitory computer readable medium of claim 15 , wherein the input parameters further comprise asset constraints, three-dimensional reconstruction constraints, mission constraints, or a combination thereof.

18. The non-transitory computer readable medium of claim 15 , wherein the constraints further comprise environmental constraints, vehicle constraints, task constraints, or a combination thereof.

Assignees

• Gen Electric

Inventors

• Sen Shiraj
• Gray steven robert
• Jain Arpit
• Tan Huan
• Forman Douglas
• Guzzo Judith Ann

Classifications

• G01B21/20

  for measuring contours or curvatures, e.g. determining profile · CPC title

• G01B17/06

  for measuring contours or curvatures · CPC title

• G06T2207/10036

  Multispectral image; Hyperspectral image · CPC title

• G01B11/24

  for measuring contours or curvatures · CPC title

• G06T17/05

  Geographic models · CPC title