Supervised autonomous robotic system for complex surface inspection and processing

Patent metadata

Field	Value
Publication number	US-9796089-B2
Application number	US-201414774731-A
Country	US
Kind code	B2
Filing date	Mar 17, 2014
Priority date	Mar 15, 2013
Publication date	Oct 24, 2017
Grant date	Oct 24, 2017

Abstract

The invention disclosed herein describes a supervised autonomy system designed to precisely model, inspect and process the surfaces of complex three-dimensional objects. The current application context for this system is laser coating removal of aircraft, but this invention is suitable for use in a wide variety of applications that require close, precise positioning and maneuvering of an inspection or processing tool over the entire surface of a physical object. For example, this system, in addition to laser coating removal, could also apply new coatings, perform fine-grained or gross inspection tasks, deliver and/or use manufacturing process tools or instruments, and/or verify the results of other manufacturing processes such as but not limited to welding, riveting, or the placement of various surface markings or fixtures.

First claim

We claim:

1. A method of performing surface processing on a three-dimensional object using a robotic manipulator having and end effector, with a stationary or mobile base, in a work environment comprising the steps of:
obtaining a base model of said three-dimensional object;
generating a surface model from said base model by performing a surface scan of said three-dimensional object and augmenting said base model with range data and surface property data obtained from said scan;
planning a temporal sequence of states of said robotic manipulator to maximize processing of said surface of said three-dimensional object while avoiding a collision between said three-dimensional object and other objects in said work environment;
placing said robotic manipulator in each state in said sequence of states, and, for each state:
scanning said surface of said three-dimensional object before said surface is processed to control or modulate a processing tool connected to said robotic manipulator;
processing said surface with said processing tool and;
scanning said surface of said three-dimensional object after said surface has been processed and updating said surface model with new surface property data;

adjusting said temporal sequence of states for said robotic manipulator based on said updated surface model;
measuring the three-dimensional position and orientation of said end effector relative to a work area on said surface using one or more relative position sensors to minimize pose error due to relative motion between work room positioning sensors and said robotic manipulator; and

contintinuing processing of said surface and said updating of said surface model until said surface reaches a desired state.

2. The method of claim 1 further comprising the steps of:
refining the position of said surface model with respect to said work environment; and
refining the position of said robotic manipulator with respect to said work environment.

3. The method of claim 2 wherein said robotic manipulator is mounted on a mobile base which is capable of moving with respect to said work environment.

4. The method of claim 3 wherein said robotic manipulator is mounted on a rail base capable of moving said robotic manipulator along a rail.

5. The method of claim 1 , wherein said base model is obtained from a pre-existing 3-D model with subsurface properties.

6. The method of claim 1 , wherein said base model is generated by scanning said three dimensional object using an optical sensor capable of capturing the shape of said three dimensional object.

7. The method of claim 1 further comprising the step of augmenting said base model with processing policies regarding the processing of said surface.

8. The method of claim 7 further comprising the step of allowing user interaction with said surface model.

9. The method of claim 8 wherein said user interaction includes allowing the specification of masks which preclude portions of said surface from being processed.

10. The method of claim 8 wherein said user interaction includes allowing the specification of keep out areas or volumes which prevents the positioning of said robotic manipulator over said specified area or inside said specified volume.

11. The method of claim 8 wherein said user interaction includes allowing the marking of an area that needs further processing.

12. The method of claim 8 wherein said user interaction allows the overriding of said processing policies.

13. The method of claim 1 wherein said robotic manipulator includes a LIDAR device and further wherein said range data is generated by said LIDAR device.

14. The method of claim 1 wherein said robotic manipulator includes one or more sensors, and further wherein said surface property data is generated by said one or more sensors.

15. The method of claim 14 wherein said one or more sensors is selected from a group comprising multi-spectral cameras, color cameras, monochrome cameras and cameras with specific wavelength sensitivities.

16. The method of claim 14 wherein said one or more sensors is selected from a group comprising inspection sensors, including non-destructive inspection sensors, ultrasonic sensors, and eddy current based crack detection sensors.

17. The method of claim 1 wherein said temporal sequence of states of said robotic manipulator comprises a set of positions with respect to said work environment and poses.

18. The method of claim 1 wherein said step of planning a temporal sequence of states of said robotic manipulator precludes processing the same area of said surface more than once within a predetermined minimum period of time.

19. The method of claim 1 wherein said surface model consists of a set of two dimensional manifolds, each of said manifolds composed of a plurality of cells, each of said cells representing a physical unit of surface area and each of said cells having a set of properties associated therewith.

20. The method of claim 19 where said properties associated with each of said cells includes the type of material and coating properties.

21. The method of claim 1 wherein said processing tool is a laser capable of ablating paint from said surface.

22. The method of claim 1 further comprising the steps of:
querying the most recently observed state of said surface along a trajectory of said processing tool; and
planning a sequence of processing actions and parameters to be carried out by said processing tool based on said observed state.

23. The method of claim 1 further comprising the step of analyzing a processed surface to determine if further processing is necessary for any portion of said surface.

24. The method of claim 1 wherein said robotic manipulator includes an end effector, further comprising the step of measuring the 3-D position and orientation of said end effector using an optical positioning camera sighting said surface to measure relative motion between said end effector and said surface as said processing tool is processing said surface.

25. A system for performing surface processing on a three-dimensional object in a work environment comprising:
one or more robotic manipulators, each of said robotic manipulators including an end effector having a surface processing tool, a ranging sensor and one or more surface property sensors attached thereto, said one or more robotic manipulators being mounted on a stationary base or mobile base capable of moving on a surface;
a computer running software for controlling said one or more robotic manipulators, said software comprising:
a surface property analyzer, for processing surface properties sensor data before and after the processing point of said surface processing tool to classify the current state of said surface;
a surface model for storage of the geometry of said surface and said current state of said surface, wherein the software is adapted to generate the surface model by refining a three-dimensional base model by the surface property analyzer performing a surface scan of said three-dimensional object and augmenting said base model with range data and surface property data obtained from said scan;
a surface process planner for planning a sequence of processing actions to be carried out by said processing tool based on an observed state of said surface stored in said surface model, wherein the surface process planner is adapted to create a temporal sequence of states of said one or more r

Assignees

• Univ Carnegie Mellon

Inventors

• Lawrence Iii Stuart Edwin
• Baker Christopher L
• Baker Christopher Randolph
• Galati David G
• Haines Justin C
• Herman Herman
• Kelly Alonzo J
• Meyhofer Eric
• Stentz Anthony
• Valois Jean-Sebastien
• Strat Andrew

Classifications

• B25J9/1664

  characterised by motion, path, trajectory planning · CPC title

• G05B2219/37449

  Inspection path planner · CPC title

• G05B2219/45071

  Aircraft, airplane, ship cleaning manipulator, paint stripping · CPC title

• Y10S901/28

  Joint · CPC title

• Y10S901/44

  inspection · CPC title