Integrated robotic system and method for autonomous vehicle maintenance

Patent metadata

Field	Value
Publication number	US-11020859-B2
Application number	US-201916240237-A
Country	US
Kind code	B2
Filing date	Jan 4, 2019
Priority date	May 1, 2015
Publication date	Jun 1, 2021
Grant date	Jun 1, 2021

Abstract

A robotic system includes a controller configured to obtain image data from one or more optical sensors and to determine one or more of a location and/or pose of a vehicle component based on the image data. The controller also is configured to determine a model of an external environment of the robotic system based on the image data and to determine tasks to be performed by components of the robotic system to perform maintenance on the vehicle component. The controller also is configured to assign the tasks to the components of the robotic system and to communicate control signals to the components of the robotic system to autonomously control the robotic system to perform the maintenance on the vehicle component.

First claim

What is claimed is:

1. A robotic system comprising:
a controller configured to obtain image data from one or more sensors, the controller also configured to determine a location and a pose of a vehicle component based on the image data and to determine a model of an external environment of the robotic system based on the image data, the controller configured to determine a mapping of a location of the robotic system in the model of the external environment, the controller also configured to determine tasks to be performed by components of the robotic system to perform maintenance on the vehicle component and to assign the tasks to the components of the robotic system, wherein the controller also is configured to communicate control signals to the components of the robotic system to autonomously control the robotic system to perform the maintenance on the vehicle component;
a propulsion system that moves the robotic system based on the control signals; and
a manipulator arm configured to actuate the vehicle component based on the control signals,
wherein the model of the external environment of the robotic system provides locations of objects external to the robotic system relative to the robotic system, grades off a surface on which the robotic system is traveling, and obstructions in the moving path of the robotic system relative to the location of the robotic system in the model of the external environment; and
wherein the model of the external environment of the robotic system is determined only for a designated volume around the manipulator arm.

2. The robotic system of claim 1 , wherein the controller is configured to obtain two dimensional (2D) and three dimensional (3D) image data from the one or more optical sensors as the image data.

3. The robotic system of claim 1 , wherein the controller is configured to determine the model of the external environment as a grid-based representation of the external environment based on the image data.

4. The robotic system of claim 1 , wherein the controller is configured to determine the tasks to be performed by the propulsion system and the manipulator arm.

5. The robotic system of claim 1 , wherein the controller is configured to determine the tasks to be performed by the robotic system based on the model of the external environment and the location and the pose of the vehicle component.

6. The robotic system of claim 1 , wherein the controller is configured to determine waypoints for the propulsion system of the robotic system to move the robotic system based on one or more of the tasks assigned to the propulsion system by the controller and the mapping of the location of the robotic system in the model of the external environment.

7. The robotic system of claim 1 , wherein the controller is configured to receive a feedback signal from one or more touch sensors representative of contact between a manipulator arm of the robotic system and an external body to the robotic system, the controller also configured to assign one or more of the tasks to the manipulator arm based also on the feedback signal.

8. The robotic system of claim 1 , wherein the controller is configured to determine a movement trajectory of one or more of the propulsion system of the robotic system or a manipulator arm of the robotic system based on the tasks that are assigned and the model of the external environment.

9. The robotic system of claim 1 , wherein the vehicle component is a brake component of an air brake for a vehicle.

10. A method comprising:
obtaining image data from one or more optical sensors;
determining a location and a pose of a vehicle component based on the image data;
determining a model of an external environment of a robotic system based on the image data;
determining a mapping of a location of the robotic system in the model of the external environment;
determining tasks to be performed by components of the robotic system to perform maintenance on the vehicle component;
assigning the tasks to the components of the robotic system;
communicating control signals to the components of the robotic system to autonomously control the robotic system to perform the maintenance on the vehicle component;
utilizing the control signals to move the robotic system; and
actuating the vehicle component based on the control signals.

11. The method of claim 10 , wherein the image data that is obtained includes two dimensional (2D) and three dimensional (3D) image data from the one or more optical sensors.

12. The method of claim 10 , wherein the model of the external environment is a grid-based representation of the external environment based on the image data.

13. The method of claim 10 , wherein the tasks are determined to be performed by a propulsion system that moves the robotic system and a manipulator arm configured to actuate the vehicle component.

14. The method of claim 10 , wherein the tasks are determined based on the model of the external environment and the location and the pose of the vehicle component.

15. The method of claim 10 , further comprising determining waypoints for a propulsion system of the robotic system to move the robotic system based on one or more of the tasks assigned to the propulsion system and on the mapping of the location of the robotic system in the model of the external environment.

16. The method of claim 10 , further comprising receiving a feedback signal from one or more touch sensors representative of contact between a manipulator arm of the robotic system and an external body to the robotic system, wherein one or more of the tasks are assigned to the manipulator arm based on the feedback signal.

17. The method of claim 10 , further comprising determining a movement trajectory of one or more of a propulsion system of the robotic system or a manipulator arm of the robotic system based on the tasks that are assigned and the model of the external environment.

18. A robotic system comprising:
one or more optical sensors configured to generate image data representative of an external environment;
a controller configured to obtain the image data and to determine one or more of a location and pose of a vehicle component based on the image data, the controller also configured to determine tasks to be performed by components of the robotic system to perform maintenance on the vehicle component and to assign the tasks to the components of the robotic system based on the image data and based on a model of the external environment, the controller configured to determine a location of the robotic system in the model of the external environment, wherein the controller also is configured to communicate control signals to the components of the robotic system to autonomously control the robotic system to perform the maintenance on the vehicle component;
a propulsion system that moves the robotic system based on the control signals; and
a manipulator arm configured to actuate the vehicle component based on the control signals;
wherein the controller is configured to determine waypoints for the propulsion system of the robotic system to move the robotic system based on one or more of the tasks assigned to the propulsion system by the controller and on the mapping of the location of the robotic system in the model of the external environment;
wherein 3D image data is examined using real-time simultaneous localization and mapping to determine the model of the external environment of the robotic system.

19. The robotic system of claim 18 , wherein the controller also is configured

Assignees

• Ge Global Sourcing Llc
• Transp Ip Holdings Llc

Inventors

• Patrick Romano
• Sen Shiraj
• Jain Arpit
• Tan Huan
• Gefen Yonatan
• Li Shuai
• Liu Shubao
• Sharma Pramod
• Kannan Balajee
• Holovashchenko Viktor
• Forman Douglas
• Lizzi John Michael
• Theurer Charles Burton
• Al Assad Omar
• Baloch Ghulam Ali
• Wheeler Frederick Wilson
• Tian Tai-Peng

Classifications

• G05B2219/40323

  Modeling robot environment for sensor based robot system · CPC title

• G05D1/00

  Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots (drive control systems specially adapted for autonomous road vehicles B60W60/00) · CPC title

• G05B2219/40298

  Manipulator on vehicle, wheels, mobile · CPC title

• B25J9/06

  characterised by multi-articulated arms · CPC title

• B25J19/021

  Optical sensing devices · CPC title