Artificial intelligence-based hierarchical planning for manned/unmanned platforms

Patent metadata

Field	Value
Publication number	US-11960994-B2
Application number	US-202117151506-A
Country	US
Kind code	B2
Filing date	Jan 18, 2021
Priority date	Jan 16, 2020
Publication date	Apr 16, 2024
Grant date	Apr 16, 2024

Abstract

A method, apparatus and system for artificial intelligence-based HDRL planning and control for coordinating a team of platforms includes implementing a global planning layer for determining a collective goal and determining, by applying at least one machine learning process, at least one respective platform goal to be achieved by at least one platform, implementing a platform planning layer for determining, by applying at least one machine learning process, at least one respective action to be performed by the at least one of the platforms to achieve the respective platform goal, and implementing a platform control layer for determining at least one respective function to be performed by the at least one of the platforms. In the method, apparatus and system despite the fact that information is shared between at least two of the layers, the global planning layer, the platform planning layer, and the platform control layer are trained separately.

First claim

The invention claimed is:

1. An artificial intelligence-based method for coordinating a team of platforms, the method comprising:
implementing a global planning layer for;
determining a collective goal for the team of the platforms; and
determining, by applying at least one machine learning process, at least one respective platform goal to be achieved by at least one of the platforms to achieve the determined collective goal;

implementing a platform planning layer for determining, by applying at least one machine learning process, at least one respective action to be performed by the at least one of the platforms to achieve the respective platform goal; and
implementing a platform control layer for determining at least one respective function to be performed by the at least one of the platforms to perform the at least one respective action;
wherein information is shared between at least two of the global planning layer, the platform planning layer, and the platform control layer to assist in determining at least one of the collective goal for the team of the platforms, the at least one respective platform goal, the at least one respective action, and the at least one respective function; and
wherein the global planning layer, the platform planning layer, and the platform control layer are trained separately.

2. The method of claim 1 , wherein the at least one machine learning process applied by at least one of the global planning layer and the platform planning layer comprises a policy planning process, which rewards a platform for performing an action which advances an achievement of at least one of the determined collective goal for the team of the platforms, the determined at least one respective platform goal, and the determined at least one respective action.

3. The method of claim 1 , further comprising:
communicating information determined as a result of at least one function performed by at least one of the platforms to at least one of the global planning layer, the platform planning layer, and the platform control layer to assist in determining at least one of the collective goal for the team of the platforms, the at least one respective platform goal, the at least one respective action, and the at least one respective function.

4. The method of claim 1 , wherein at least one of the platforms comprises a sensor device for capturing sensor data of at least one environment of the team of platforms.

5. The method of claim 4 , further comprising;
communicating the captured sensor data to the platform planning layer for determining at least one of a physical layout and a semantic layout of the at least one environment.

6. The method of claim 5 , further comprising:
at the platform planning layer, considering at least one of the physical layout and the semantic layout of the at least one environment when determining the at least one respective action; and
communicating information regarding the at least one of the physical layout and the semantic layout of the at least one environment to the global planning layer to be considered when determining at least one of the collective goal for the team of the platforms and the at least one respective platform goal.

7. The method of claim 1 , wherein operations of the global planning layer, the platform planning layer, and the platform control layer are performed in parallel.

8. The method of claim 1 , further comprising applying a machine learning process to predict movement and actions of at least one of independent and adversarial platforms in an environment of the team of platforms.

9. The method of claim 1 , wherein the at least one machine learning process applied by at least one of the global planning layer and the platform planning layer comprises at least a graph network, which models non-local connections in data for assisting in determining at least one of the determined collective goal for the team of the platforms, the determined at least one respective platform goal, and the determined at least one respective action.

10. The method of claim 1 , further comprising:
considering user inputs when determining at least one of the collective goal, the determined respective platform goals, the respective platform actions, and the respective platform functions.

11. The method of claim 1 , wherein a global planning layer, a platform planning layer and a platform control layer are implemented in at least one of 1) a platform of the team of platforms, 2) a central computing device, or 3) both, in order to create a decentralized system, centralized system or combination system, respectively, for coordinating the team of platforms.

12. The method of claim 1 , further comprising implementing the platform control layer for controlling the at least one of the platforms to perform the at least one respective function.

13. A system for coordinating a team of platforms, comprising:
a global planning module implementing a global planning layer for;
determining a collective goal for the team of the platforms; and
determining, by applying at least one machine learning process, at least one respective platform goal to be achieved by at least one of the platforms to achieve the determined collective goal;

a platform planning module implementing a platform planning layer for determining, by applying at least one machine learning process, at least one respective action to be performed by the at least one of the platforms to achieve the respective platform goal; and
a platform control module implementing a platform control layer for determining at least one respective function to be performed by the at least one of the platforms to perform the at least one respective action;
wherein information is shared between at least two of the global planning layer, the platform planning layer, and the platform control layer to assist in determining at least one of the collective goal for the team of the platforms, the at least one respective platform goal, the at least one respective action, and the at least one respective function; and
wherein the global planning layer, the platform planning layer, and the platform control layer are trained separately.

14. The system of claim 13 , further comprising:
a storage device for storing information determined by at least one of a platform of the team of platforms, the global planning module, the platform planning module and the platform control module to be used at any time to assist in determining at least one of the determined collective goal for the team of the platforms, the determined at least one respective platform goal, the determined at least one respective action, and the determined at least one respective function.

15. The system of claim 13 , wherein at least one of the platforms comprises a sensing device for collecting information of at least one environment of the team of platforms.

16. The apparats of claim 15 , wherein the platform planning module is configured to:
receive collected information and determine at least one of a physical layout and a semantic layout of the at least one environment;
consider the determined at least one of the physical layout and the semantic layout of the at least one environment when determining the at least one respective action; and
communicate information regarding the at least one of the physical layout and the semantic layout of the at least one environment to the global planning module to be considered by the global planning module when determining at least one of the collective goal for the team of platforms and the at least one respective platform goal.

Assignees

• Stanford Res Inst Int

Inventors

• Chiu Han-Pang
• Brookshire Jonathan D
• Seymour Zachary
• Mithun Niluthpol C
• Samarasekera Supun
• Kumar Rakesh
• Wang Qiao

Classifications

• G06N3/08 Primary

  Learning methods · CPC title

• G06N3/084

  Backpropagation, e.g. using gradient descent · CPC title

• G06N3/096

  Transfer learning · CPC title

• G06N3/098

  Distributed learning, e.g. federated learning · CPC title

• G06N20/00

  Machine learning · CPC title