Behavior management of multiple vehicles

Patent metadata

Field	Value
Publication number	US-12431027-B2
Application number	US-202318315229-A
Country	US
Kind code	B2
Filing date	May 10, 2023
Priority date	May 10, 2023
Publication date	Sep 30, 2025
Grant date	Sep 30, 2025

Abstract

A vehicle management system comprising a computer system and an agent. The agent comprises a machine learning model and a rule system. The machine learning model system is trained to receive observations for the vehicle system and select a behavior for the vehicle system in response to receiving the observations. The rule system is configured to select a set of actions to execute the behavior for the vehicle system in response to a selection of the behavior by the machine learning model system.

First claim

What is claimed is:

1. An aircraft management system comprising:
a computer system; and
an agent comprising:
a machine learning model system configured to:
receive observations for an aircraft system; and
select a maneuver behavior for an aircraft system in response to receiving the observation; and
a rule system configured to:
select a set of maneuver actions to execute the maneuver behavior for the aircraft system in response to a selection of the maneuver behavior the by machine learning model system.

2. The aircraft management system of claim 1 , wherein the rule system is further configured to:
select a set of non-maneuver actions in response to receiving the observations.

3. The aircraft management system of claim 2 , wherein the set of non-maneuver actions is selected from at least one of a weapon guidance, a weapon select, and a weapon firing.

4. The aircraft management system of claim 1 further comprising:
a controller for the aircraft system, wherein the agent sends the maneuver actions to controller and wherein the controller creates instructions from the maneuver actions and sends the instructions to the aircraft system.

5. The aircraft management system of claim 4 , wherein the instructions control a behavior of an aircraft in the aircraft system individually.

6. The aircraft management system of claim 4 , wherein the instructions control a behavior of aircraft in the aircraft system in a coordinated manner.

7. The aircraft management system of claim 1 , wherein the machine learning model system selects the maneuver behavior using an action mask that avoids selection of an invalid maneuver behavior.

8. The aircraft management system of claim 1 further comprising:
a sensor system configured to generate sensor data for the observations.

9. The aircraft management system of claim 1 , wherein the machine learning model system is trained to select a behavior selected from a group comprising the maneuver behavior and a non-maneuver action in response to receiving the observations for the aircraft system.

10. The aircraft management system of claim 1 , wherein the maneuver behavior is selected from a group comprising a route vectoring, a route formation, an ingress vectoring, an ingress formation, an intercept, a missile intercept, a pure pursuit, a vectoring, a crank, a grinder, a pump, an egress, a vector relative to a primary enemy aircraft, an aircraft vector relative to a primary enemy aircraft centroid, and a missile vector relative to a primary enemy missile centroid.

11. The aircraft management system of claim 1 wherein the aircraft system is selected from a group comprising one of a single aircraft, a plurality of aircraft, and the plurality of aircraft on a team.

12. The aircraft management system of claim 1 , wherein the aircraft system is selected from at least one of an aircraft in a simulation or a physical aircraft.

13. The aircraft management system of claim 1 , wherein the set of machine learning models is selected from a group comprising a neural network, a reinforcement learning neural network, a multi-layer perceptron, a reinforcement learning machine learning model, and a proximal policy optimization machine learning model.

14. An agent management system comprising:
a computer system;
a simulator in the computer system configured to run a simulation environment; receive actions for the simulation environment; input the actions into the simulation environment; and output observations from the simulation environment; and
an agent comprising:
a machine learning model configured to receive relevant observations for an aircraft system and select a maneuver behavior for the aircraft system in response to receiving the relevant observations;
a rule system configured to select a set of actions to execute the maneuver behavior for the aircraft system selected by machine learning model;
an agent environment interface in the computer system, wherein the agent environment interface is configured to send actions selected by the agent to the simulator; receive observations from the simulator in response to the actions sent to the simulator; select the observations for the aircraft system to form the relevant observations; send the relevant observations to the agent; and determine a reward from using the observations from the simulation; and
an agent optimizer in the computer system, wherein the agent optimizer is configured receive the reward from the agent environment interface in response to the set of actions sent the simulator; determine a set of weight adjustments based on the reward; and send set of weight adjustments to the machine learning model.

15. The agent management system of claim 14 , wherein the machine learning model is selected from a group comprising a neural network, a reinforcement learning neural network, a multi-layer perceptron, a reinforcement learning machine learning model, and a proximal policy optimization machine learning model.

16. A vehicle management system comprising:
a computer system; and
an agent comprising:
a machine learning model system trained to receive observations for the vehicle system and select a behavior for the vehicle system in response to receiving the observations; and
a rule system configured to select a set of actions to execute the behavior for the vehicle system in response to a selection of the behavior by the machine learning model system.

17. The vehicle management system of claim 16 , wherein the rule system is configured to select a selected action in response to receiving the observations, wherein the selected action is for a different behavior type from a behavior type for the behavior selected by the machine learning model system.

18. The vehicle management system of claim 17 , wherein the selected action is selected from a group comprising selecting a weapon, targeting an object, firing the weapon, and capturing an image.

19. The vehicle management system of claim 16 , wherein the agent controls one of a single vehicle, a plurality of vehicles, and a plurality of vehicles on a team.

20. The vehicle management system of claim 16 , wherein the behavior is a maneuver behavior and is selected from a group comprising a route vectoring, a route formation, an ingress vectoring, an ingress formation, an intercept, a missile intercept, a pure pursuit, a vectoring, a crank, a grinder, a pump, an egress, a vector relative to a primary enemy aircraft, an aircraft vector relative to a primary enemy aircraft centroid, and a missile vector relative to a primary enemy missile centroid.

21. The vehicle management system of claim 16 , wherein the machine learning model system is a set of machine learning models that are selected from at least one of a neural network, a reinforcement learning neural network, a multi-layer perceptron, a reinforcement learning machine learning model, or a proximal policy optimization machine learning model.

22. The vehicle management system of claim 16 , wherein vehicle system is selected from at least one of a vehicle in a simulation or a physical vehicle.

23. The vehicle management system of claim 16 , wherein the vehicle system is selected at least one of a mobile platform, an aircraft, a fighter, a commercial airplane, a tilt-rotor aircraft, a tilt wing aircraft, a vertical takeoff and landing aircraft, an electrical vertical takeoff and landing vehicle, a personal air vehicle, a surface ship, a tank, a personnel carrier, a train, a spacecr

Assignees

• Boeing Co

Inventors

• Fadaie Joshua Gould
• Soleyman Sean
• Hung Fan Hin
• Khosla Deepak
• Tullock Charles Richard

Classifications

• G08G5/20

  Arrangements for acquiring, generating, sharing or displaying traffic information (arrangements for monitoring traffic G08G5/72) · CPC title

• G06N20/00 Primary

  Machine learning · CPC title

• G08G5/30 Primary

  Flight plan management · CPC title

• G06N3/006

  based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO] · CPC title

• G06N3/08

  Learning methods · CPC title