Systems and Methods for Navigating Aerial Vehicles Using Deep Reinforcement Learning
US-2021123741-A1 · Apr 29, 2021 · US
US11527165B2 · US · B2
| Field | Value |
|---|---|
| Publication number | US-11527165-B2 |
| Application number | US-201916554768-A |
| Country | US |
| Kind code | B2 |
| Filing date | Aug 29, 2019 |
| Priority date | Aug 29, 2019 |
| Publication date | Dec 13, 2022 |
| Grant date | Dec 13, 2022 |
A practical reading order for non-experts. Skip the full description unless you need deep technical detail.
What the patent document calls the invention.
A short plain-language summary of the technical disclosure.
Who owns or filed the patent and who is credited as inventor.
Filing, priority, publication, and grant dates set the timeline.
The legal scope of protection — read this for what is actually claimed.
Technology tags used to group this patent with similar filings.
Prior art links and similar publications in this corpus.
Official abstract text for this publication.
A method, apparatus, and system for controlling an aircraft. A target state for the aircraft is identified. A current mission state is determined for the aircraft. A sequence of actions is selected from a pool of potential actions to reach the target state from the current mission state for the aircraft. The sequence of actions is selected based on the current mission state. The actions in the sequence of actions for which preconditions for the actions that have been met are performed. The actions are performed in an order defined by the sequence of actions.
Opening claim text (preview).
What is claimed is: 1. A method comprising: receiving, by a computer system, a goal for an aircraft; identifying, by the computer system, a set of state variables defining a target state that satisfies the goal for the aircraft; determining, by the computer system, a set of state variables defining a current mission state for the aircraft; lacking a predetermined decision tree pathway that comprises tactical actions to the goal; generating, by the computer system, a sequence of actions for changing the set of state variables defining the current mission state for the aircraft to the set of state variables defining the target state for the aircraft, by evaluating an effect virtually, respectively, on each state variable in the set of variables defining the current mission state, of all potential actions from the current mission state for the aircraft and virtually creating a temporary mission state resultant, respectively, from taking each one of the all potential actions and repeating iteratively for all potential actions from each temporary mission state until generating the sequence of actions producing the set of state variables defining the target state; assigning each action in the sequence of actions a cost; the computer system selecting, using a cost of performing the actions, one sequence of actions for performing; and performing, by the computer system, actions in the sequence of actions changing the set of state variables defining the current mission state for the aircraft to the set of state variables defining the target state after meeting preconditions for the actions in an order defined by the sequence of actions changing the set of state variables defining the current mission state for the aircraft to the set of state variables defining the target state. 2. The method of claim 1 , wherein the method further comprises the computer system: generating more than one sequence of actions changing the set of state variables defining the current mission state for the aircraft to the set of state variables defining the target state for the aircraft. 3. The method of claim 1 , further comprising the computer system: planning, while performing actions in the sequence of actions changing the set of state variables defining the current mission state for the aircraft to the set of state variables defining the target state, and using a change in a state variable that defines the current mission state, a new sequence of actions for reaching the target state. 4. The method of claim 1 , wherein determining, by the computer system, the current mission state for the aircraft comprises: receiving all state variables from aircraft systems in the aircraft; and determining the current mission state using the state variables for the current mission state. 5. The method of claim 1 , wherein generating the sequence of actions comprises: identifying nodes in a path from the current mission state to the target state based on the nodes having a lowest cost; and selecting the sequence of actions changing the set of state variables defining the current mission state for the aircraft to the set of state variables defining the target state based on the nodes identified in the path. 6. The method of claim 5 , wherein a cost used to determine the lowest cost comprises a set of factors selected from at least one of a number of effects of an action, a monetary cost, an amount of time, a maintenance cost to be incurred, an amount of fuel to be consumed, a personnel cost connected to time, or passenger comfort. 7. The method of claim 1 , wherein generating the sequence of actions changing the set of state variables defining the current mission state for the aircraft to the set of state variables defining the target state further comprises selecting, by the computer system, actions, for reaching the target state based on the current mission state for the aircraft using a set of path planning algorithms. 8. The method of claim 7 , wherein the set of path planning algorithms is selected from at least one an A* search algorithm, a Dijkstra's algorithm, a D*, an incremental search algorithm, a Backtracking algorithm, a Fringe search, an Any-angle path planning algorithm, an iterative deepening A* search algorithm, a Bellman-Ford search algorithm, a Floyd-Warshall algorithm, a Hill climbing algorithm, a Bidirectional search algorithm, or a Johnson's algorithm. 9. The method of claim 1 further comprising: responsive to an event, generating the sequence of actions, by the computer system basing the sequence of actions changing the set of state variables defining the current mission state for the aircraft to the set of state variables defining the target state on the current mission state. 10. The method of claim 9 , wherein the event is selected from a periodic event, a non-periodic event, a current mission state change, reaching a sub target state, an expiration of a timer, a state variable change, a change in a configuration of the aircraft, and a performance of an action. 11. The method of claim 1 , wherein the current mission state of the aircraft comprises set of state variables for the aircraft. 12. The method of claim 1 , wherein preconditions for an action comprise at least one of a current location of the aircraft, a performance of a selected action, a configuration of the aircraft, a position of a control surface, a weather condition, or an instruction from an air traffic controller. 13. The method of claim 1 , wherein the target state is one of an operational target state and a spatial target state. 14. The method of claim 1 , wherein the set of state variables define the target state with a set of values for the target state. 15. The method of claim 1 further comprising: receiving the goal from a human machine interface system. 16. The method of claim 15 , wherein the human machine interface system is at least one of located in the aircraft, a remote location in communication with the aircraft, an air traffic control system, or an airline system. 17. An aircraft control system that comprises: a computer system; and an action manager in the computer system, wherein the action manager is configured to: identify a first set of variables that define a target state that satisfies a goal for an aircraft; determine a second set of variables that define a current mission state for the aircraft from state variables received from aircraft systems in the aircraft; generate, based upon a set of path planning algorithms, a sequence of actions that change the second set of variables to the first set of variables based upon an evaluation of a virtual effect, respectively, on each variable in the second set of variables that define the current mission state, of all potential actions from the current mission state to create a temporary mission state resultant, respectively, from each one of the all potential actions and an iterative repeat thereof for all potential actions from each temporary mission state until dynamic generation of the sequence of actions that change the second set of variables to the first set of variables, wherein the computer system lacks a predetermined decision tree pathway that comprises tactical actions to the goal, and selections of the actions and the sequence of actions that change the second set of state variables to the set of state variables that define the target state are based on a cost of performance of each of the actions; select a sequence of actions from a list of generated sequences of actions that reach the target state from the current mission state
Pool · CPC title
Address formation of the next instruction, e.g. by incrementing the instruction counter (G06F9/38 takes precedence) · CPC title
Instruments for performing navigational calculations (G01C21/24, G01C21/26 take precedence) · CPC title
Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 (measuring distance traversed on the ground by a vehicle G01C22/00; control of position, course, altitude or attitude of vehicles G05D1/00; traffic control systems for road vehicles involving transmission of navigation instructions to the vehicle G08G1/0968) · CPC title
Aircraft indicators or protectors not otherwise provided for · CPC title
Related publications grouped by family.
Answers are generated from the same data shown on this page.