Spatial-temporal forecasting for predictive situational awareness

Patent metadata

Field	Value
Publication number	US-9952591-B2
Application number	US-201514951212-A
Country	US
Kind code	B2
Filing date	Nov 24, 2015
Priority date	Nov 24, 2015
Publication date	Apr 24, 2018
Grant date	Apr 24, 2018

Abstract

An interface includes a human machine interface (HMI) to provide a visualization of current and future states of a mission associated with one or more unmanned vehicles. A time specifier provides input to the HMI to specify a current time or a future time for the visualization of the current and future states of the mission. A prediction engine generates predictions of the current and future states of the mission for the visualization based on the current time or the future time specified by the time specifier.

First claim

What is claimed is:

1. A system, comprising:
one or more computers executing computer executable components, the computer executable components comprising:
a state aggregator that aggregates mission plan inputs and data feeds to generate a current world state output for an unmanned vehicle (UxV) mission, the current world state output describes the current state of the mission plan inputs and the data feeds, the mission plan inputs describes current mission destinations and tasks to be executed by the UxV and the data feeds describe current detected events that affect the current world state of the UxV mission;
a time specifier to generate a command to specify a time window to execute the current UxV mission based on a user input;
a prediction engine that receives the current world state output of the UxV mission from the state aggregator and generates a future world state projection for the specified time window to predict a future time projection of the UxV mission, wherein the future world state projection is based on a prediction from at least one model of one or more future states of the data feeds received by the state aggregator; and
a human machine interface (HMI) to generate a future world state visualization of the future world state projection for the UxV mission for the specified time window.

2. The system of claim 1 , wherein the time specifier comprises a time slider graphical input to specify a current time or the future time for the visualization, a voice input to specify the current time or the future time for the visualization, a hand or figure gesture to specify the current time or the future time for the visualization, or a retinal scan input to specify the current time or the future time for the visualization.

3. The system of claim 1 , wherein the data feeds include at least one of a mission plan input to provide primary and contingency route information for the unmanned vehicle, a tasking input to describe actions to perform, and a weather input to provide information relating to current and future weather patterns.

4. The system of claim 3 , wherein the data feeds include at least one of a threat input to provide threat information relating to external threats to the UxV, a track input to relating to the status of an entity encountered by the UxV, a sensor input to provide data collected by the UxV, and a communications input to provide information regarding the status of communications assets that are available to the unmanned vehicle.

5. The system of claim 1 , wherein the at least one model includes a UxV predictor to predict future flight paths and routes of the UxV, an environment predictor to predict future environment status for the UxV, a communications predictor to predict the status of communications assets that are available in the future.

6. The system of claim 5 , wherein the at least one model includes a weather predictor to predict future weather patterns to be encountered by the UxV, a tasking predictor to predict actions that can be performed in the future by the UxV, and a threat predictor to predict future threats to the UxV.

7. The system of claim 6 , the at least one model includes a sensor predictor to predict the status of sensors that are available to the UxV in the future and a tracks predictor to predict the status of entities that may be encountered in the future by the UxV.

8. The system of claim 1 , further comprising a world state rendering pipeline to generate the future visualization for the HMI based on the future state prediction from the prediction engine.

9. The system of claim 1 , wherein the future world state visualization is variable for an alternative scenario based on a change related to at least one of the data feeds.

10. The system of claim 9 , wherein the change is defined by a user input.

11. The system of claim 1 , wherein the at least one model employs a heuristic algorithm to provide the future world state projection within predefined acceptable error bounds.

12. A non-transitory computer readable medium having computer executable instructions stored thereon, the computer executable instructions configured to:
aggregate mission plan inputs and data feeds to generate a current world state output for an unmanned vehicle (UxV) mission, the current world state output describes the current state of the mission plan inputs and the data feeds, the mission plan inputs describes current mission destinations and tasks to be executed by the UxV and the data feeds describe current detected events that affect the current world state of the UxV mission;
receive a user input comprising a specified time window to execute the current UxV mission;
generate a future world state projection for the UxV mission for the specified time window based on the current world state and a prediction from at least one model of one or more future states of the data feeds; and
generate a future world state visualization for the UxV mission based on the future world state projection for the specified time window.

13. The computer readable medium of claim 12 , wherein the data feeds include at least one of a mission plan input to provide primary and contingency route information for the unmanned vehicle, a tasking input to describe actions to perform, and a weather input to provide information relating to current and future weather patterns.

14. The computer readable medium of claim 13 , wherein the data feeds include at least one of a threat input to provide threat information relating to external threats to the UxV, a track input to relating to the status of an entity encountered by the UxV, a sensor input to provide data collected by the UxV, and a communications input to provide information regarding the status of communications assets that are available to the unmanned vehicle.

15. The computer readable medium of claim 12 , wherein the at least one model includes a UxV predictor to predict future flight paths and routes of the UxV, an environment predictor to predict future environment status for the UxV, a communications predictor to predict the status of communications assets that are available in the future.

16. The computer readable medium of claim 15 , wherein the at least one model includes a weather predictor to predict future weather patterns to be encountered by the UxV, a tasking predictor to predict actions that can be performed in the future by the UxV, and a threat predictor to predict future threats to the UxV.

17. The computer readable medium of claim 16 , the at least one model includes a sensor predictor to predict the status of sensors that are available to the UxV in the future and a tracks predictor to predict the status of entities that may be encountered in the future by the UxV.

18. A method, comprising:
aggregating, by a system comprising a processor, mission plan inputs and data feeds to generate a current world state output for an unmanned vehicle (UxV) mission, the mission plan inputs describe current mission destinations and tasks to be executed by the UxV and the data feeds describe current detected events that affect the current world state of the UxV mission;
receiving, by the system, a specified time window to execute the current UxV mission;
generating, by the system, a future world state projection for the UxV mission for the specified time window based on the current world state output and a prediction from at least one model of one or more future states of the data feeds; and
generating, by the system, a future world state visualization for the UxV mission based on the future world st

Assignees

• Parekh Devang R
• Fung Henry H
• Cao Feng
• Oddo Louis
• Patel Miteshkumar K
• Parsons William
• Northrop Grumman Systems Corp

Inventors

• Parekh Devang R
• Fung Henry H
• Cao Feng
• Oddo Louis
• Patel Miteshkumar K
• Parsons William

Classifications

• B64U2201/20

  Remote controls · CPC title

• B64C2201/12

  Operations & Transport · mapped topic

• G05D1/0202

  specially adapted to aircraft · CPC title

• B64D45/00 Primary

  Aircraft indicators or protectors not otherwise provided for · CPC title

• G05D1/0016 Primary

  characterised by the operator's input device (input arrangements for computing systems in general G06F3/00) · CPC title