Intelligent trajectory adviser system for unmanned aerial vehicles in complex environments

Patent metadata

Field	Value
Publication number	US-11046430-B1
Application number	US-201815955661-A
Country	US
Kind code	B1
Filing date	Apr 17, 2018
Priority date	Apr 17, 2017
Publication date	Jun 29, 2021
Grant date	Jun 29, 2021

Abstract

Systems and methods are provided for improving the flight safety of fixed- and rotary-wing unmanned aerial systems (UAS) operating in complex dynamic environments, including urban cityscapes. Sensors and computations are integrated to predict local winds and promote safe operations in dynamic urban regions where GNSS and other network communications may be unavailable. The system can be implemented onboard a UAS and does not require in-flight communication with external networks. Predictions of local winds (speed and direction) are created using inputs from sensors that scan the local environment. These predictions are then used by the UAS guidance, navigation, and control (GNC) system to determine safe trajectories for operations in urban environments.

First claim

What is claimed is:

1. An improved method for determining real-time trajectory for an unmanned aerial system, the method comprising:
training a first neural network, said training including providing a machine learning module with pre-existing landscape data in a selected flight area with location data associated with the landscape data to train the first neural network;
training a second neural network, said training including providing the machine learning module with the pre-existing landscape data in the selected flight area with pre-existing wind prediction data for the selected flight area to train the second neural network;
uploading the first neural network and the second neural network to at least one computer on board the unmanned aerial system;
while being flown along an intended route, scanning, by a sensor on board the unmanned aerial system, a peripheral environment around the unmanned aerial system to produce imagery of the unmanned aerial system's current surroundings;
determining, by the at least one on board computer, actual positions of the unmanned aerial system using the first trained neural network and the imagery;
determining, by the at least one on board computer, predicted wind vectors using the second trained neural network and the imagery; and
providing the predicted wind vectors and actual position to a trajectory module to create or modify the trajectory of the unmanned aerial system.

2. The method of claim 1 , wherein the pre-existing landscape data include a synthetic representation of the landscape.

3. The method of claim 1 , wherein the pre-existing landscape data include scanned images from a previous flight of the unmanned aerial system.

4. The method of claim 1 , wherein the pre-existing wind prediction data are from a computer simulation.

5. The method of claim 1 , wherein the pre-existing wind prediction data are from calculations from a previous flight of the unmanned aerial system.

6. The method of claim 1 , wherein scanning the peripheral environment around the unmanned aerial system is performed with a camera onboard the unmanned aerial system.

7. The method of claim 1 , wherein scanning the peripheral environment around the unmanned aerial system is performed with LIDAR onboard the unmanned aerial system.

8. The method of claim 1 , wherein the trajectory module is a neural network.

9. The method of claim 1 , further comprising:
training at least a third neural network, said training including providing the machine learning module with the pre-existing landscape data in the selected flight area of the unmanned aerial system and the pre-existing wind prediction data for the selected flight area of the unmanned aerial system, wherein the second and at least third neural networks are associated with specific weather conditions; and
checking a weather forecast before takeoff, allowing the unmanned aerial system to select from the second and at least third neural networks to calculate a wind prediction given the weather forecast.

10. The method of claim 9 , wherein the unmanned aerial system is configured to communicate with established wind sensors throughout the flight to update which neural network is being used for wind predictions.

11. The method of claim 1 , wherein the first neural network is trained in-flight using past sensor data and location data collected by the unmanned aerial system.

12. The method of claim 1 , wherein the second neural network is trained using multiple wind directions, and the unmanned aerial system is configured for determining its orientation with respect to oncoming winds.

13. The method of claim 1 , wherein the second neural network is trained using multiple wind speeds, and the second neural network receives data on current wind speed to create a prediction of wind field based on wind speed and urban geometry.

14. A system for providing real-time trajectory input to an unmanned aerial system, the system comprising:
a trajectory module;
a computer disposed on the unmanned aerial system, the computer comprising a processor and a memory;
at least one sensor disposed on the unmanned aerial system and configured for scanning a peripheral environment around the unmanned aerial system to produce imagery of the unmanned aerial system's current surroundings;
a first neural network trained with pre-existing landscape data in a selected flight area with location data associated with the landscape data, the first neural network stored in the memory;
a second neural network trained with the pre-existing landscape data in the selected flight area with pre-existing wind prediction data for the selected flight area, the second neural network stored in the memory;
computer-readable instructions stored in the memory that, when executed by the processor when the unmanned aerial system is in flight, cause the processor to:
scan the peripheral environment around the unmanned aerial system to produce the imagery of the unmanned aerial system's current surroundings;
determine actual positions of the unmanned aerial system using the first trained neural network and the imagery;
determine the predicted wind vectors using the second trained neural network and the imagery; and
provide the predicted wind vectors and actual position to the trajectory module to create or modify the trajectory of the unmanned aerial system.

15. The system of claim 14 , wherein the trajectory module is a neural network.

16. The system of claim 14 , wherein scanning the peripheral environment around the unmanned aerial system is performed with a camera onboard the unmanned aerial system.

17. The system of claim 14 , wherein scanning the peripheral environment around the unmanned aerial system is performed with LIDAR onboard the unmanned aerial system.

18. The system of claim 14 , wherein the pre-existing landscape data include a synthetic representation of the landscape, scanned images from a previous flight of the unmanned aerial system, or a combination thereof.

19. The system of claim 14 , wherein the pre-existing wind prediction data are from a computer simulation, a previous flight of the unmanned aerial system, or a combination thereof.

Assignees

• Us Administrator Of The Nasa
• Nasa

Inventors

• Melton John Earl
• Nikaido Ben Edward

Classifications

• B64U2201/10

  autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS] · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06N3/08

  Learning methods · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/09

  Supervised learning · CPC title