Monitoring Aerial Application Tasks and Recommending Corrective Actions

Patent metadata

Field	Value
Publication number	US-2019166752-A1
Application number	US-201715828575-A
Country	US
Kind code	A1
Filing date	Dec 1, 2017
Priority date	Dec 1, 2017
Publication date	Jun 6, 2019
Grant date	—

Abstract

Recommending corrective action during aerial application is provided. An unmanned aerial vehicle is navigated to a geolocation where an aerial application task is currently carried out. First sensor data is received from the unmanned aerial vehicle that characterize a quality of the aerial application task. Measures to be carried out to increase the quality of the aerial application task are determined based on the first sensor data. The measures are outputted while the aerial application task is ongoing so that the aerial application task can be adapted while the aerial application task occurs.

First claim

What is claimed is:

1 . A method for recommending corrective action during aerial application, the method comprising:
navigating an unmanned aerial vehicle to a geolocation where an aerial application task is currently carried out; receiving first sensor data from the unmanned aerial vehicle that characterize a quality of the aerial application task; determining measures to be carried out to increase the quality of the aerial application task based on the first sensor data; and outputting the measures while the aerial application task is ongoing so that the aerial application task can be adapted while the aerial application task occurs.

2 . The method of claim 1 further comprising:
receiving second sensor data from a second aerial vehicle performing the aerial application task that characterize properties corresponding to the second aerial vehicle, wherein the determining of the measures is based on the first sensor data and the second sensor data.

3 . The method of claim 2 , wherein the second aerial vehicle is an artificial intelligence controlled aerial vehicle, the method further comprising:
calculating control commands based on the measures; and sending the control commands to the artificial intelligence controlled aerial vehicle to adapt the aerial application task while the aerial application task is ongoing.

4 . The method of claim 3 , wherein the second aerial vehicle is a human controlled aerial vehicle, the method further comprising:
generating a recommendation with instructions on how to adapt the aerial application task based on the control commands; and sending the recommendation on how to adapt the aerial application task to the human controlled aerial vehicle while the aerial application task is ongoing.

5 . The method of claim 1 further comprising:
generating a lookahead tree having a root node that represents a current situation in a timeline corresponding to the aerial application task;
setting a first timer for a first computation phase, which estimates a level of risk corresponding to the aerial application task, to a first computation time threshold limit;
setting a first prediction limit for the first computation phase that defines a maximum number of defined time periods to predict into a future;
setting a random walk node selection strategy to select nodes in the lookahead tree;
selecting a leaf node in the lookahead tree utilizing the set node selection strategy;
setting the leaf node as a new current situation; and
adding a path from the root node to the leaf node to the timeline to obtain a hypothetical timeline of what occurred up to the new current situation.

6 . The method of claim 5 further comprising:
selecting a pilot decision for the new current situation that an agricultural aircraft pilot may take while performing the aerial application task based on previous pilot decisions taken during same or similar situations;
generating a forecast of a weather change that can occur during the new current situation;
applying the pilot decision and the weather change to the current situation one defined time period into the future;
predicting a consequence of the new current situation that is a result of the pilot decision and the weather change applied to the new current situation one defined time period into the future; and
adding the consequence of the new current situation to the lookahead tree.

7 . The method of claim 6 further comprising:
selecting a predicted consequence of the new current situation from the lookahead tree using the set node selection strategy;
setting the predicted consequence as the new current situation;
running a simulation of possible series of events occurring from the predicted consequence of the new current situation that extends the hypothetical timeline by applying pilot decisions, weather changes, and situation consequences recursively up to the first prediction limit for the first computation phase; and
utilizing a last situation reached in the hypothetical timeline at the first prediction limit corresponding to the first computation phase as a situation simulation.

8 . The method of claim 7 further comprising:
generating a score indicating how desirable the situation simulation is for the aerial application task; and
updating scores of the predicted consequence and its ancestor nodes in the lookahead tree to account for the score corresponding to the situation simulation reflecting an impact of a possible outcome of the situation simulation.

9 . The method of claim 8 further comprising:
determining whether the first timer reached the first computation time threshold limit;
responsive to determining that the first timer has reached the first computation time threshold limit, determining whether a second computation phase for generating corrective action recommendations has already started;
responsive to determining that the second computation phase has not already started, evaluating the level of risk corresponding to the current situation at the root node of the lookahead tree taking into account what occurred before the new current situation in the timeline and what could happen next in the lookahead tree considering all possible consequences;
determining whether the level of risk is greater than or equal to a risk level threshold value; and
responsive to determining that the level of risk is greater than or equal to the risk level threshold value, starting the second computation phase.

10 . The method of claim 9 further comprising:
setting a second timer for the second computation phase to a second computation time threshold limit;
setting a second prediction limit, which defines a maximum number of defined time periods to predict into the future, for the second computation phase;
starting the second timer for the second computation phase; and
setting an Upper Confidence Bound 1 selection strategy to select nodes in the lookahead tree.

11 . The method of claim 10 further comprising:
determining whether the second timer reached the second computation time threshold limit; and
responsive to determining that the second timer has reached the second computation time threshold limit, selecting a corrective action recommendation for the agricultural aircraft pilot that has a best outcome in the lookahead tree of decreasing the level of risk.

12 . The method of claim 1 , wherein the unmanned aerial vehicle is a dedicated monitoring unmanned aerial vehicle that captures images of a second aerial vehicle performing the aerial application task, a target area of the aerial application task, and a cloud of product emitted by the second aerial vehicle while performing the aerial application task.

13 . The method of claim 12 , wherein a remote computer controls the dedicated monitoring unmanned aerial vehicle.

14 . A computer system for recommending corrective action during aerial application, the computer system comprising:
a bus system; a storage device connected to the bus system, wherein the storage device stores program instructions; and a processor connected to the bus system, wherein the processor executes the program instructions to:
navigate an unmanned aerial vehicle to a geolocation where an aerial application task is currently carried out;
receive first sensor data from the unmanned aerial vehicle that characterize a quality of the aerial application task;
determine measures to be carried out to increase the quality of the aerial application task based on the first sensor data; and
output the measures while the aerial application task is ongoing so that the aerial application

Assignees

• IBM

Inventors

• Sidoti Stefano
• Sturaro Agostino

Classifications

• B64D1/16

  Dropping or releasing powdered, liquid, or gaseous matter, e.g. for fire-fighting (jettisoning fuel B64D37/26) · CPC title

• B64U2101/20

  for use as communications relays, e.g. high-altitude platforms · CPC title

• B64U2101/30

  for imaging, photography or videography · CPC title

• A01B79/005 Primary

  Precision agriculture · CPC title

• G06T11/60

  Creating or editing images; Combining images with text · CPC title