System and method to operate a drone

Patent metadata

Field	Value
Publication number	US-10095243-B2
Application number	US-201615232366-A
Country	US
Kind code	B2
Filing date	Aug 9, 2016
Priority date	Dec 8, 2015
Publication date	Oct 9, 2018
Grant date	Oct 9, 2018

Abstract

A method for controlling a drone includes receiving a natural language request for information about a spatial location, parsing the natural language request into data requests, configuring a flight plan and controlling one or more drones to fly over the spatial location to obtain data types based on the data requests, and extracting and analyzing data to answer the request. The method can include extracting data points from the data types, obtaining labels from a user for one or more of the data points, predicting labels for unlabeled data points from a learning algorithm using the labels obtained from the user, determining the predicted labels are true labels for the unlabeled data points and combining the extracted data, the user labeled data points and the true labeled data points to answer the request for information. The learning algorithm may be active learning using a support vector machine.

First claim

What is claimed is:

1. A method for controlling one or more drones to respond to a request for information, comprising;
receiving a natural language request for information about a spatial location;
parsing the natural language request into a plurality of data requests, each data request of the plurality of data requests corresponding to a portion of data necessary to answer the natural language request;
configuring a flight plan for one or more drones over the spatial location based on the plurality of data requests;
controlling one or more drones to fly over the spatial location according to the configured flight plan to obtain a plurality of data types from the spatial location based on the plurality of data requests, each data type providing a corresponding portion of the data necessary to answer the natural language request;
extracting data responsive to the plurality of data requests from the plurality of data types obtained by the one or more drones; and
analyzing the responsive data to provide an answer to the natural language request for information.

2. The method of claim 1 , wherein the data types include one or more of data obtained from an imaging system and data obtained from one or more sensors and wherein the plurality of data requests include one or more of a set of data to be collected, a location from which the data set is to be collected, analytics to be performed on the data set a timeframe to collect the data set.

3. The method of claim 1 , further comprising selecting one or more drones based on matching drone capabilities to one or more of the plurality of data requests and uploading flight plans to the one or more drones, receiving real-time telemetry from the drone and performing analytics on the real-time telemetry to determine real-time flight conditions.

4. The method of claim 3 , further comprising displaying the real-time telemetry and real-time flight conditions on a user interface (UI) in a mobile application and manually controlling the flight path of the one or more drones from the UI.

5. The method of claim 1 , further comprising:
extracting a plurality of data points responsive to the plurality of data requests from the plurality of data types obtained by the one or more drones;
obtaining labels for one or more of the plurality of data points; and
combining the extracted data and the labeled data points to provide an answer to the first natural language request for information.

6. The method of claim 5 , further comprising:
obtaining labels from a user for one or more of the plurality of data points;
predicting labels for unlabeled data points from a learning algorithm using the labels obtained from the user;
determining the predicted labels are true labels for the unlabeled data points; and
combining the extracted data, the user labeled data points and the true labeled data points to provide an answer to the first natural language request for information.

7. The method of claim 1 , further comprising;
searching for existing sources for the plurality of data requests;
determining that there are one or more existing sources for one or more of the plurality of data requests;
analyzing the existing sources to obtain first data responsive to the plurality of data requests;
determining that there are no existing sources for two or more of the plurality of data requests and identifying the data requests with no existing source as missing data requests;
wherein configuring a flight plan comprises configuring a flight plan for one or more drones over the spatial location based on the missing data requests;
wherein controlling one or more drones comprises controlling one or more drones to fly over the spatial location according to the configured flight plan to obtain a plurality of data types from the spatial location based on the missing data requests;
wherein extracting data comprises extracting a plurality of data points responsive to the plurality of data requests from the plurality of data types obtained by the one or more drones obtaining labels from a user for one or more of the plurality of data points;
determining whether there are unlabeled data points;
predicting labels for the unlabeled data points from a learning algorithm using the labels obtained from the user;
determining the predicted labels are true labels for the unlabeled data points; and
wherein analyzing responsive data comprises combining the first data, the user labeled data points and the true labeled data points to provide an answer to the first natural language request for information.

8. The method of claim 7 , wherein the learning algorithm comprises active learning using a support vector machine.

9. The method of claim 8 , further comprising:
receiving a second natural language request for information about the spatial location;
parsing the second natural language request into a plurality of second data requests;
searching for existing sources for the plurality of second data requests;
determining that there are one or more existing sources for one or more of the plurality of second data requests;
analyzing the existing sources to obtain second data responsive to the plurality of second data requests;
determining that there are no existing sources for two or more of the plurality of second data requests and identifying the data requests with no existing source as missing data requests;
configuring a flight plan for one or more drones over the spatial location based on the missing data requests;
controlling one or more drones to fly over the spatial location according to the configured flight plan to obtain a plurality of data types from the spatial location based on the missing data requests;
extracting a plurality of data points responsive to the plurality of data requests from the plurality of data types obtained by the one or more drones;
determining that there is a user label or a predicted true label for each of the plurality of data points; and
combining the second data, the user labeled data points and the predicted true labeled data points to provide an answer to the second natural language request for information.

10. A non-transitory article of manufacture tangibly embodying computer readable instructions, which when implemented, cause a computer to perform the steps of a method for controlling one or more drones to respond to a request for information, comprising;
receiving a natural language request for information about a spatial location;
parsing the natural language request into a plurality of data requests, each data request of the plurality of data requests corresponding to a portion of data necessary to answer the natural language request;
configuring a flight plan for one or more drones over the spatial location based on the plurality of data requests;
controlling one or more drones to fly over the spatial location according to the configured flight plan to obtain a plurality of data types from the spatial location based on the plurality of data requests, each data type providing a corresponding portion of the data necessary to answer the natural language request;
extracting data responsive to the plurality of data requests from the plurality of data types obtained by the one or more drones; and
analyzing the responsive data to provide an answer to the natural language request for information.

11. The non-transitory article of manufacture of claim 10 , further comprising computer readable instructions, which when implemented, cause a computer to perform the steps of:
extracting a plurality of data points responsive to the plurality of data requests from the plurality of data types obtained by the one or more drones,

Assignees

• IBM

Inventors

• Boland Gregory F
• Kozloski James R
• Ma Yu
• Manweiler Justin G
• Siemonsen Kevin E
• Topkara Umut
• Vogt Katherine
• Weisz Justin D

Classifications

• G06N7/01

  Probabilistic graphical models, e.g. probabilistic networks · CPC title

• B64U2201/104

  using satellite radio beacon positioning systems, e.g. GPS · CPC title

• G06N20/00

  Machine learning · CPC title

• G06F40/205

  Parsing · CPC title

• G05D1/0088

  characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours (using knowledge based models G06N5/00) · CPC title