Systems, devices, and methods for on-board sensing and control of micro aerial vehicles

Patent metadata

Field	Value
Publication number	US-10037028-B2
Application number	US-201615218938-A
Country	US
Kind code	B2
Filing date	Jul 25, 2016
Priority date	Jul 24, 2015
Publication date	Jul 31, 2018
Grant date	Jul 31, 2018

Abstract

Systems, devices, and methods for on-board sensing and control of robotic vehicles (e.g., MAVs) using commercial off-the-shelf hand-held electronic devices as a sensing and control system are provided. In one aspect, a system for controlling a micro aerial vehicle may include one or more sensors, a state estimation module in communication with the one or more sensors, the state estimation module being configured to generate an estimated pose of the micro aerial vehicle based on inputs from the one or more sensors, and a position controller in communication with the state estimation module and configured to communicate attitude commands to an attitude controller of the micro aerial vehicle. Each of the one or more sensors, the state estimation module, and the position controller may be contained in a commercial off-the-shelf hand-held electronic device that is configured to be coupled to the micro aerial vehicle.

First claim

What is claimed is:

1. A system for controlling a micro aerial vehicle comprising:
one or more sensors, wherein the one or more sensors comprise one or more cameras and one or more of a gyroscope and/or an accelerometer;
a state estimation module in communication with the one or more sensors, the state estimation module comprising:
a visual inertial odometry system in communication with the one or more cameras and the one or more of a gyroscope and/or an accelerometer; and
an estimator in communication with the visual inertial odometry system and the one or more sensors, the estimator being configured to generate a state estimate of the micro aerial vehicle based on inputs from the one or more of a gyroscope and/or an accelerometer and to generate an estimated pose by updating the state estimate based on inputs from the visual inertial odometry system; and

a position controller in communication with the state estimation module and configured to communicate attitude commands to an attitude controller of the micro aerial vehicle;
wherein the state estimation module is configured to transmit the estimated pose to the position controller at a rate of about 200 Hz or greater; and
wherein each of the one or more sensors, the state estimation module, and the position controller are contained in a hand-held electronic device that is mounted to the micro aerial vehicle.

2. The system of claim 1 , wherein the estimator comprises an Unscented Kalman Filter in communication with the one or more sensors.

3. The system of claim 1 , wherein the state estimation module is configured to send visualization data of the estimated pose to a remote user interface.

4. The system of claim 1 , wherein the position controller comprises a nonlinear controller.

5. The system of claim 1 , wherein the position controller is configured to receive trajectory planning information from a remote user interface.

6. The system of claim 1 , wherein the attitude controller is contained in the hand-held electronic device.

7. A method for controlling a micro aerial vehicle using a hand-held electronic device, the method comprising:
mounting the hand-held electronic device to the micro aerial vehicle;
receiving inputs from one or more sensors in the hand-held electronic device, wherein the one or more sensors comprise one or more cameras and one or more of a gyroscope and/or an accelerometer;
generating an estimated pose of the micro aerial vehicle based on the inputs from the one or more sensors, wherein generating the estimated pose comprises:
generating a state estimate of the micro aerial vehicle based on inputs from the one or more of a gyroscope and/or an accelerometer; and
using visual inertial odometry to localize dynamics of a body of the micro aerial vehicle with respect to an inertial frame and update the state estimate based on inputs from the one or more cameras;

generating attitude commands based on the estimated pose of the micro aerial vehicle; and
communicating the attitude commands to a control component of the micro aerial vehicle;
wherein generating an estimated pose of the micro aerial vehicle comprises updating the state estimate at a rate of about 200 Hz or greater;
wherein receiving inputs, generating an estimated pose, and generating attitude commands are all performed on the hand-held electronic device.

8. The method of claim 7 , wherein generating an estimated pose comprises applying an Unscented Kalman Filter to the inputs from one or more sensors to estimate a full state of the micro aerial vehicle.

9. The method of claim 7 , wherein generating attitude commands further comprises receiving trajectory planning information from a remote user interface.

10. The method of claim 7 , wherein generating attitude commands further comprises coordinating a trajectory plan with at least one additional micro aerial vehicle.

11. The method of claim 10 , wherein coordinating a trajectory plan comprises:
at a remote user interface, determining a solution for a system of equations that represent straight-line trajectories of each of the micro aerial vehicle and the at least one additional micro aerial vehicle;
sending coefficients of a respective one of the system of equations to the micro aerial vehicle; and
generating the attitude commands based on the coefficients.

12. The method of claim 7 , wherein the control component is contained in the hand-held electronic device.

13. The method of claim 7 , further comprising sending visualization data of the estimated pose to a remote user interface.

14. The method of claim 13 , wherein the visualization data of the estimated pose comprises point clouds.

Assignees

• Univ Pennsylvania

Inventors

• Loianno Giuseppe
• Mulgaonkar Yash Shailesh
• Kumar R Vijay

Classifications

• G01C21/206

  specially adapted for indoor navigation · CPC title

• B64U2201/20

  Remote controls · CPC title

• B64U2201/10

  autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS] · CPC title

• G05D1/0088 Primary

  characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours (using knowledge based models G06N5/00) · CPC title

• G05D1/0808

  specially adapted for aircraft · CPC title