Controlling an unmanned aerial vehicle by re-training a sub-optimal controller

Patent metadata

Field	Value
Publication number	US-11822345-B2
Application number	US-202017078356-A
Country	US
Kind code	B2
Filing date	Oct 23, 2020
Priority date	Oct 23, 2020
Publication date	Nov 21, 2023
Grant date	Nov 21, 2023

Abstract

A nonlinear dynamic control system is defined by a set of equations that include a state vector and one or more control inputs. Via a machine learning method, a sub-optimal controller is derived that stabilizes the nonlinear dynamic control system at an equilibrium point. The sub-optimal controller is retrained to be used as a stabilizing controller for the nonlinear dynamic control system under general operating conditions.

First claim

The invention claimed is:

1. A method of controlling an unmanned aerial vehicle, comprising:
defining a nonlinear dynamic control system by a set of equations that include a state vector z and one or more control inputs, the nonlinear dynamic control system comprising a physical plant of the unmanned aerial vehicle, the physical plant comprising three motors and at least one other motor that all provide thrust for the unmanned aerial vehicle;
via a machine learning method, deriving a sub-optimal controller that stabilizes the nonlinear dynamic control system at an equilibrium point, the machine learning method optimizing controller parameters by manipulating spectral properties of a Jacobian of the state vector at the equilibrium point; and
retraining the sub-optimal controller to be a stabilizing controller for the nonlinear dynamic control system under operating conditions of the physical plant; and
using the stabilizing controller to provide closed loop control of the three motors when the unmanned aerial vehicle is affected by the at least one other motor losing thrust.

2. The method of claim 1 , wherein deriving the sub-optimal controller comprises determining a stabilizing controller for a linear approximation of the nonlinear dynamic control system around the equilibrium point.

3. The method of claim 1 , wherein a time rate of change 2 of the state vector z is a function f(z;β) wherein β is a set of parameters of the sub-optimal controller, and wherein deriving the sub-optimal controller comprises manipulating spectral properties of a Jacobian map

A
⁡

(
β
)

=

∂

f
⁡

(

0
⁢

;

⁢
β

)

∂
z

at the equilibrium point.

4. The method of claim 3 , wherein:
the Jacobian map A(β) is generated using auto-differentiation;
a discrete time version A d (β) of A(β) is computed using a Taylor series expansion; and
β is found by solving an optimization problem min β {0,∥A d (β) k ∥−λ k }, wherein k≥1 and λ is a positive, real scalar having a value less than one.

5. The method of claim 1 , wherein retraining the sub-optimal controller comprises learning a parameterized map for the stabilizing controller that minimizes a quadratic loss function.

6. The method of claim 1 , wherein retraining the sub-optimal controller comprises using a model predictive control (MPC) approach in which non-parameterized control inputs of the sub-optimal controller are used as initial conditions to solve a finite horizon optimal control problem that explicitly generates optimal control inputs.

7. The method of claim 6 , wherein automatic differentiation is used to compute a gradient of a loss function used to solve the finite horizon optimal control problem.

8. The method of claim 6 , wherein automatic differentiation is used to compute a Hessian matrix of a loss function used to solve the finite horizon optimal control problem.

9. The method of claim 1 , wherein retraining the sub-optimal controller comprises learning a parameterized, state-dependent control map by solving a sequence of finite horizon optimal control problems.

10. The method of claim 1 , wherein retraining the sub-optimal controller comprises using automatic differentiation to determine a time rate of change 2 of the state vector z.

11. The method of claim 1 , wherein retraining the sub-optimal controller comprises using automatic differentiation to determine a control objective and a constraints function.

12. The method of claim 1 , wherein the stabilizing controller is used for a safe recovery of the unmanned aerial vehicle under loss of thrust of the at least one other motor.

13. A non-transitory computer-readable medium storing instructions operable by a processor to perform the method of claim 1 .

14. A method of controlling an unmanned aerial vehicle, comprising:
defining a nonlinear dynamic control system by a set of equations that include a state vector z and one or more control inputs u, the nonlinear dynamic control system comprising a physical plant of the unmanned aerial vehicle, the physical plant comprising three motors and at least one other motor that all provide thrust for the unmanned aerial vehicle;
via a machine learning method, deriving a sub-optimal controller that stabilizes the nonlinear dynamic control system at an equilibrium point; and
retraining the sub-optimal controller to be a stabilizing controller for the nonlinear dynamic control system under operating conditions of the physical plant, wherein retraining the sub-optimal controller comprises a model predictive control (MPC) approach in which a parameterized, state-dependent control map is learned by solving a sequence of finite horizon optimal control problems; and
using the stabilizing controller to provide closed loop control of the three motors when the unmanned aerial vehicle is affected by the at least one other motor losing thrust.

15. The method of claim 14 , wherein retraining the sub-optimal controller further comprises using automatic differentiation for a time rate of change 2 of the state vector z.

16. The method of claim 14 , wherein automatic differentiation is used to compute at least one of gradient and a Hessian matrix of a loss function used to solve the finite horizon optimal control problem.

17. A system comprising:
an unmanned aerial vehicle comprising a physical plant, the physical plant comprising three motors and at least one other motor that all provide thrust for the unmanned aerial vehicle; and
a computer comprising a memory coupled to a processor, the memory comprising instructions that cause the processor to:
define a non-linear dynamic control system by a set of equations that include a state vector z of the unmanned aerial vehicle and one or more control inputs u to the physical plant of the unmanned aerial vehicle;
via a machine learning method, derive a sub-optimal controller that stabilizes the unmanned aerial vehicle at an equilibrium point, the machine learning method optimizing controller parameters by manipulating spectral properties of a Jacobian of the state vector at the equilibrium point; and
retrain the sub-optimal controller to be a stabilizing controller for the unmanned aerial vehicle under operating conditions of the physical plant of the unmanned aerial vehicle, wherein the stabilizing controller is transferred to the unmanned aerial vehicle, the stabilizing controller used by the unmanned aerial vehicle to provide closed loop control of the three motors when the unmanned aerial vehicle is affected by the at least one other motor losing thrust.

18. The system of claim 17 , wherein the stabilizing controller is used for a safe recovery of the unmanned aerial vehicle under loss of thrust of the at least one other motor.

Assignees

• Palo Alto Res Ct Inc
• Xerox Corp

Inventors

• Matei Ion
• Minhas Rajinderjeet Singh
• De Kleer Johan
• Zhenirovskyy Maksym

Classifications

• B64U10/14

  with four distinct rotor axes, e.g. quadcopters · CPC title

• G05B2219/13009

  State machine instructions · CPC title

• G05B13/042

  in which a parameter or coefficient is automatically adjusted to optimise the performance · CPC title

• G05D1/0825 Primary

  using mathematical models · CPC title

• B64C39/024

  of the remote controlled vehicle type, i.e. RPV · CPC title