Optimal online adaptive controller

Patent metadata

Field	Value
Publication number	US-9134707-B2
Application number	US-201213715116-A
Country	US
Kind code	B2
Filing date	Dec 14, 2012
Priority date	Mar 30, 2012
Publication date	Sep 15, 2015
Grant date	Sep 15, 2015

Abstract

Various embodiments are disclosed for optimal online adaptive control. One such method includes a cost function determination by a critic network coupled to the system under control. The cost function is one produces a minimum value for a cost of the system under control when applied by an action network. The method also includes a control input determination by an action network. The control input determination uses the cost function to determine a control input to apply to the system under control. The control input is one that produces the minimum value for the cost of the system under control. The method also includes simultaneously tuning respective parameters of the critic network and the action network by applying respective tuning laws that do not involve the system dynamics function f(x) for the system under test.

First claim

Therefore, the following is claimed:

1. A method of adaptively controlling a continuous-time system under control, the continuous-time system under control being described by a system dynamics function f(x), the method comprising:
in a critic network coupled to the continuous-time system under control, determining a cost function that produces a minimum value for a cost of the continuous-time system under control when applied by an action network;
in the action network also coupled to the continuous-time system under control, determining, using the cost function, a control input to apply to the continuous-time system under control that produces the minimum value for the cost of the continuous-time system under control; and
tuning respective parameters of the critic network and the action network together and continuously in time by applying respective tuning laws that do not involve the system dynamics function f(x).

2. The method of claim 1 , wherein the tuning laws use an integral reinforcement learning (IRL) form of a Bellman equation.

3. The method of claim 1 , wherein the only data obtained from the continuous-time system under test that is used during the tuning is measured input/output data from the continuous-time system under test.

4. The method of claim 1 , wherein the tuning law for the critic network is given by

W
^

.

1

=

-

a
1

⁢

Δ
⁢

⁢

ϕ
⁡

(

x
⁡

(
t
)

)

T

(

1
+

Δ
⁢

⁢

ϕ
⁡

(

x
⁡

(
t
)

)

T

⁢
Δ
⁢

⁢

ϕ
⁡

(

x
⁡

(
t
)

)

)

2

⁡

[

∫

t
-
T

t

⁢

(

Q
⁡

(
x
)

+

u
T

⁢
Ru

)

⁢

⁢

ⅆ
τ

+

Δ
⁢

⁢

ϕ
⁡

(

x
⁡

(
t
)

)

T

⁢

W
^

1

]

Assignees

• Univ Texas

Inventors

• Vamvoudakis Kyriakos
• Vrabie Draguna L
• Lewis Frank

Classifications

• G06N3/0499

  Feedforward networks · CPC title

• G06N3/092

  Reinforcement learning · CPC title

• G06N5/041

  Abduction · CPC title

• G06N99/005

  Physics · mapped topic

• G05B13/02 Primary

  electric · CPC title