Meta-learned, evolution strategy black box optimization classifiers

Patent metadata

Field	Value
Publication number	US-12020166-B2
Application number	US-202016888136-A
Country	US
Kind code	B2
Filing date	May 29, 2020
Priority date	May 29, 2020
Publication date	Jun 25, 2024
Grant date	Jun 25, 2024

Abstract

A computational method for training a meta-learned, evolution strategy black box optimization classifier. The method includes receiving one or more training functions and one or more initial meta-learning parameters of the optimization classifier. The method further includes sampling a sampled objective function from the one or more training functions and an initial mean of the sampled function. The method also includes computing a set of T number of means by running the meta learned, evolution strategy classifier on the sampled objective function using the initial mean for T number of steps in t=1, . . . , T. The method also includes computing a loss function from the set of T number of means. The method further includes updating the one or more initial meta-learning parameters of the optimization classifier in response to a characteristic of the loss function.

First claim

What is claimed is:

1. A computational method for training a meta-learned, evolution strategy black box optimization classifier to learn an actuator control command to actuate a computer-controlled machine, the method comprising:
receiving one or more training functions and one or more initial meta-learning parameters of the meta-learned, evolution strategy black box optimization classifier;
sampling a generation of samples λ of a sampled objective function from the one or more training functions and an initial mean (m) of the sampled objective function, the generation of samples λ is z i ˜N(0, I) with the initial mean value (m) of 0 and a covariance matrix (C) scaled and shifted into x i samples using the following equation:

x
i

=

m
+

σ
⁢

C

1
2

⁢

z
i

,

where σ is a step-size;
for T number of steps in t=1, . . . , T, computing a set of T number of means by running the meta-learned, evolution strategy black box optimization classifier on the sampled objective function using the initial mean (m);
computing a loss function from the set of T number of means;
updating the one or more initial meta-learning parameters of the meta-learned, evolution strategy black box optimization classifier in response to a characteristic of the loss function to obtain an updated meta-learned, evolution strategy black box optimization classifier including a weighted combination of the generation of samples λ with a greater weight placed on the samples λ in the generation of samples λ having better characteristics of the sampled objective function;
sending input signals obtained from a sensor into the updated meta-learned, evolution strategy block box optimization classifier to obtain output signals configured to characterize a classification of the input signals;
transmitting an actuator control command to an actuator of a computer-controlled machine in response to the output signals; and
actuating the computer-controlled machine in response to the actuator control command.

2. The computational method of claim 1 , further comprising:
sending input signals obtained from a sensor into the updated meta-learned, evolution strategy black box optimization classifier to obtain output signals configured to characterize a classification of the input signals; and
transmitting an actuator control command to an actuator of a computer-controlled machine in response to the output signals.

3. The computational method of claim 1 , wherein the characteristic of the loss function is gradients of the loss function.

4. The computational method of claim 1 , wherein the characteristic of the loss function is gradient descent of the loss function.

5. The computational method of claim 1 , wherein the updating step is carried out by a deep learning optimizer.

6. The computational method of claim 1 , wherein the sampling step, the first computing step, the second computing step and the updating step are interactively performed within a loop until a stopping condition is met.

7. The computational method of claim 6 , wherein the stopping condition is convergence of the meta-learned, evolution strategy black box optimization classifier.

8. A computational method for learning an actuator control command from a meta-learned, evolution strategy black box optimization classifier to actuate a computer-controlled machine, the method comprising:
sampling and transforming a generation of samples λ into a generation of transformed samples λ, the generation of transformed samples λ is z i ˜N(0, I) with a mean value (m) of 0 and a covariance matrix (C) scaled and shifted into x i samples using the following equation:

x
i

=

m
+

σ
⁢

C

1
2

⁢

z
i

,

where σ is a step-size;
ranking the generation of transformed samples A in response to one or more function evaluations of the generation of transformed samples A to obtain a ranked generation of transformed samples λ;
updating one or more parameters of the meta-learned, evolution strategy black box optimization classifier in response to the ranked generation of transformed samples λ and one or more learned parameters trained on a set of objective functions sharing a functional characteristic with the learned, evolution strategy black box optimization classifier to obtain an updated meta-learned, evolution strategy black box optimization classifier including a weighted combination of the ranked generation of transformed samples λ with a greater weight placed on the transformed samples λ in the ranked generation of transformed samples λ having better values of the one or more functional evaluations;
sending input signals obtained from a sensor into the updated meta-learned, evolution strategy black box optimization classifier to obtain output signals configured to characterize a classification of the input signals;
transmitting an actuator control command to an actuator of a computer-controlled machine in response to the output signals; and
actuating the computer-controlled machine in response to the actuator control command.

9. The computational method of claim 8 , wherein the number of parameters include m, σ, and C.

10. The computational method of claim 9 , wherein a neural network is used to update m and σ.

11. The computational method of claim 10 , wherein the number of learned parameters represent one or both of weights and biases of the neural network.

12. The computational method of claim 8 , wherein the generation of transformed samples λ are produced from a multivariate Gaussian N(m, σC).

13. The computational method of claim 8 , wherein the one or more functional evaluations are one or more objective functional evaluations.

14. The computational method of claim 8 , wherein the ranked generation of transformed samples λ satisfies ƒ(x 1 )≤ƒ(x 2 )≤ . . . ≤ƒ(x ν ).

15. A computational method for training and using a meta-learned, evolution strategy black box optimization classifier to learn an actuator control command to actuate a computer-controlled machine, the method comprising:
receiving one or more learned parameters trained on a set of objective functions sharing a functional characteristic with the learned, evolution strategy black box optimization classifier and sampled from a generation of samples λ, the generation of samples λ is z i ˜N(0, I) with the initial mean value (m) of 0 and

Assignees

• Bosch Gmbh Robert

Inventors

• Willmott Devin T
• Daniel Christian
• Kolter Jeremy

Classifications

• G06N3/09

  Supervised learning · CPC title

• G06N3/0442

  characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] · CPC title

• G06N3/0985

  Hyperparameter optimisation; Meta-learning; Learning-to-learn · CPC title

• G06N3/044

  Recurrent networks, e.g. Hopfield networks · CPC title

• G06N3/084

  Backpropagation, e.g. using gradient descent · CPC title