Implementing artificial intelligence agents to perform machine learning tasks using predictive analytics to leverage ensemble policies for maximizing long-term returns

Patent metadata

Field	Value
Publication number	US-11188797-B2
Application number	US-201816175048-A
Country	US
Kind code	B2
Filing date	Oct 30, 2018
Priority date	Oct 30, 2018
Publication date	Nov 30, 2021
Grant date	Nov 30, 2021

Abstract

A method for implementing artificial intelligence agents to perform machine learning tasks using predictive analytics to leverage ensemble policies for maximizing long-term returns includes obtaining a set of inputs including a set of ensemble policies and a meta-policy parameter, selecting an action for execution within the system environment using a meta-policy function determined based in part on the set of ensemble policies and the meta-policy function parameter, causing the artificial intelligence agent to execute the selected action within the system environment, and updating the meta-policy function parameter based on the execution of the selected action.

First claim

What is claimed:

1. A system for implementing artificial intelligence agents to perform machine learning tasks using predictive analytics to leverage ensemble policies for maximizing long-term returns, comprising:
an artificial intelligence agent;
a memory device for storing program code; and
at least one processor device operatively coupled to the memory device and configured to execute program code stored on the memory device to:
obtain a set of inputs including a set of ensemble policies, a meta-policy parameter, and initial sufficient statistics;
select an action for execution within the system environment using a meta-policy function determined based in part on the set of ensemble policies and the meta-policy parameter;
cause the artificial intelligence agent to execute the selected action within the system environment;
execute program code stored on the memory device to update the initial sufficient statistics based on the execution of the selected action; and
update the meta-policy parameter and an initial baseline based on the execution of the selected action.

2. The system of claim 1 , wherein each policy in the set of ensemble polices has an internal state that can be represented as a state vector having a fixed length.

3. The system of claim 1 , wherein the set of inputs further includes state vectors, state value, the initial baseline, a statistics forgetting rate, a baseline learning rate, a meta-policy parameter learning rate, and a training length.

4. The system of claim 1 , wherein the at least one processor device is further configured to execute program code stored on the memory device to:
obtain an observation and reward in a system environment; and
update a history based on the observation and the reward.

5. The system of claim 4 , wherein the meta-policy function is further based on the updated history.

6. The system of claim 1 , wherein the at least one processor device is further configured to execute program code stored on the memory device to return the updated meta-policy parameter as output for selecting a future action for execution within the system environment.

7. A computer-implemented method for implementing artificial intelligence agents to perform machine learning tasks using predictive analytics to leverage ensemble policies for maximizing long-term returns, comprising:
obtaining a set of inputs including a set of ensemble policies, a meta-policy parameter, and initial sufficient statistics;
selecting an action for execution within the system environment using a meta-policy function determined based in part on the set of ensemble policies and the meta-policy parameter;
causing an artificial intelligence agent to execute the selected action within the system environment;
execute program code stored on a memory device to update the initial sufficient statistics based on the execution of the selected action; and
updating the meta-policy parameter and an initial baseline based on the execution of the selected action.

8. The method of claim 7 , wherein each policy in the set of ensemble polices has an internal state that can be represented as a state vector having a fixed length.

9. The method of claim 7 , wherein the set of inputs further includes state vectors, state value, the initial baseline, a statistics forgetting rate, a baseline learning rate, a meta-policy parameter learning rate, and a training length.

10. The method of claim 7 , wherein the at least one processor device is further configured to execute program code stored on the memory device to:
obtain an observation and reward in a system environment; and
update a history based on the observation and the reward.

11. The method of claim 10 , wherein the meta-policy function is further based on the updated history.

12. The method of claim 7 , wherein the at least one processor device is further configured to execute program code stored on the memory device to return the updated meta-policy function as output for selecting a future action for execution within the system environment.

13. A computer program product comprising a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to perform a method for implementing artificial intelligence agents to perform machine learning tasks using predictive analytics to leverage ensemble policies for maximizing long-term returns, the method performed by the computer comprising:
obtaining a set of inputs including a set of ensemble policies, a meta-policy parameter, and initial sufficient statistics;
selecting an action for execution within the system environment using a meta-policy function determined based in part on the set of ensemble policies and the meta-policy parameter;
causing an artificial intelligence agent to execute the selected action within the system environment;
execute program code stored on a memory device to update the initial sufficient statistics based on the execution of the selected action; and
updating the meta-policy parameter and an initial baseline based on the execution of the selected action.

14. The computer program product of claim 13 , wherein each policy in the set of ensemble polices has an internal state that can be represented as a state vector having a fixed length.

15. The computer program product of claim 13 , wherein the set of inputs further includes state vectors, state value, the initial baseline, a statistics forgetting rate, a baseline learning rate, a meta-policy parameter learning rate, and a training length.

16. The computer program product of claim 13 , wherein the at least one processor device is further configured to execute program code stored on the memory device to:
obtain an observation and reward in a system environment; and
update a history based on the observation and the reward;
wherein the meta-policy parameter is further based on the updated history.

17. The computer program product of claim 13 , wherein the at least one processor device is further configured to execute program code stored on the memory device to return the updated meta-policy parameter as output for selecting a future action for execution within the system environment.

Assignees

• IBM

Inventors

• Morimura Tetsuro
• Yanagisawa Hiroki
• Takase Toshiro
• Koseki Akira

Classifications

• G06N3/006 Primary

  based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO] · CPC title

• G06F18/2185

  the supervisor being an automated module, e.g. intelligent oracle · CPC title

• G06F18/295

  Markov models or related models, e.g. semi-Markov models; Markov random fields; Networks embedding Markov models · CPC title

• G06F18/2148

  characterised by the process organisation or structure, e.g. boosting cascade · CPC title

• G06F18/2193

  based on specific statistical tests · CPC title