Online optimal control under constraints

Patent metadata

Field	Value
Publication number	US-12554247-B2
Application number	US-202117386589-A
Country	US
Kind code	B2
Filing date	Jul 28, 2021
Priority date	Jul 28, 2021
Publication date	Feb 17, 2026
Grant date	Feb 17, 2026

Abstract

Embodiments of the present invention provide computer-implemented methods, computer program products and computer systems. Embodiments of the present invention can identify a plurality of constraints on states of data and actions of data associated with a data model. Embodiments of the present invention can then identify constraints on safety policy parameters associated with a computing device. Embodiments of the present invention can then convert the identified constraints into a uniform domain syntax that considers coupled and decoupled constraints and introduce buffer data within the converted constraints, wherein the buffer data filters outlier constraints within the plurality of constraints. Embodiments of the present invention can then dynamically generate optimal safety policies associated with the computing device based on the remaining constraints.

First claim

What is claimed is:

1 . A computer-implemented method comprising:
identifying a plurality of constraints on states of data and actions of data associated with a data model; identifying constraints on safety policy parameters associated with a computing device; converting the identified constraints into a uniform domain syntax that considers coupled and decoupled constraints; introducing buffer data within the converted constraints, wherein the buffer data filters outlier constraints within the plurality of constraints; and dynamically generating optimal control policies and optimal safety policies associated with the computing device based on the remaining constraints, wherein the optimal control policies simultaneously minimizes the sum of adversarial varying costs and satisfy constraints despite disturbances by computing a feasibly policy class, computing a safety policy class, and utilizing an optimization tool.

2 . The computer-implemented method of claim 1 , wherein identifying the plurality of constraints on states of data and actions of data comprises:
performing a query on an external data source for optimal performance for each state of data; storing at least one result of the performed query within a database on the computing device; and introducing approximate states and actions that correspond with at least one historical policy associated with the state and action of the data.

3 . The computer-implemented method of claim 1 , wherein identifying constraints on safety policy parameters comprises receiving preferences of a user associated with the safety policy parameters encompassing the computing device.

4 . The computer-implemented method of claim 1 , wherein converting the identified constraints comprises:
using the identified constraints as input data; generating a safety policy configuration via a safety translation algorithm; and imposing constraints on the approximate states and actions of data that are dynamically translated as constraints on the at least one historical policy using the safety translation algorithm and a policy configuration data model.

5 . The computer-implemented method of claim 1 , wherein introducing buffer data comprises:
adding buffer data to the generated safety policy configuration, wherein the buffer data filters outlier constraints within the plurality of constraints and reduce the range of the identified constraints associated with the state and action of data and the safety policy parameters of the computing device.

6 . The computer-implemented method of claim 1 , wherein dynamically generating optimal safety policies comprises:
analyzing the generated safety policy configuration; generating a parallel subroutine to the generated safety policy configuration; predicting the cost output of the estimated data model; predicting the utility output of the estimated data model; and validating that the generated safety policy configuration does not meet or exceed the upper bounds of the identified constraints associated with the computing device.

7 . The computer-implemented method of claim 1 , further comprising:
identifying the constraints associated with the approximate state and action data; identifying a temporal-coupled constraints associated with the policy parameters of the computing device; and determining the variations associated with the temporal-decoupled online policies.

8 . A computer program product comprising:
one or more computer readable storage media and program instructions stored on the one or more computer readable storage media, the program instructions comprising:
program instructions to identify a plurality of constraints on states of data and actions of data associated with a data model;
program instructions to identify constraints on safety policy parameters associated with a computing device;
program instructions to convert the identified constraints into a uniform domain syntax that considers coupled and decoupled constraints;
program instructions to introduce buffer data within the converted constraints, wherein the buffer data filters outlier constraints within the plurality of constraints; and
program instructions to dynamically generate optimal control policies and optimal safety policies associated with the computing device based on the remaining constraints, wherein the optimal control policies simultaneously minimizes the sum of adversarial varying costs and satisfy constraints despite disturbances by computing a feasibly policy class, computing a safety policy class, and utilizing an optimization tool.

9 . The computer program product of claim 8 , wherein the program instructions to identify the plurality of constraints on states of data and actions of data comprise:
program instructions to perform a query on an external data source for optimal performance for each state of data; program instructions to store at least one result of the performed query within a database on the computing device; and program instructions to introduce approximate states and actions that correspond with at least one historical policy associated with the state and action of the data.

10 . The computer program product of claim 8 , wherein the program instructions to identify constraints on safety policy parameters comprise:
program instructions to receive preferences of a user associated with the safety policy parameters encompassing the computing device.

11 . The computer program product of claim 8 , wherein the program instructions to convert the identified constraints comprises:
program instructions to use the identified constraints as input data; program instructions to generate a safety policy configuration via a safety translation algorithm; and program instructions to impose constraints on the approximate states and actions of data that are dynamically translated as constraints on the at least one historical policy using the safety translation algorithm and a policy configuration data model.

12 . The computer program product of claim 8 , wherein the program instructions to introduce buffer data comprises:
program instructions to add buffer data to the generated safety policy configuration, wherein the buffer data filters outlier constraints within the plurality of constraints and reduce the range of the identified constraints associated with the state and action of data and the safety policy parameters of the computing device.

13 . The computer program product of claim 8 , wherein the program instructions to dynamically generate optimal safety policies comprise:
program instructions to analyze the generated safety policy configuration; program instructions to generate a parallel subroutine to the generated safety policy configuration; program instructions to predict the cost output of the estimated data model; program instructions to predict the utility output of the estimated data model; and program instructions to validate that the generated safety policy configuration does not meet or exceed the upper bounds of the identified constraints associated with the computing device.

14 . The computer-implemented method of claim 8 , wherein the program instructions stored on the one or more computer readable storage media further comprise:
program instructions to identify the constraints associated with the approximate state and action data; program instructions to identify a temporal-coupled constraints associated with the policy parameters of the computing device; and program instructions to determine the variations associated with the temporal-decoupled online policies.

Assignees

• IBM

Inventors

• Li Yingying
• Das Subhro

Classifications

• G06N20/00

  Machine learning · CPC title

• G06N5/00

  Computing arrangements using knowledge-based models · CPC title

• G06F16/903

  Querying (for retrieval from the web G06F16/953) · CPC title

• G05B2219/50333

  Temperature · CPC title

• G05B2219/2614

  HVAC, heating, ventillation, climate control · CPC title