Artificial intelligence-based gamification for service background

Patent metadata

Field	Value
Publication number	US-2024311684-A1
Application number	US-202318184700-A
Country	US
Kind code	A1
Filing date	Mar 16, 2023
Priority date	Mar 16, 2023
Publication date	Sep 19, 2024
Grant date	—

Abstract

For AI-based recommendations in a service management system, the AI is machine trained using gamification. A model of the service management system is used in simulation to train a policy in reinforcement learning to implement strategies for improvement of KPI(s). By varying sampling of distribution of parameters of the model and/or varying the distributions of parameters used in the model, the policy learns to deal with a variety of situations using the simulations from the model. The resulting AI (machine-learned policy) is used to make recommendations for the service management system.

First claim

I (we) claim:

1 . A method for machine training an artificial intelligence to make recommendations in a service management system, the method comprising:
modeling the service management system, the modeling being a model including machines, locations of the machines, service personnel, locations of the service personnel, and service times; machine training, by a processor, the artificial intelligence with reinforcement learning, the artificial intelligence being trained to make the recommendations for service by the service personnel of the machines based on simulations from the modeling of the service system and based on rewards from a performance indicator from the service times; and storing a policy of the artificial intelligence as trained by the machine training.

2 . The method of claim 1 , wherein modeling comprises modeling with a distribution of the service times based, at least in part, on travel times, and wherein machine training comprises simulating using different samples from the distribution for the simulations and/or variance of the distribution.

3 . The method of claim 1 , wherein machine training comprises machine training with an adversarial machine-learned agent configured by past training to perturb values of parameters of the model in the simulations such that an adverse reward is received for the adversarial machine-learned agent where the artificial intelligence fails to improve the rewards for the artificial intelligence.

4 . The method of claim 1 , wherein modeling comprises representing the service management system as a random process defined over states of the machines, locations of the machines, service personnel, locations of the service personnel, the service times, service personnel shifts, and service agreement information with state transition functions defining probabilities of change in the states.

5 . The method of claim 4 , wherein modeling comprises refining the states and the state transition functions based on matching observations from the modeling of the service management system to observations from the service management system.

6 . The method of claim 5 , wherein refining comprises refining based on actions and resulting values of the performance indicator.

7 . The method of claim 1 , wherein machine training comprises estimating states, taking actions, and receiving the rewards based on the simulations.

8 . The method of claim 1 , wherein machine training comprises the reinforcement learning using perturbation of the modeling in the simulations, the perturbations being for different initial conditions and/or state transitions.

9 . The method of claim 1 , further comprising updating the model with statistical testing of the service times and/or other parameters of the model.

10 . The method of claim 1 , further comprising re-training the policy of the artificial intelligence when an actual distribution of a parameter of the model is a threshold difference from a distribution or distributions used in the machine training.

11 . The method of claim 1 , further comprising re-training the policy of the artificial intelligence based on review results for the recommendations by a service manager.

12 . A method for machine training an artificial intelligence to make recommendations in a service management system, the method comprising:
modeling the service management system, the modeling using a model with state parameters and state transition parameters for the service management system; machine training, by a processor, a policy with reinforcement learning, the policy being trained to make the recommendations based on simulations using the model, the simulations perturbing sampling of distributions and/or selection of distributions for the state parameters and/or the state transition parameters; and storing the policy as trained by the machine training.

13 . The method of claim 12 , wherein modeling comprises modeling with the state parameters comprising including machines, locations of the machines, service personnel, locations of the service personnel and the state transition parameters comprising service times and travel times, and wherein machine training comprises the reinforcement learning using rewards from performance indicators for the service times and the travel times.

14 . The method of claim 12 , wherein machine training comprises machine training with an adversarial machine-learned agent configured by past training to perturb values of the state parameters and/or the state transition parameters of the model in the simulations such that an adverse reward is received where the policy fails to improve rewards of the reinforcement learning.

15 . The method of claim 12 , further comprising updating the model with statistical testing the state transition parameters of the model and/or with replacement of values of the state parameters.

16 . A system for machine-learned model service assistance, the system comprising:
a memory configured to store a policy of the machine-learned model, the policy having been learned by reinforcement machine learning in a gamification using simulation of a service environment in combination with the reinforcement machine learning of the policy; a processor configured to input measurements from the service environment to the policy and to output a recommendation from the policy in response to the input of the measurements; and a display configured to display the recommendation from the policy.

17 . The system of claim 16 , wherein the policy was learned using rewards based on a key performance indicator of the service environment, and wherein the display of the recommendation includes an expected value of the key performance indicator given the recommendation and a period for the expected value.

18 . The system of claim 16 , wherein the policy was learned using rewards based on a key performance indicator of the service environment, and wherein the display of the recommendation includes display of a value of the key performance indicator with no change and a value of the key performance indicator when the recommendation is followed.

19 . The system of claim 16 , wherein the processor is configured to adapt the display of the recommendation with a priority based on frequency of assessment of results by a service manager.

20 . The system of claim 16 , wherein the gamification comprised use of the simulation with a model fit to the service environment, the simulations having used perturbation of distributions and/or sampling of parameters of the model as fit to the service environment and resulting changes in performance indicators as rewards in the reinforcement machine learning.

Assignees

• Siemens Healthineers Ag

Inventors

• Singh Vivek
• Comaniciu Dorin
• Kapoor Ankur
• Georgescu Bogdan
• Ullaskrishnan Poikavila
• Wendt Michael
• Biehn Neil
• Mohan Sarith

Classifications

• G06N7/01

  Probabilistic graphical models, e.g. probabilistic networks · CPC title

• G06N3/006

  based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO] · CPC title

• G06N20/00 Primary

  Machine learning · CPC title