Evaluating software license usage using reinforcement learning agents

Patent metadata

Field	Value
Publication number	US-11854102-B2
Application number	US-201916401610-A
Country	US
Kind code	B2
Filing date	May 2, 2019
Priority date	May 2, 2019
Publication date	Dec 26, 2023
Grant date	Dec 26, 2023

Abstract

Techniques are provided for reinforcement learning-based evaluation of software product usage. One method comprises obtaining key performance indicators indicating software product usage by a user; determining, for a predefined time window: (i) a mean and/or a median of the obtained KPIs; (ii) an amount of time that the software product was active; and (iii) an amount of interactions by the user with a user interface; evaluating possible login states of the software product using at least one reinforcement learning agent, wherein the evaluating comprises (a) observing the plurality of possible login states, including a current state comprising a current login state of the software product, and (b) obtaining an expected utility score for changing from the current login state to a different login state of the software product; and determining whether to change from the current login state to a different login state of the software product based on the expected utility score.

First claim

What is claimed is:

1. A method, comprising:
obtaining a plurality of key performance indicators indicating usage of a software product by a user of an organization;
determining, by at least one software-based agent using at least one processing device, a set of metrics for a predefined time window, the set of metrics comprising:
(i) one or more of a mean and a median of one or more of the obtained key performance indicators;
(ii) an amount of time that the software product was active on a display of the user; and
(iii) an amount of interactions by the user with a user interface in the predefined time window; and
applying, by the at least one software-based agent using the at least one processing device, a reinforcement learning process comprising:
evaluating, using at least one neural network, a plurality of possible login states of the software product based at least in part on the set of metrics for the predefined time window, wherein the evaluating comprises observing the plurality of possible login states, including a current state comprising a current login state of the software product, and obtaining an expected utility score for changing from the current login state of the software product to a different login state of the software product based at least in part on a comparison of the plurality of key performance indicators to a corresponding set of predefined thresholds, wherein the set of predefined thresholds are defined for a plurality of users of the organization;
determining whether to change from the current login state of the software product to the different login state of the software product based on the expected utility score;
triggering the change to the different login state based on a result of said determining;
prompting the user to provide feedback corresponding to the change;
in response to obtaining negative feedback, from the user, corresponding to the change, adjusting at least one of the predefined thresholds in the set of predefined thresholds based at least in part on the negative feedback; and
applying the adjusted at least one predefined threshold for the user for determining an adjusted expected utility score, wherein the adjusted at least one predefined threshold is further applied to one or more additional users of the plurality of users of the organization for determining respective expected utility scores for the one or more additional users;

wherein the method is performed by at least one processing device comprising a processor coupled to a memory.

2. The method of claim 1 , wherein the plurality of key performance indicators indicating usage by the user of the software product comprise utilization indicators for one or more of processing resources, memory resources, network resources and input/output activity.

3. The method of claim 1 , wherein the plurality of possible login states of the software product comprises a logged-in state, a logged-out state, and a state in which a potentially inactive user is prompted before logging out the potentially inactive user.

4. The method of claim 1 , wherein the expected utility score further comprises a positive reward if the user does not reconnect to the software product after the user is logged out from the software product based on the determining whether to change from the current login state of the software product and a negative reward if the user reconnects to the software product within a predefined time interval after the user was logged out from the software product based on the determining whether to change from the current login state of the software product.

5. The method of claim 4 , wherein at least one of: the positive reward and the negative reward is weighted based at least in part on a demand for licenses of the software product from a floating license pool.

6. The method of claim 1 , wherein the evaluating further comprises determining if a given session of the software product is active based on the comparison of the plurality of key performance indicators for the predefined time window to the corresponding predefined threshold.

7. The method of claim 1 , wherein the negative feedback from the user indicates that the change from the current login state of the software product to the different login state of the software product was an incorrect action.

8. The method of claim 1 , further comprising selecting between a permanent user license and a floating user license for the user based on the evaluating.

9. The method of claim 1 , wherein the user of the organization is assigned at least a first license of the software product, and wherein at least one of the one or more additional users of the organization is assigned at least a second license of the software product.

10. A computer program product, comprising a non-transitory machine-readable storage medium having encoded therein executable code of one or more software programs, wherein the one or more software programs when executed by at least one processing device perform the following steps:
obtaining a plurality of key performance indicators indicating usage of a software product by a user of an organization;
determining, by at least one software-based agent using the at least one processing device, a set of metrics for a predefined time window, the set of metrics comprising:
(i) one or more of a mean and a median of one or more of the obtained key performance indicators;
(ii) an amount of time that the software product was active on a display of the user; and
(iii) an amount of interactions by the user with a user interface in the predefined time window; and
applying, by the at least one software-based agent using the at least one processing device, a reinforcement learning process comprising:
evaluating, using at least one neural network, a plurality of possible login states of the software product based at least in part on the set of metrics for the predefined time window, wherein the evaluating comprises observing the plurality of possible login states, including a current state comprising a current login state of the software product, and obtaining an expected utility score for changing from the current login state of the software product to a different login state of the software product based at least in part on a comparison of the plurality of key performance indicators to a corresponding set of predefined thresholds, wherein the set of predefined thresholds are defined for a plurality of users of the organization;
determining whether to change from the current login state of the software product to the different login state of the software product based on the expected utility score;
triggering the change to the different login state based on a result of said determining;
prompting the user to provide feedback corresponding to the change;
in response to obtaining negative feedback, from the user, corresponding to the change, adjusting at least one of the predefined thresholds in the set of predefined thresholds based at least in part on the negative feedback; and
applying the adjusted at least one predefined threshold for the user for determining an adjusted expected utility score, wherein the adjusted at least one predefined threshold is further applied to one or more additional users of the plurality of users of the organization for determining respective expected utility scores for the one or more additional users.

11. The computer program product of claim 10 , wherein the expected utility score further comprises a positive reward if the user does not reconnect to the software product after the user is logged out from the software product based on the determining whether to change from the current login s

Assignees

• Emc Ip Holding Co Llc

Inventors

• Savir Amihai
• Natanzon Assaf
• Gaber Shiri

Classifications

• G06Q50/184 Primary

  Intellectual property management · CPC title

• G06F11/34

  Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation {; Recording or statistical evaluation of user activity, e.g. usability assessment} · CPC title

• G06F11/3409

  for performance assessment · CPC title

• G06F11/3423

  where the assessed time is active or idle time · CPC title

• G06F11/3452

  Performance evaluation by statistical analysis · CPC title