Adaptive key performance indicator thresholds updated using training data

Patent metadata

Field	Value
Publication number	US-10776719-B2
Application number	US-201916244817-A
Country	US
Kind code	B2
Filing date	Jan 10, 2019
Priority date	Oct 9, 2014
Publication date	Sep 15, 2020
Grant date	Sep 15, 2020

Abstract

Techniques are disclosed for providing adaptive thresholding technology for Key Performance Indicators (KPIs) that are updated using training data. Adaptive thresholding technology may automatically assign new values or adjust existing values for one or more thresholds of one or more time policies. Assigning threshold values using adaptive thresholding may involve identifying training data (e.g., historical data, simulated data, or example data) for the time frames and analyzing the training data to identify variations within the data (e.g., patterns, distributions, trends). A threshold value may be determined based on the variations and may be assigned to one or more of the thresholds without additional user intervention.

First claim

What is claimed is:

1. A method comprising:
accessing information that defines one or more time frames associated with a key performance indicator (KPI), each of the time frames having a set of one or more thresholds, wherein each threshold is associated with a range of values corresponding to a particular state of the KPI, and wherein the KPI is defined by a search query that derives a value indicative of performance of a service during a time frame, the value derived from machine data pertaining to one or more entities that provide the service;
identifying, for a first time frame of the one or more time frames, training data associated with the first time frame;
determining, based on the training data, a first threshold value corresponding to a first state of the KPI and a second threshold value corresponding to a second state of the KPI, wherein the first threshold value is computed by applying a first statistical metric to the training data and the second threshold value is computed by applying a second statistical metric to the training data; and
updating, based on the first threshold value and the second threshold value, the set of one or more thresholds;
wherein the method is performed by a computer system comprising one or more processors.

2. The method of claim 1 , wherein updating the set of one or more thresholds is performed automatically based on a schedule, a frequency interval, or an event.

3. The method of claim 1 , wherein the first threshold value and the second threshold value are based on the training data that is from different time durations.

4. The method of claim 1 , wherein the training data comprises simulated data, historical data, or example data.

5. The method of claim 1 , wherein the training data comprises simulated values, historical values, or example values of the KPI.

6. The method of claim 1 , wherein the training data comprises training data that was generated by or about the one or more entities during a fixed duration of time.

7. The method of claim 1 , wherein the training data is the most current historical data.

8. The method of claim 1 , wherein the one or more time frames occur multiple times within a time cycle, wherein the time cycle is based on a daily time cycle, a weekly time cycle, or a monthly time cycle.

9. The method of claim 1 , wherein determining the first threshold value comprises determining a change to an existing threshold value, wherein the change is based on a delta value, a percentage value, or an absolute value.

10. The method of claim 1 , further comprising causing for display a graphical user interface including a presentation schedule with one or more time slots corresponding to each of the time frames, the one or more time slots having a threshold marker for each of the one or more thresholds of the set.

11. The method of claim 1 , further comprising causing for display a graphical user interface including a presentation schedule with a plurality of time slots, wherein one or more of the time slots correspond to a first time frame and have a unifying appearance to distinguish the one or more time slots from time slots corresponding to another time frame.

12. The method of claim 1 , further comprising executing the search query defining the KPI to derive a KPI value and assigning the particular state of the KPI when the KPI value is within a range bounded by the one or more thresholds.

13. The method of claim 1 , wherein the machine data is stored as time-stamped events.

14. The method of claim 1 , wherein the machine data is stored as time-stamped events, where each time-stamped event includes a portion of raw machine data.

15. The method of claim 1 , wherein the machine data is stored as time-stamped events including portions of raw machine data and is accessed using a late-binding schema.

16. The method of claim 1 , wherein the search query uses a late-binding schema to extract values indicative of the performance of the service from time-stamped events after the search query is initiated.

17. The method of claim 1 , wherein the machine data pertaining to the entity comprises heterogeneous machine data from multiple sources.

18. The method of claim 1 , wherein the machine data pertaining to the entity comprises machine data from the entity and another entity.

19. A system comprising:
a memory; and
a processing device coupled with the memory to:
access information that defines one or more time frames associated with a key performance indicator (KPI), each of the time frames having a set of one or more thresholds, wherein each threshold is associated with a range of values corresponding to a particular state of the KPI, and wherein the KPI is defined by a search query that derives a value indicative of performance of a service during a time frame, the value derived from machine data pertaining to one or more entities that provide the service;
identify, for a first time frame of the one or more time frames, training data associated with the first time frame;
determine, based on the training data, a first threshold value corresponding to a first state of the KPI and a second threshold value corresponding to a second state of the KPI, wherein the first threshold value is computed by applying a first statistical metric to the training data and the second threshold value is computed by applying a second statistical metric to the training data; and
update, based on the first threshold value and the second threshold value, the set of one or more thresholds.

20. The system of claim 19 , wherein updating the set of one or more thresholds is performed automatically based on a schedule, a frequency interval, or an event.

21. The system of claim 19 , wherein the first threshold value and the second threshold value are based on the training data that is from different time durations.

22. The system of claim 19 , wherein the training data comprises simulated data, historical data, or example data.

23. The system of claim 19 , wherein the training data comprises simulated values, historical values, or example values of the KPI.

24. The system of claim 19 , wherein the training data comprises training data that was generated by or about the one or more entities during a fixed duration of time.

25. A non-transitory computer readable storage medium encoding instructions thereon that, in response to execution by one or more processing devices, causes the processing device to perform operations comprising:
accessing information that defines one or more time frames associated with a key performance indicator (KPI), each of the time frames having a set of one or more thresholds, wherein each threshold is associated with a range of values corresponding to a particular state of the KPI, and wherein the KPI is defined by a search query that derives a value indicative of performance of a service during a time frame, the value derived from machine data pertaining to one or more entities that provide the service;
identifying, for a first time frame of the one or more time frames, training data associated with the first time frame;
determining, based on the training data, a first threshold value corresponding to a first state of the KPI and a second threshold value corresponding to a second state of the KPI, wherein the first threshold value is computed by applying a first statistical metric to the training data and the second threshold value is computed by applying a second

Assignees

• Splunk Inc

Inventors

• Maheshwari Sonal
• Sainani Manish
• Alekseyev Leonid
• Hardin Alan
• Leverich Jacob Barton
• Oliner Adam Jamison
• Reyes Brian
• Bhide Alok Anant

Classifications

• G06N20/00 Primary

  Machine learning · CPC title