Adaptive telemetry sampling

Patent metadata

Field	Value
Publication number	US-11516308-B1
Application number	US-202117544594-A
Country	US
Kind code	B1
Filing date	Dec 7, 2021
Priority date	Dec 7, 2021
Publication date	Nov 29, 2022
Grant date	Nov 29, 2022

Abstract

A data processing system implements adaptive telemetry sampling by obtaining first telemetry data from a plurality of telemetry data sources, analyzing the first telemetry data to identify a subset of telemetry data sources for which a reduced sampling rate may be implemented, determining a reduced sampling rate for each event type of the plurality of event types, selecting a subset of the event types for which the reduced sampling rate is to be applied, obtaining second telemetry data from the subset of telemetry data sources at the reduced sampling rate associated with each event type of the subset of event types, analyzing the second telemetry data to determine one or more estimated metric values for one or more metrics, and generating a report comprising the one or more estimated metric values and an estimated total cost saving based on an estimated cost saving associated with each event type.

First claim

What is claimed is:

1. A data processing system comprising:
a processor; and
a machine-readable medium storing executable instructions that, when executed, cause the processor to perform operations comprising:
obtaining first telemetry data from a plurality of telemetry data sources;
analyzing the first telemetry data to identify a subset of telemetry data sources of the plurality of telemetry data sources for which a reduced sampling rate may be implemented, the first telemetry data being associated with a plurality of event types;
determining a reduced sampling rate for each event type of the plurality of event types, the reduced sampling rate indicating a percentage of the subset of telemetry data sources from which telemetry data associated with that event type is to be obtained;
selecting a subset of the event types for which the reduced sampling rate is to be applied;
obtaining second telemetry data from the subset of telemetry data sources at the reduced sampling rate associated with each event type of the subset of event types;
analyzing the second telemetry data to determine one or more estimated metric values for one or more metrics, the one or more estimated metric values representing an estimate of what the one or more metric values would have been had the second telemetry data not been sampled at the reduced sampling rate; and
generating a report comprising the one or more estimated metric values and an estimated total cost saving based on an estimated cost saving associated with each event type.

2. The data processing system of claim 1 , wherein analyzing the first telemetry data to identify the subset of sampling data sources further comprises:
executing a plurality of first simulations on the first telemetry data to identify the subset of the telemetry data sources.

3. The data processing system of claim 2 , wherein determining the reduced sampling rate for each event type of the plurality of event types further comprises:
executing a plurality of second simulations on the first telemetry data to determine the reduced sampling rate for each event type of the plurality of event types.

4. The data processing system of claim 2 , wherein the machine-readable medium includes instructions configured to cause the processor to perform operations of:
receiving one or more first input parameters, each input parameter identifying an attribute of the subset of the telemetry data sources,
wherein analyzing the first telemetry data to identify the subset of telemetry data sources includes selecting telemetry data sources from the plurality of telemetry data sources that have an attribute matching the one or more first input parameter.

5. The data processing system of claim 2 , wherein the machine-readable medium includes instructions configured to cause the processor to perform operations of:
prior to obtaining the second telemetry based on the reduced sampling rate, determining that a threshold condition for obtaining the second telemetry data at the reduced sampling rate has been satisfied.

6. The data processing system of claim 5 , wherein the machine-readable medium includes instructions configured to cause the processor to perform operations of:
prior to obtaining the second telemetry data, sending instructions to first plurality of telemetry data sources of the subset of the plurality of telemetry data sources to stop providing telemetry data to reduce a volume of the second telemetry data to the reduced sampling rate.

7. The data processing system of claim 5 , wherein the machine-readable medium includes instructions configured to cause the processor to perform operations of:
determining that the threshold condition for obtaining the second telemetry data at the reduced sampling rate is no longer satisfied; and
sending instructions to the first plurality of telemetry data sources to resume providing telemetry data responsive to determining that the threshold condition is no longer satisfied.

8. The data processing system of claim 1 , wherein selecting a subset of the event types for which the reduced sampling rate is to be applied further comprises:
selecting the subset of the event types based on a minimum savings threshold associated with each event type, a maximum error rate associated with each event type, a minimum reduction in volume of telemetry data associated with an entire subset of event types.

9. A method implemented in a data processing system for adaptive telemetry sampling, the method comprising:
obtaining first telemetry data from a plurality of telemetry data sources;
analyzing the first telemetry data to identify a subset of telemetry data sources of the plurality of telemetry data sources for which a reduced sampling rate may be implemented, the first telemetry data being associated with a plurality of event types;
determining a reduced sampling rate for each event type of the plurality of event types, the reduced sampling rate indicating a percentage of the subset of telemetry data sources from which telemetry data associated with that event type is to be obtained;
selecting a subset of the event types for which the reduced sampling rate is to be applied;
obtaining second telemetry data from the subset of telemetry data sources at the reduced sampling rate associated with each event type of the subset of event types;
analyzing the second telemetry data to determine one or more estimated metric values for one or more metrics, the one or more estimated metric values representing an estimate of what the one or more metric values would have been had the second telemetry data not been sampled at the reduced sampling rate; and
generating a report comprising the one or more estimated metric values and an estimated total cost saving based on an estimated cost saving associated with each event type.

10. The method of claim 9 , wherein analyzing the first telemetry data to identify the subset of sampling data sources further comprises:
executing a plurality of first simulations on the first telemetry data to identify the subset of the telemetry data sources.

11. The method of claim 10 , wherein determining the reduced sampling rate for each event type of the plurality of event types further comprises:
executing a plurality of second simulations on the first telemetry data to determine the reduced sampling rate for each event type of the plurality of event types.

12. The method of claim 10 , further comprising:
receiving one or more first input parameters, each input parameter identifying an attribute of the subset of the telemetry data sources,
wherein analyzing the first telemetry data to identify the subset of telemetry data sources includes selecting telemetry data sources from the plurality of telemetry data sources that have an attribute matching the one or more first input parameter.

13. The method of claim 10 , further comprising:
prior to obtaining the second telemetry based on the reduced sampling rate, determining that a threshold condition for obtaining the second telemetry data at the reduced sampling rate has been satisfied.

14. The method of claim 13 , further comprising:
prior to obtaining the second telemetry data, sending instructions to first plurality of telemetry data sources of the subset of the plurality of telemetry data sources to stop providing telemetry data to reduce a volume of the second telemetry data to the reduced sampling rate.

15. The method of claim 13 , further comprising:
determining that the threshold condition for obtaining the second telemetry data at the reduced sampling rate is no longer satisfied; and
sendin

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Dubynskiy Sergiy
• Shubin Tatiana
• Ganguly Sandipan

Classifications

• G06F11/3476

  Data logging (G06F11/14, G06F11/2205 take precedence) · CPC title

• G06F11/3457

  Performance evaluation by simulation · CPC title

• G06F11/3072 Primary

  where the reporting involves data filtering, e.g. pattern matching, time or event triggered, adaptive or policy-based reporting · CPC title

• G06F11/0766

  Error or fault reporting or storing · CPC title

• G06F11/3452

  Performance evaluation by statistical analysis · CPC title