Dynamically adjusting sample rates based on performance of a machine-learning based model for performing a network assurance function in a network assurance system

Patent metadata

Field	Value
Publication number	US-10691082-B2
Application number	US-201715831482-A
Country	US
Kind code	B2
Filing date	Dec 5, 2017
Priority date	Dec 5, 2017
Publication date	Jun 23, 2020
Grant date	Jun 23, 2020

Abstract

In one embodiment, a network assurance service receives data regarding a monitored network. The service analyzes the received data using a machine learning-based model, to perform a network assurance function for the monitored network. The service detects a lowered performance of the machine learning-based model when a performance metric of the machine learning-based model is below a threshold for the performance metric. When it is determined that the lowered performance of the machine-learning based model is correlated with the sample rate of the received data, the service adjusts the sample rate of the data.

First claim

What is claimed is:

1. A method comprising:
receiving, at a network assurance service, data regarding a monitored network, the received data including data provided to the network assurance service on a push basis;
interleaving, by the network assurance service, additional data regarding the monitored network received by polling one or more network elements in the monitored network on a pull basis with the data provided to the network assurance service on the push basis;
analyzing, by the network assurance service, the additional data received on the pull basis interleaved with the data provided to the network assurance service on the push basis using a machine learning-based model for performing a network assurance function for the monitored network;
detecting, by the network assurance service, a lowered performance of the machine learning-based model when a performance metric of the machine learning-based model is below a threshold for the performance metric;
determining, by the network assurance service, whether the lowered performance of the machine learning-based model is correlated with a sample rate of the received data; and
increasing, by the network assurance service, the sample rate of the received data when it is determined that the lowered performance of the machine-learning based model is correlated with the sample rate of the received data.

2. The method as in claim 1 , wherein the detecting of the lowered performance of the machine learning-based model comprises:
identifying, by the network assurance service, a drop in precision, recall, or prediction accuracy of the machine learning-based model.

3. The method as in claim 1 , wherein the network assurance function comprises at least one of: cognitive analytics of the monitored network to identify a device behavior in the monitored network, predictive analytics to predict a future condition of the monitored network, or trending analysis for capacity planning in the monitored network.

4. The method as in claim 1 , further comprising:
providing, by the network assurance service and to a user interface, an indication that the lowered performance of the machine learning-based model is correlated with the sample rate of the received data; and
receiving, at the network assurance service and from the user interface, a policy that controls the adjustment of the sample rate of the received data.

5. The method as in claim 4 , wherein the policy indicates a particular protocol to be used to adjust the sample rate of the received data.

6. The method as in claim 1 , further comprising:
disabling, by the network assurance service, the machine learning-based model until the sample rate of the received data is adjusted.

7. The method as in claim 1 , wherein the data regarding the monitored network comprises data reported by a wireless local area network controller.

8. An apparatus comprising:
one or more network interfaces to communicate with a monitored network;
a memory configured to store computer program instructions for performing a process; and
a processor coupled to the one or more network interfaces and configured to execute the computer program instructions, wherein, upon execution of the program instructions, the processor is configured to:
receive data regarding the monitored network, the received data including data provided to the network assurance service on a push basis;
interleaving, by the network assurance service, additional data regarding the monitored network received by polling one or more network elements in the monitored network on a pull basis with the data provided to the network assurance service on the push basis;
analyze the additional data received on the pull basis interleaved with the data provided to the network assurance service on the push basis using a machine learning-based model for performing a network assurance function for the monitored network;
detect a lowered performance of the machine learning-based model when a performance metric of the machine learning-based model is below a threshold for the performance metric;
determine whether the lowered performance of the machine learning-based model is correlated with a sample rate of the received data; and
increase the sample rate of the received data when it is determined that the lowered performance of the machine learning-based model is correlated with the sample rate of the received data.

9. The apparatus as in claim 8 , wherein, when the apparatus detects the lowered performance of the machine learning-based model, the processor is configured to:
identify a drop in precision, recall, or prediction accuracy of the machine learning-based model.

10. The apparatus as in claim 8 , wherein the network assurance function comprises at least one of: cognitive analytics of the monitored network to identify a device behavior in the monitored network, predictive analytics to predict a future condition of the monitored network, or trending analysis for capacity planning in the monitored network.

11. The apparatus as in claim 8 , wherein the processor is further configured to:
provide, to a user interface, an indication that the lowered performance of the machine learning-based model is correlated with the sample rate of the received data; and
receive, from the user interface, a policy that controls the adjustment of the sample rate of the received data.

12. The apparatus as in claim 11 , wherein the policy indicates a particular protocol to be used to adjust the sample rate of the received data.

13. The apparatus as in claim 8 , wherein the processor is further configured to:
disable the machine learning-based model until the sample rate of the data is adjusted.

14. The apparatus as in claim 8 , wherein the data regarding the monitored network comprises data reported by a wireless local area network controller.

15. A tangible, non-transitory, computer-readable medium storing program instructions that cause a network assurance service to execute a process comprising:
receiving, at the network assurance service, data regarding a monitored network, the received data including data provided to the network assurance service on a push basis;
interleaving, by the network assurance service, additional data regarding the monitored network received by polling one or more network elements in the monitored network on a pull basis with the data provided to the network assurance service on the push basis;
analyzing, by the network assurance service, the additional data received on the pull basis interleaved with the data provided to the network assurance service on the push basis using a machine learning-based model for performing a network assurance function for the monitored network;
detecting, by the network assurance service, a lowered performance of the machine learning-based model when a performance metric of the machine learning-based model is below a threshold for the performance metric;
determining, by the network assurance service, whether the lowered performance of the machine learning-based model is correlated with a sample rate of the received data; and
increasing, by the network assurance service, the sample rate of the received data when it is determined that that the lowered performance of the machine learning-based model is correlated with the sample rate of the received data.

16. The computer-readable medium as in claim 15 , wherein the process further comprises:
using a uniform data model to interleave the additional data received on the pull basis with the data provided to the network assurance service on the push basis.

Assignees

• Cisco Tech Inc

Inventors

• Di Pietro Andrea
• Vasseur Jean-Philippe
• Mota Javier Cruz

Classifications

• G05B13/045 Primary

  using a perturbation signal · CPC title

• H04L43/024 Primary

  by adaptive sampling · CPC title

• H04L41/16

  using machine learning or artificial intelligence · CPC title

• H04L43/16

  Threshold monitoring · CPC title

• H04L41/12

  Discovery or management of network topologies · CPC title