Data source modeling to detect disruptive changes in data dynamics

Patent metadata

Field	Value
Publication number	US-10623273-B2
Application number	US-201815860017-A
Country	US
Kind code	B2
Filing date	Jan 2, 2018
Priority date	Jan 2, 2018
Publication date	Apr 14, 2020
Grant date	Apr 14, 2020

Abstract

In one embodiment, a network assurance service receives, from a reporting entity, data regarding a monitored network for input to a machine learning-based analyzer of the network assurance service. The service forms a reporting entity model of the reporting entity, based on at least a portion of the data received from the reporting entity. The service identifies a behavioral change of the reporting entity by comparing a sample of the data received from the reporting entity to the reporting entity model. The service correlates the behavioral change of the reporting entity to a change made to the reporting entity. The service causes performance of a mitigation action, to prevent the behavioral change from affecting operation of the machine learning-based analyzer.

First claim

What is claimed is:

1. A method comprising:
receiving, at a network assurance service and from a reporting entity, data regarding a monitored network for input to a machine learning-based analyzer of the network assurance service;
forming, by the service, a reporting entity model of the reporting entity, based on at least a portion of the data received from the reporting entity;
identifying, by the service, a behavioral change of the reporting entity by comparing a sample of the data received from the reporting entity to the reporting entity model, wherein the behavioral change is a change in dynamics of the data received from the reporting entity;
correlating, by the service, the behavioral change of the reporting entity to a change made to the reporting entity; and
causing, by the service, performance of a mitigation action, to prevent the behavioral change from affecting operation of the machine learning-based analyzer.

2. The method as in claim 1 , wherein the behavioral change corresponds to a change in a semantics, reporting frequency, or data type associated with the data received from the reporting entity.

3. The method as in claim 1 , wherein the reporting entity model comprises at least one of: a Mixtures of Gaussians model, an autoregressive integrated moving average (ARIMA) model, a Hidden Markov model, or a Recurrent Neural Network (RNN)-based model.

4. The method as in claim 1 , wherein correlating the behavioral change of the reporting entity to a change made to the reporting entity comprises:
receiving, at the service, an indication of the change made to the reporting entity from a network controller or network management entity in the monitored network.

5. The method as in claim 1 , wherein the change made to the reporting entity comprises at least one of: a software or firmware version change applied to the reporting entity, a configuration change made to the reporting entity, or a routing change made to the monitored network that affects the reporting entity.

6. The method as in claim 1 , further comprising:
providing, by the service and to a user interface, an indication of the behavioral change of the reporting entity and the change made to the reporting entity.

7. The method as in claim 1 , wherein the mitigation action comprises: reverting the reporting entity to a previously installed version of software or firmware or reverting the reporting entity to a prior configuration.

8. The method as in claim 1 , wherein the mitigation action comprises:
filtering, by the service, the data from the reporting entity from input to the machine learning-based analyzer of the network assurance service.

9. The method as in claim 1 , wherein the mitigation action comprises:
performing, by the service, a data transformation to the data from the reporting entity, prior to input to the machine learning-based analyzer of the network assurance service.

10. An apparatus, comprising:
one or more network interfaces to communicate with a network;
a processor coupled to the network interfaces and configured to execute one or more processes; and
a memory configured to store a process executable by the processor, the process when executed configured to:
receive, from a reporting entity, data regarding a monitored network for input to a machine learning-based analyzer of a network assurance service;
form a reporting entity model of the reporting entity, based on at least a portion of the data received from the reporting entity;
identify a behavioral change of the reporting entity by comparing a sample of the data received from the reporting entity to the reporting entity model, wherein the behavioral change is a change in dynamics of the data received from the reporting entity;
correlate the behavioral change of the reporting entity to a change made to the reporting entity; and
cause performance of a mitigation action, to prevent the behavioral change from affecting operation of the machine learning-based analyzer.

11. The apparatus as in claim 10 , wherein the behavioral change corresponds to a change in semantics, reporting frequency, or data type associated with the data received from the reporting entity.

12. The apparatus as in claim 10 , wherein the reporting entity model comprises at least one of: a Mixtures of Gaussians model, an autoregressive integrated moving average (ARIMA) model, a Hidden Markov model, or a Recurrent Neural Network (RNN)-based model.

13. The apparatus as in claim 10 , wherein the apparatus correlates the behavioral change of the reporting entity to a change made to the reporting entity by:
receiving an indication of the change made to the reporting entity from a network controller in the monitored network.

14. The apparatus as in claim 10 , wherein the change made to the reporting entity comprises at least one of: a software or firmware version change applied to the reporting entity, a configuration change made to the reporting entity, or a routing change made to the monitored network that affects the reporting entity.

15. The apparatus as in claim 10 , wherein the process when executed is further configured to:
provide, to a user interface, an indication of the behavioral change of the reporting entity and the change made to the reporting entity.

16. The apparatus as in claim 10 , wherein the mitigation action comprises: reverting the reporting entity to a previously installed version of software or firmware or reverting the reporting entity to a prior configuration.

17. The apparatus as in claim 10 , wherein the mitigation action comprises: filtering, by the service, the data from the reporting entity from input to the machine learning-based analyzer of the network assurance service.

18. The apparatus as in claim 10 , wherein the mitigation action comprises:
performing, by the service, a data transformation to the data from the reporting entity, prior to input to the machine learning-based analyzer of the network assurance service.

19. A tangible, non-transitory, computer-readable medium storing program instructions that cause a network assurance service to execute a process comprising:
receiving, at the network assurance service and from a reporting entity, data regarding a monitored network for input to a machine learning-based analyzer of the network assurance service;
forming, by the service, a reporting entity model of the reporting entity, based on at least a portion of the data received from the reporting entity;
identifying, by the service, a behavioral change of the reporting entity by comparing a sample of the data received from the reporting entity to the reporting entity model, wherein the behavioral change is a change in dynamics of the data received from the reporting entity;
correlating, by the service, the behavioral change of the reporting entity to a change made to the reporting entity; and
causing, by the service, performance of a mitigation action, to prevent the behavioral change from affecting operation of the machine learning-based analyzer.

20. The computer-readable medium as in claim 19 , wherein the behavioral change of the reporting entity is specified via a user interface.

Assignees

• Cisco Tech Inc

Inventors

• Di Pietro Andrea
• Vasseur Jean-Philippe
• Mota Javier Cruz

Classifications

• G06N3/08

  Learning methods · CPC title

• H04L43/022

  by sampling · CPC title

• H04L41/0213

  Standardised network management protocols, e.g. simple network management protocol [SNMP] · CPC title

• H04L43/14

  using software, i.e. software packages (network security related monitoring H04L63/1408) · CPC title

• H04L41/16

  using machine learning or artificial intelligence · CPC title