Scheduling predictive models for machine learning systems

Patent metadata

Field	Value
Publication number	US-9794145-B2
Application number	US-201514591079-A
Country	US
Kind code	B2
Filing date	Jan 7, 2015
Priority date	Jul 23, 2014
Publication date	Oct 17, 2017
Grant date	Oct 17, 2017

Abstract

In one embodiment, a device in a network monitors performance data for a first predictive model. The first predictive model is used to make proactive decisions in the network. The device maintains a supervisory model based on the monitored performance data for the first predictive model. The device identifies a time period during which the supervisory model predicts that the first predictive model will perform poorly. The device causes a switchover from the first predictive model to a second predictive model at a point in time associated with the time period, in response to identifying the time period.

First claim

What is claimed is:

1. A method comprising:
monitoring, by a device in a network, performance data for a first predictive model, wherein the first predictive model is used to make proactive decisions in the network;
maintaining, by the device, a supervisory model based on the monitored performance data for the first predictive model;
identifying, by the device, a time period during which the supervisory model predicts that the first predictive model will perform poorly at a future time;
based on the identified time period, scheduling, by the device, a switchover from the first predictive model to a second predictive model at a point in time associated with the time period;
using, by the device, change point analysis to detect a change in the performance data for the first predictive model; and
causing, by the device, a switchover from the first predictive model to a default predictive model, in response to detecting the change in the performance data using change point analysis.

2. The method as in claim 1 , further comprising:
causing, by the device, the second predictive model to be trained using input data associated with the time period and used as input to the first predictive model during the time period.

3. The method as in claim 2 , further comprising:
requesting, by the device, the input data from a node in the network that executes the first predictive model.

4. The method as in claim 1 , wherein the first predictive model predicts at least one of: a traffic pattern, a bandwidth, an amount of jitter, a retransmission rate, a packet loss rate, or an amount of delay in the network.

5. The method as in claim 1 , wherein the supervisory model uses a time base that is greater than that of the first predictive model.

6. The method as in claim 1 , wherein the supervisory model is a Hidden Markov Model.

7. The method as in claim 1 , wherein the default predictive model corresponds to one or more rules received from a user interface device.

8. The method as in claim 1 , further comprising:
providing, by the device, an alert to a user interface device, in response to detecting the change in the performance data using change point analysis.

9. An apparatus, comprising:
one or more network interfaces to communicate with a network;
a processor coupled to the one or more network interfaces and configured to execute a process; and
a memory configured to store the process executable by the processor, the process when executed operable to:
monitor performance data for a first predictive model, wherein the first predictive model is used to make proactive decisions in the network;
maintain a supervisory model based on the monitored performance data for the first predictive model;
identify a time period during which the supervisory model predicts that the first predictive model will perform poorly;
based on the identified time period, schedule a switchover from the first predictive model to a second predictive model at a point in time associated with the time period;
use change point analysis to detect a change in the performance data for the first predictive model; and
cause a switchover from the first predictive model to a default predictive model, in response to detecting the change in the performance data using change point analysis.

10. The apparatus as in claim 9 , wherein the process when executed is further operable to:
cause the second predictive model to be trained using input data associated with the time period and used as input to the first predictive model during the time period.

11. The apparatus as in claim 10 , wherein the process when executed is further operable to:
request the input data from a node in the network that executes the first predictive model.

12. The apparatus as in claim 9 , wherein the first predictive model predicts at least one of: a traffic pattern, a bandwidth, an amount of jitter, a retransmission rate, a packet loss rate, or an amount of delay in the network.

13. The apparatus as in claim 9 , wherein the supervisory model uses a time base that is greater than that of the first predictive model.

14. The apparatus as in claim 9 , wherein the supervisory model is a Hidden Markov Model.

15. The apparatus as in claim 9 , wherein the default predictive model corresponds to one or more rules received from a user interface device.

16. The apparatus as in claim 9 , wherein the process when executed is further operable to:
provide an alert to a user interface device, in response to detecting the change in the performance data using change point analysis.

17. A tangible, non-transitory, computer-readable media having software encoded thereon, the software when executed by a processor on a device in a computer network operable to:
monitor performance data for a first predictive model, wherein the first predictive model is used to make proactive decisions in the network;
maintain a supervisory model based on the monitored performance data for the first predictive model;
identify a time period during which the supervisory model predicts that the first predictive model will perform poorly at a future time;
based on the identified time period, schedule a switchover from the first predictive model to a second predictive model at a point in time associated with the time period;
use change point analysis to detect a change in the performance data for the first predictive model; and
cause a switchover from the first predictive model to a default predictive model, in response to detecting the change in the performance data using change point analysis.

18. The computer-readable media as in claim 17 , wherein the software when executed is further operable to:
cause the second predictive model to be trained using input data associated with the time period and used as input to the first predictive model during the time period.

Assignees

• Cisco Tech Inc

Inventors

• Vasseur Jean-Philippe
• Dasgupta Sukrit
• Mermoud Grégory

Classifications

• H04L41/147

  for predicting network behaviour · CPC title

• H04L41/145

  involving simulating, designing, planning or modelling of a network · CPC title

• H04L43/08 Primary

  Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters · CPC title

• H04L41/16 Primary

  using machine learning or artificial intelligence · CPC title

• H04L41/149

  for prediction of maintenance · CPC title