Predicting application quality of experience metrics using adaptive machine learned probes

Patent metadata

Field	Value
Publication number	US-10708144-B2
Application number	US-201816191075-A
Country	US
Kind code	B2
Filing date	Nov 14, 2018
Priority date	Nov 14, 2018
Publication date	Jul 7, 2020
Grant date	Jul 7, 2020

Abstract

In general, the disclosure describes techniques for evaluating application quality of experience metrics over a software-defined wide area network. For instance, a network device may receive an application data packet of a data flow. In response to receiving the application data packet, the network device determines whether a packet size of the application data packet is represented in a reference data store. In response to determining that the packet size is not represented in the reference data store, the network device predicts, based on the reference data store, flow metrics for the packet size for each of a plurality of Wide Area Network (WAN) links. The network device selects a WAN link on which to send the application data packet based on the predicted flow metrics.

First claim

The invention claimed is:

1. A method comprising:
in response to receiving, by a network device, an application data packet of a data flow, determining whether a packet size of the application data packet is represented in a reference data store, wherein the reference data store includes data indicative of a plurality of probe packet sizes and a corresponding set of one or more metrics for each of a plurality of Wide Area Network (WAN) links;
in response to determining that the packet size is not represented in the reference data store, predicting, by the network device and based on at least a portion of the one or more metrics included in the reference data store, flow metrics for the packet size for each of the plurality of WAN links;
selecting, by the network device and based on the predicted flow metrics, a WAN link of the plurality of WAN links on which to send the application data packet; and
for each of the plurality of WAN links:
sending, by the network device and to the respective WAN link, a plurality of probe packets that includes a probe packet having the packet size to obtain an updated set of one or more metrics for the respective WAN link that includes metrics for the packet size; and
updating the reference data store to include the updated set of one or more flow metrics for the each of the plurality of probe packets for the respective WAN link.

2. The method of claim 1 , further comprising:
in response to determining that the packet size is represented in the reference data store:
determining, by the network device, a WAN link of the plurality of WAN links on which to send the application data packet of the data flow based on the respective set of one or more metrics for each WAN link of the plurality of WAN links; and
sending, by the network device, the application data packet of the data flow to an application server via the WAN link of the plurality of WAN links.

3. The method of claim 2 , wherein determining the WAN link comprises:
comparing, by the network device, the respective set of one or more metrics the packet size on each WAN link of the plurality of WAN links to a set of one or more service-level agreement parameters; and
selecting, by the network device, a WAN link from the plurality of WAN links that has respective metrics that most closely matches the set of one or more service-level agreement parameters as the WAN link on which to send the application data packet of the second data flow.

4. The method of claim 1 , wherein sending the plurality of probe packets comprises:
sending, by the network device, a probe packet having the packet size to each of the plurality of WAN links to obtain respective metrics for the packet size;
for each of the probe packet sizes in the plurality of probe packet sizes, sending, by the network device, a probe packet having the respective probe packet size to each of the plurality of WAN links to obtain the corresponding set of one or more metrics for the respective WAN link-probe packet size pair, wherein each probe packet size is a unique size and is different than the packet size; and
storing, by the network device, each set of one or more metrics and the respective metrics for the packet size in the reference data store as the updated set of one or more metrics.

5. The method of claim 1 , further comprising:
while transmitting the data flow, comparing, by the network device, the updated set of one or more metrics for the WAN link to a set of one or more service-level agreement parameters; and
in response to determining that the updated set of one or more metrics for the WAN link satisfy the set of one or more service-level agreement parameters, maintaining, by the network device, the data flow on the WAN link.

6. The method of claim 5 , wherein the WAN link comprises a first WAN link, the method further comprising:
in response to determining that the updated set of one or more metrics for the first WAN link do not satisfy the set of one or more service-level agreement parameters:
determining, by the network device, a second WAN link of the plurality of WAN links with a respective updated set of one or more metrics that satisfy the set of one or more service-level agreement parameters; and
re-assigning, by the network device, the data flow to the second WAN link of the plurality of WAN links.

7. The method of claim 1 , further comprising:
determining, by the network device, a burst value for the data flow;
in response to determining that the burst flow is not represented in the reference data store, predicting, by the network device and based on the reference data store, flow metrics the burst flow for each of a plurality of WAN links;
updating, based on the burst flow, a probe packet profile stored by the network device; and
obtaining one or more flow metrics for each of the plurality of WAN links based on the updated probe packet profile.

8. The method of claim 1 , wherein predicting the flow metrics comprises predicting the flow metrics using a machine learning model using one or more of a logistic regression model, a linear discriminant analysis model, a K-Nearest neighbors model, a classification and regression trees model, a Gaussian naïve Bayes model, and a support vector machines model.

9. The method of claim 1 , wherein the flow metrics comprise one or more of jitter, round-trip time, packet drop, and bandwidth.

10. The method of claim 1 , further comprising:
removing, by the network device and from the reference data store, a particular probe packet size of the plurality of probe packet sizes if a machine learning model indicates that a likelihood of receiving a future data flow with a packet size equal to the particular probe packet size is below a threshold.

11. The method of claim 1 , wherein each WAN link of the plurality of WAN links comprises a plurality of queues, and wherein sending the plurality of probe packets comprises:
for each queue of the plurality of queues in each WAN link of the plurality of WAN links, sending, by the network device and to the respective queue, the plurality of probe packets that includes the probe packet having the packet size to obtain an updated set of one or more metrics for the respective queue that includes metrics for the packet size.

12. The method of claim 1 , further comprising updating a probe packet profile stored by the network device, wherein sending the plurality of probe packets comprises sending the plurality of probe packets based on the updated probe packet profile.

13. A network device comprising:
a memory configured to store a reference data store that includes data indicative of a plurality of probe packet sizes and a corresponding set of one or more metrics for each of a plurality of Wide Area Network (WAN) links; and
one or more processors in communication with the memory, the one or more processors configured to:
in response to receiving an application data packet of a data flow, determine whether a packet size of the application data packet is represented in the reference data store;
in response to determining that the packet size is not represented in the reference data store, predict, based on at least a portion of the one or more metrics included in the reference data store, flow metrics for the packet size for each of the plurality of WAN links;
select, based on the predicted flow metrics, a WAN link of the plurality of WAN links on which to send the application data packet; and
for each of the plurality of WAN links:
send, to the respective WAN link, a plurality of probe packets that includes a probe packet having the packet size to obtain an updated set of one or more metrics

Assignees

• Juniper Networks Inc

Inventors

• Mohan Hemachandran Karnam
• Habeeb Mohamed Abdul Kadhar Jeelany
• K S Nithyananda Vishnu

Classifications

• H04L41/40

  using virtualisation of network functions or resources, e.g. SDN or NFV entities · CPC title

• H04L41/147 Primary

  for predicting network behaviour · CPC title

• H04L43/20

  the monitoring system or the monitored elements being virtualised, abstracted or software-defined entities, e.g. SDN or NFV · CPC title

• H04L12/2854 Primary

  Wide area networks, e.g. public data networks · CPC title

• H04L43/55

  Testing of service level quality, e.g. simulating service usage · CPC title