Machine learning-based troubleshooting of VoLTE calls

Patent metadata

Field	Value
Publication number	US-10339456-B2
Application number	US-201715586451-A
Country	US
Kind code	B2
Filing date	May 4, 2017
Priority date	Jan 5, 2017
Publication date	Jul 2, 2019
Grant date	Jul 2, 2019

Abstract

A VoLTE diagnostic engine may receive VoLTE call records of VoLTE calls that are carried by a wireless carrier network for multiple user devices. The VoLTE call records may include performance indicators and call features for the voice calls. Each call feature of a VoLTE call may represent a circumstance under which the VoLTE call is established and ended. The VoLTE diagnostic engine may apply a decision tree to the VoLTE call records to identify critical features of one or more call conditions that lead to Quality of Experience (QoE) problems for the VoLTE calls captured in the VoLTE call records. Each call condition may include a subset of the call features. Further, the VoLTE diagnostic engine may apply a classifier on the critical features to determine a root cause of a QoE problem for at least one call conditions.

First claim

What is claimed is:

1. One or more non-transitory computer-readable media storing computer-executable instructions that upon execution cause one or more processors to perform acts comprising:
receiving Voice over LTE (VoLTE) call records of VoLTE calls that are carried by a wireless carrier network for multiple user devices, the VoLTE call records including internet protocol (IP) multimedia subsystem (IMS) messages, performance indicators, and call features for the voice calls, each call feature of a VoLTE call representing a circumstance under which the VoLTE call is established and ended;
labeling each of the VoLTE calls as a call with a Quality of Experience (QoE) problem or a call without a QoE problem based on IMS message information of the VoLTE calls included in the VoLTE call records;
applying a decision tree to the VoLTE call records to identify critical features of one or more call conditions that lead to QoE problems for the VoLTE calls captured in the VoLTE call records, each call condition including a subset of the call features, wherein applying the decision tree to the VoLTE call records includes:
designating a first call feature of the subset of call features included in a corresponding first call condition to assess an effect of the first call feature on the QoE problem;
determining a first number of the VoLTE calls that experienced the QoE problem and were made with the first call condition;
determining a second total number of the VoLTE calls that were made with the first call condition;
determining a third total number of the VoLTE calls that were made with the first call condition but without the first call feature;
determining that the first call feature is a critical feature in causing the QoE problem in response to a difference between a first ratio and a second ratio being larger than zero, wherein the first ratio is a ratio between the first number and the second total number, and wherein the second ratio is a ratio between the first number and the third total number; and

applying a classifier on the critical features to determine a root cause of a corresponding QoE problem for at least one call condition.

2. The one or more non-transitory computer-readable media of claim 1 , wherein the acts further comprise generating a response for resolving the root cause of the corresponding QoE problem for the at least one call condition.

3. The one or more non-transitory computer-readable media of claim 2 , wherein the response includes a suggestion of a measure or action to be implemented by one or more of a wireless network operator, a protocol designer, or a device manufacturer.

4. The one or more non-transitory computer-readable media of claim 1 , wherein the performance indicators are collected by performance monitoring software on the user devices, the performance indicators including one or more of user interface (UI) interaction messages, IMS message transaction data, call setting data, call state data, Real-time Transport Protocol (RTP) performance data, or call handover data.

5. The one or more non-transitory computer-readable media of claim 1 , wherein the performance indicators include information collected from multiple device architectural layers of a user device, the information including Packet Data Units (PDUs) of a radio link layer and TCP/IP packets of an application layer.

6. The one or more non-transitory computer-readable media of claim 1 , wherein the call features for a VoLTE call includes one or more of an initial Reference Signal Received Power (RSRP) of the VoLTE call, an end RSRP of the VoLTE call, a start hour of the VoLTE call, an end hour of the VoLTE call, or a location where the VoLTE call is made.

7. The one or more non-transitory computer-readable media of claim 1 , wherein the QoE problems include a high unintended call drop rate, an unintentional call muting, or a call connect failure.

8. The one or more non-transitory computer-readable media of claim 1 , wherein the applying the decision tree to the VoLTE call records includes:
receiving the VoLTE call records that include one or more records for VoLTE calls with the QoE problems and one or more records for VoLTE calls without QoE problems as input for the decision tree;
determining a threshold value of a new call feature of the call features for a decision tree learning algorithm to separate the VoLTE call records associated with a new QoE problem into two subsets;
separating the VoLTE call records into the two subsets according to the threshold value to create new leaf nodes for the decision tree;
determining an additional threshold value of an additional new call features of the call features for the decision learning algorithm to separate corresponding VoLTE call records in each subset of the two subsets into two new subsets with respect to the QoE problem;
separating the corresponding VoLTE call records in each subset into two new subsets according to the additional threshold value to create additional leaf nodes for the decision tree;
repeating the determining of the additional threshold and the separating the corresponding VoLTE call records in each subset in response to a separation of the corresponding VoLTE call records in each subset leading to a smaller inequality of distribution for a pair of new subsets; and
terminating creation of new leaf nodes in the decision tree for the QoE problem in response to the separation of the corresponding VoLTE call records in each subset no longer leading to a smaller inequality of distribution for any pair of new subsets.

9. The one or more non-transitory computer-readable media of claim 8 , wherein the inequality of distribution is measured via a gini index.

10. The one or more non-transitory computer-readable media of claim 8 , wherein the applying the decision tree to the VoLTE call records further includes:
extracting decision tree leaf nodes that related to one or more corresponding call conditions for a QoE problem from the decision tree; and
calculating a confidence level for each of the one or more call conditions that indicate a certainty of a corresponding call condition causing the QoE problem.

11. The one or more non-transitory computer-readable media of claim 8 , wherein the VoLTE calls without the QoE problems are down sampled to a number that is comparable to a number of VoLTE calls with QoE problems.

12. The one or more non-transitory computer-readable media of claim 1 , wherein the applying the decision tree to the VoLTE call records includes:
determining that the first call feature is not a critical feature in causing the QoE problem in response to the difference between the first ratio and the second ratio not being larger than zero.

13. The one or more non-transitory computer-readable media of claim 12 , wherein the applying the decision tree to the VoLTE call records further includes ranking a plurality of critical features for importance to the call condition based on corresponding difference values of the plurality of critical features, the plurality of features including the critical feature and one or more other critical features.

14. The one or more non-transitory computer-readable media of claim 1 , wherein the applying the classifier on the critical features includes:
receiving multiple lists of critical features that includes a corresponding list of one or more critical features for each call condition of a plurality of call conditions as ranked according to importance for causing a QoE problem;
labeling a subset of the VoLTE call records as having congestion issues or coverage issues based on the multiple lists of critical features to generat

Assignees

• T Mobile Usa Inc

Inventors

• Hui Jie
• Jia Yunhan
• Guo Yihua
• Yoon Alex
• Tran Antoine
• Sontineni Kranthi
• Mao Z Morley

Classifications

• G06N5/01

  Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound · CPC title

• H04M2207/185

  wireless packet-switched · CPC title

• H04L65/1016

  IP multimedia subsystem [IMS] · CPC title

• H04L65/80

  Responding to QoS · CPC title

• G06N5/04 Primary

  Inference or reasoning models · CPC title