Using unsupervised machine learning to produce interpretable routing rules

Patent metadata

Field	Value
Publication number	US-11568278-B2
Application number	US-202016885831-A
Country	US
Kind code	B2
Filing date	May 28, 2020
Priority date	Jun 3, 2019
Publication date	Jan 31, 2023
Grant date	Jan 31, 2023

Abstract

Embodiments of the disclosure relate to systems and methods for leveraging unsupervised machine learning to produce interpretable routing rules. In various embodiments, a training dataset comprising a plurality of data records is created. The plurality of data records includes message data comprising a plurality of messages and action data comprising a plurality of actions that correspond to the plurality of messages. A first machine learning model is trained using the training dataset. The first machine learning model as trained provides cluster data that indicates, for each data record of the plurality of data records of the training dataset, membership in a cluster of a plurality of clusters. An enhanced training dataset is created that comprises the message data from the training dataset, the action data from the training dataset, and the cluster data. A set of second machine learning models is trained using the enhanced training dataset, each respective second machine learning model of the set of second machine learning models providing a decision tree of a plurality of decision trees and corresponding to a distinct cluster of the plurality of clusters. Rules can be extracted from each decision tree of the plurality of decision trees and used as a basis for creating and transmitting alerts based on incoming messages.

First claim

What is claimed is:

1. A computer-implemented method comprising:
creating a training dataset comprising a plurality of data records, the plurality of data records including message data comprising a plurality of messages and action data comprising a plurality of actions that correspond to the plurality of messages;
training a first machine learning model using the training dataset, the first machine learning model as trained providing cluster data that indicates, for each data record of the plurality of data records of the training dataset, membership in a cluster of a plurality of clusters;
creating an enhanced training dataset comprising the message data from the training dataset, the action data from the training dataset, and the cluster data; and
training a set of second machine learning models using the enhanced training dataset, each respective second machine learning model of the set of second machine learning models providing a decision tree of a plurality of decision trees;
wherein:
the decision tree corresponds with a particular cluster of the plurality of clusters; and
the decision tree comprises a set of rules corresponding to the particular cluster of the plurality of clusters, the set of rules describing an action to be performed in response to receiving a message having a particular message data.

2. The method of claim 1 , wherein the plurality of data records includes at least one of timestamp data, integration data, tag data, and recipient data.

3. The method of claim 1 , wherein:
based on the particular message data of the received message, a particular cluster of the plurality of clusters is associated with the received message; and
the action includes programmatically generating an alert in response to the received message.

4. The method of claim 3 , wherein:
the set of rules are extracted from the particular decision tree, the set of rules including one or more rules that are optimized to determine whether the alert should be generated.

5. The method of claim 3 , wherein the received message comprises an HTTP response.

6. The method of claim 1 , further comprising:
partitioning the enhanced training dataset into a plurality of localized training datasets, each localized training dataset of the plurality of localized training datasets representing membership in a distinct cluster of the plurality of clusters;
wherein training the set of second machine learning models using the enhanced training dataset comprises training each second machine learning model of the set of second machine learning models using a distinct localized training dataset of the plurality of localized training datasets.

7. The method of claim 1 , wherein the first machine learning model comprises an unsupervised machine learning model.

8. The method of claim 1 , wherein each second machine learning model of the set of second machine learning models comprises a decision tree classification model.

9. The method of claim 1 , wherein the first machine learning model is trained using a K-Means algorithm, a Mean Shift algorithm, or Agglomerative Hierarchical Clustering algorithm.

10. The method of claim 1 , wherein the set of second machine learning models is trained using a gradient boosting algorithm or a random forest algorithm.

11. A computer system comprising:
one or more processors;
one or more memories storing instructions which, when executed by the one or more processors, cause:
creating a training dataset comprising a plurality of data records, the plurality of data records including message data comprising a plurality of messages and action data comprising a plurality of actions that correspond to the plurality of messages;
training a first machine learning model using the training dataset, the first machine learning model as trained providing cluster data that indicates, for each data record of the plurality of data records of the training dataset, membership in a cluster of a plurality of clusters;
creating an enhanced training dataset comprising the message data from the training dataset, the action data from the training dataset, and the cluster data; and
training a set of second machine learning models using the enhanced training dataset, each respective second machine learning model of the set of second machine learning models providing a decision tree of a plurality of decision trees;
wherein:
the decision tree corresponds with a particular cluster of the plurality of clusters; and
the decision tree comprising a set of rules corresponding to the particular cluster of the plurality of clusters, the set of rules describing an action to be performed in response to receiving a message having a particular message data.

12. The system of claim 11 , wherein the plurality of data records includes at least one of timestamp data, integration data, tag data, and recipient data.

13. The system of claim 11 , wherein:
in accordance with the particular message data of the received message, a particular cluster of the plurality of clusters is associated with the received message; and
the action includes programmatically generating an alert in response to the received message.

14. The system of claim 13 , wherein:
the set of rules are extracted from the particular decision tree, the set of rules including one or more rules that are optimized to determine whether the alert should be generated.

15. The system of claim 13 , wherein the received message comprises an HTTP response.

16. The system of claim 11 , further comprising:
partitioning the enhanced training dataset into a plurality of localized training datasets, each localized training dataset of the plurality of localized training datasets representing membership in a distinct cluster of the plurality of clusters;
wherein training the set of second machine learning models using the enhanced training dataset comprises training each second machine learning model of the set of second machine learning models using a distinct localized training dataset of the plurality of localized training datasets.

17. The system of claim 11 , wherein the first machine learning model comprises an unsupervised machine learning model.

18. The system of claim 11 , wherein each second machine learning model of the set of second machine learning models comprises a decision tree classification model.

19. The system of claim 11 , wherein the first machine learning model is trained using a K-Means algorithm, a Mean Shift algorithm, or Agglomerative Hierarchical Clustering algorithm.

20. The system of claim 11 , wherein the set of second machine learning models is trained using a gradient boosting algorithm or a random forest algorithm.

Assignees

• Atlassian Pty Ltd
• Atlassian Inc
• Atlassian Us Inc

Inventors

• Ramachandran Mukund
• Sarkis Rameil

Classifications

• G06N5/003

  Physics · mapped topic

• G06N20/00

  Machine learning · CPC title

• G06N5/025 Primary

  Extracting rules from data · CPC title

• G06K9/6282

  Physics · mapped topic

• G06K9/6223

  Physics · mapped topic