Method and system for implementing machine learning classifications

Patent metadata

Field	Value
Publication number	US-2016292592-A1
Application number	US-201615089226-A
Country	US
Kind code	A1
Filing date	Apr 1, 2016
Priority date	Apr 3, 2015
Publication date	Oct 6, 2016
Grant date	—

Abstract

Disclosed is a system, method, and computer program product for implementing a log analytics method and system that can configure, collect, and analyze log records in an efficient manner. Machine learning-based classification can be performed to classify logs. This approach is used to group logs automatically using a machine learning infrastructure.

First claim

What is claimed is:

1 . A method implemented with a processor, comprising:
identifying a log to analyze; generating a first type of vector data for the log; generating at least a second type of vector data for the log; performing a first similarity comparison between the first vector data and a first model of known log data to generate a first set of comparison results; performing at least a second similarity comparison between at least the second vector data and at least a second model of the known log data to generate at least a second set of comparison results; automatically classifying the log as a log type as fitting the first model and second model of known log data based at least in part on the first set of comparison results from the first similarity comparison and at least the second set of comparison results from the second similarity comparison; and parsing the log to store log items comprising fields and values based at least in part on identification of the log type.

2 . The method of claim 1 , wherein the first type of vector data corresponds to a distribution vector based upon distribution or frequency of terms within the log, and the second type of vector data corresponds to a token vector based upon identified tokens within the log.

3 . The method of claim 1 , wherein the first and second models comprise one or more centroids that correlate to known log types, and the first and second similarity comparisons identify a distance from a vector to a centroid.

4 . The method of claim 3 , wherein at least one log type corresponds to a plurality of centroids.

5 . The method of claim 1 , wherein weightings are applied to the first set of comparison results or the second set of comparison results to automatically classify the logs.

6 . The method of claim 1 , wherein a user interface is provided on a display device to identify a percentage probability of the log being classified as a given log type.

7 . The method of claim 1 , wherein common portions and variable portions are identified in the log, and at least one of the first model or the second model is generated with vectorization of the common portions and not vectorization of the variable portions.

8 . The method of claim 1 , wherein common portions, variable portions, and field types are identified in the log, and at least one of the first model or the second model is generated with vectorization of the common portions and field types, but not vectorization of the variable portions.

9 . A computer readable medium having stored thereon a sequence of instructions which, when executed by a processor causes the processor to execute a method, the method comprising:
identifying a log to analyze; generating a first type of vector data for the log; generating at least a second type of vector data for the log; performing a first similarity comparison between the first vector data and a first model of known log data to generate a first set of comparison results; performing at least a second similarity comparison between at least the second vector data and at least a second model of the known log data to generate at least a second set of comparison results; automatically classifying the log as a log type as fitting the first model and second model of known log data based at least in part on the first set of comparison results from the first similarity comparison and at least the second set of comparison results from the second similarity comparison; and parsing the log to store log items comprising fields and values based at least in part on identification of the log type.

10 . The computer readable medium of claim 9 , wherein the first type of vector data corresponds to a distribution vector based upon distribution or frequency of terms within the log, and the second type of vector data corresponds to a token vector based upon identified tokens within the log.

11 . The computer readable medium of claim 9 , wherein the first and second models comprise one or more centroids that correlate to known log types, and the first and second similarity comparisons identify a distance from a vector to a centroid.

12 . The computer readable medium of claim 11 , wherein at least one log type corresponds to a plurality of centroids.

13 . The computer readable medium of claim 9 , wherein weightings are applied to the first set of comparison results or the second set of comparison results to automatically classify the logs.

14 . The computer readable medium of claim 9 , wherein a user interface is provided on a display device to identify a percentage probability of the log being classified as a given log type.

15 . The computer readable medium of claim 9 , wherein common portions and variable portions are identified in the log, and at least one of the first model or the second model is generated with vectorization of the common portions and not vectorization of the variable portions.

16 . The computer readable medium of claim 9 , wherein common portions, variable portions, and field types are identified in the log, and at least one of the first model or the second model is generated with vectorization of the common portions and field types, but not vectorization of the variable portions.

17 . A system, comprising:
a processor; a memory having stored thereon a sequence of instructions which, when executed by a processor causes the processor to execute operations comprising: identifying a log to analyze; generating a first type of vector data for the log; generating at least a second type of vector data for the log; performing a first similarity comparison between the first vector data and a first model of known log data to generate a first set of comparison results; performing at least a second similarity comparison between at least the second vector data and at least a second model of the known log data to generate at least a second set of comparison results; automatically classifying the log as a log type as fitting the first model and second model of known log data based at least in part on the first set of comparison results from the first similarity comparison and at least the second set of comparison results from the second similarity comparison; and parsing the log to store log items comprising fields and values based at least in part on identification of the log type

18 . The system of claim 17 , wherein the first type of vector data corresponds to a distribution vector based upon distribution or frequency of terms within the log, and the second type of vector data corresponds to a token vector based upon identified tokens within the log.

19 . The system of claim 17 , wherein the first and second models comprise one or more centroids that correlate to known log types, and the first and second similarity comparisons identify a distance from a vector to a centroid.

20 . The system of claim 17 , wherein weightings are applied to the first set of comparison results or the second set of comparison results to automatically classify the logs.

21 . The system of claim 17 , wherein common portions and variable portions are identified in the log, and at least one of the first model or the second model is generated with vectorization of the common portions and not vectorization of the variable portions.

Assignees

• Oracle Int Corp

Inventors

• Patthak Anindya Chandra
• Ferrar Gregory Michael

Classifications

• G06F11/3086

  where the reporting involves the use of self describing data formats, i.e. metadata, markup languages, human readable formats · CPC title

• G06F11/3006

  where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems (multiprogramming arrangements G06F9/46; allocation of resources G06F9/50) · CPC title

• H04L41/145

  involving simulating, designing, planning or modelling of a network · CPC title

• G06F16/21

  Design, administration or maintenance of databases · CPC title

• G06N20/00

  Machine learning · CPC title