Probabilistic detect identification

Patent metadata

Field	Value
Publication number	US-10055276-B2
Application number	US-201615347341-A
Country	US
Kind code	B2
Filing date	Nov 9, 2016
Priority date	Nov 9, 2016
Publication date	Aug 21, 2018
Grant date	Aug 21, 2018

Abstract

A mechanism is provided for detecting one or more defects in an incoming log. One or more features are extracted from the incoming log. Utilizing the one or more features, one or more patterns, one or more pattern sequences of the one or more patterns, and one or more proximities associated with the one or more patterns contained in the incoming log are identified. For each defect model generated for the incoming log, a set of patterns in the defect model is compared to a set of patterns in each previously identified defect model in a set of previously identified defect models. Responsive to identifying a match and responsive to set of scores associated with the match meeting or exceeding a set of probability thresholds, an identification associated with the previously identified defect model along with any solution or workaround is passed to a support engineer.

First claim

What is claimed is:

1. A method, in a data processing system comprising a processor and a memory coupled to the processor, for detecting one or more defects in an incoming log, the method comprising:
responsive to receiving the incoming log, extracting, by the processor, one or more features from the incoming log by uncovering corpus/semantics, structural, and sequential features in the incoming log;
utilizing the one or more features, identifying, by the processor, one or more patterns, one or more pattern sequences of the one or more patterns, and one or more proximities associated with the one or more patterns contained in the incoming log;
generating, by the processor, one or more defect models for the incoming log using the identified one or more patterns, the one or more pattern sequences, and one or more proximities;
for each defect model in the one or more defect models for the incoming log, comparing, by the processor, a set of patterns in the defect model to a set of patterns in each previously identified defect model in a set of previously identified defect models;
responsive to matching the set of patterns in the defect model to the set of patterns in a previously identified defect model, determining, by the processor, whether a score associated with each pattern match meets or exceeds an associated probability threshold; and
responsive to each score meeting or exceeding the associated probability threshold, passing, by the processor, a defect identification associated with the previously identified defect model along with any solution and/or workaround for the previously identified defect to a support engineer so that the defect in the incoming log is addressed.

2. The method of claim 1 , further comprising:
responsive to the score failing to meet or exceed the probability threshold, proceeding, by the processor, to analyze a next unanalyzed defect model in the one or more defect models for the incoming log.

3. The method of claim 1 , further comprising:
responsive to a failure to match the defect model to a previously identified defect model, generating, by the processor, a new defect model to be added as a previously identified defect model to the set of previously identified defect models.

4. The method of claim 1 , wherein the previously the set of previously identified defect models is generated by the method comprising:
after normalization of a set of substrings in a set of training logs, extracting, by the processor, one or more features from the set of training logs by uncovering corpus/semantics, structural, and sequential features in the set of training logs;
utilizing the extracted features, identifying, by the processor, one or more patterns contained in the set of training logs, one or more pattern sequences of the one or more patterns contained in the set of training logs, and one or more proximities associated with the one or more patterns contained in the set of training logs;
generating, by the processor, one or more defect models for the set of training logs, using the identified one or more patterns contained in the set of training logs, one or more pattern sequences of the one or more patterns contained in the set of training logs, and the one or more proximities associated with the one or more patterns contained in the set of training logs; and
storing, by the processor, the one or more defect models as a set of previously identified defect models.

5. The method of claim 4 , wherein normalizing the set of substrings in the set of training logs comprises:
substituting, by the processor, substrings in the set of substrings that include changing values with placeholder tokens.

6. The method of claim 1 , wherein the features are at least one of a paired feature, a skipped feature, or a positional feature.

7. The method of claim 1 , wherein each proximity in the one or more proximities is a distance measure in units of time.

8. The method of claim 1 , further comprising:
transforming, by the processor, each defect model in the set of defect models for the incoming log to an incidence matrix.

9. A computer program product comprising a computer readable storage medium having a computer readable program stored therein, wherein the computer readable program, when executed on a computing device, causes the computing device to:
responsive to receiving the incoming log, extract one or more features from the incoming log by uncovering corpus/semantics, structural, and sequential features in the incoming log;
utilizing the one or more features, identify one or more patterns, one or more pattern sequences of the one or more patterns, and one or more proximities associated with the one or more patterns contained in the incoming log;
generate one or more defect models for the incoming log using the identified one or more patterns, the one or more pattern sequences, and one or more proximities;
for each defect model in the one or more defect models for the incoming log, compare a set of patterns in the defect model to a set of patterns in each previously identified defect model in a set of previously identified defect models;
responsive to matching the set of patterns in the defect model to the set of patterns in a previously identified defect model, determine whether a score associated with each pattern match meets or exceeds an associated probability threshold; and
responsive to each score meeting or exceeding the associated probability threshold, pass a defect identification associated with the previously identified defect model along with any solution and/or workaround for the previously identified defect to a support engineer so that the defect in the incoming log is addressed.

10. The computer program product of claim 9 , wherein the computer readable program further causes the computing device to:
responsive to the score failing to meet or exceed the probability threshold, proceed to analyze a next unanalyzed defect model in the one or more detect models for the incoming log.

11. The computer program product of claim 9 , wherein the computer readable program further causes the computing device to:
responsive to a failure to match the detect model to a previously identified defect model, generate a new defect model to be added as a previously identified defect model to the set of previously identified defect models.

12. The computer program product of claim 9 , wherein the previously the set of previously identified defect models is generated by the computer readable program further causes the computing device to:
after normalization of a set of substrings in a set of training logs, extract one or more features from the set of training logs by uncovering corpus/semantics, structural, and sequential features in the set of training logs;
utilizing the extracted features, identify one or more patterns contained in the set of training logs, one or more pattern sequences of the one or more patterns contained in the set of training logs, and one or more proximities associated with the one or more patterns contained in the set of training logs;
generate one or more defect models for the set of training logs, using the identified one or more patterns contained in the set of training logs, one or more pattern sequences of the one or more patterns contained in the set of training logs, and the one or more proximities associated with the one or more patterns contained in the set of training logs; and
store the one or more defect models as a set of previously identified defect models.

13. The computer program product of claim 12 , wherein the computer readable program to normalize the set of substrings in the set of t

Assignees

• IBM

Inventors

• Fredericks Christopher G
• Lee Tien Hiong
• Levinski Konstantin
• Qian Xin
• Tang Weng Sing
• Zhang Xiang

Classifications

• G06F2218/08

  Feature extraction · CPC title

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title

• G06F11/0751

  Error or fault detection not based on redundancy (power supply failures G06F1/30; network fault management H04L41/06) · CPC title

• G06F11/079 Primary

  Root cause analysis, i.e. error or fault diagnosis (in a hardware test environment G06F11/22; in a software test environment G06F11/36) · CPC title

• G06N5/02

  Knowledge representation; Symbolic representation · CPC title