Log analysis apparatus, log analysis method, and log analysis program

Patent metadata

Field	Value
Publication number	US-11243937-B2
Application number	US-201916568311-A
Country	US
Kind code	B2
Filing date	Sep 12, 2019
Priority date	Jan 24, 2019
Publication date	Feb 8, 2022
Grant date	Feb 8, 2022

Abstract

A log labeling apparatus is configured to include a label importance DB and a similarity DB configured to store importance information between a plurality of labels and an action set and action set information identifying a first action set for calculating a second similarity with each label of a first log unit, a similarity calculation unit configured to calculate the second similarity with each label of the first log unit on the basis of the importance information, an action set of the first log unit, and the action set of the action set information, a post processor configured to detect label candidates, and an accumulation determination unit configured to determine a second action set for calculating a second similarity of a second log unit and to store action set information on the second action set in the similarity DB.

First claim

What is claimed is:

1. A log analysis apparatus for recognizing a label indicating a log event included in log stream data which is a plurality of pieces of log data output consecutively, comprising:
a storage configured to store importance information indicating an importance of a plurality of labels for each of a plurality of action sets each including one or more actions included in log data, and store action set information identifying a first action set used for calculating a long-term similarity which is similarity in a long-term perspective, which is greater than a short-term perspective, with each label of a first log including one or more pieces of log data at a predetermined time point;
a processor coupled to a memory storing instructions that when executed by the processor configure the processor to:
calculate the long-term similarity with each of the labels of the first log based on the importance information, an action set included in the first log at the predetermined time point, and the action set identified by the action set information;
detect label candidates corresponding to the first log based on the long-term similarity;
determine a second action set which is an action set used for calculating a long-term similarity with each label of a second log at a time point next to the predetermined time point based on the long-term similarity and to store action set information identifying the second action set in the storage; and
calculate a long-term similarity with each of the labels of the first log on the basis of a first value based on importance corresponding to each action set included in the first log of the importance information on each of the labels and each action set that matches the first action set of the action set information, and a second value based on importance corresponding to each action set included in the first log of the importance information on each of the labels and each action set that does not match the first action set of the action set information.

2. The log analysis apparatus according to claim 1 , wherein
the processor is further configured to:
calculate a short-term similarity which is similarity at the short-term perspective between the first log and each of the labels based on an action set included in log data in the first log and the importance information; and
determine the second action set based on the long-term similarity and the short-term similarity.

3. The log analysis apparatus according to claim 1 , further comprising:
a display configured to display information on the detected label candidates.

4. The log analysis apparatus according to claim 1 , wherein
the processor is further configured to determine an optimal label for the log from the plurality of label candidates based on the long-term similarity with each of the labels.

5. The log analysis apparatus according to claim 1 , further comprising:
the processor being further configured to calculate the importance information based on occurrence situations of action sets in a plurality of pieces of learning log data with label attached.

6. The log analysis apparatus according to claim 5 , further comprising:
the processor being further configured to:
count occurrence times of action sets in learning log data with each label attached in the plurality of pieces of learning log data with label attached, wherein

the storage is configured to store occurrence number information indicating a number of occurrence times of each action set for each of the labels; and
the processor is further configured to calculate the importance information based on the occurrence number information.

7. The log analysis apparatus according to claim 1 , wherein
the processor is further configured to calculate the long-term similarity by multiplying the first value by the second value.

8. The log analysis apparatus according to claim 1 , further comprising:
the processor being further configured to extract an action from the log data and recognize an action set included in the first log.

9. The log analysis apparatus according to claim 8 , wherein
the storage is configured to manage an action set ID identifying an action set; and
the processor is further configured to register a new action set ID identifying an action set included in the first log in the storage when the action set ID identifying the action set included in the first log is not managed by the storage.

10. A log analysis method by a log analysis apparatus that recognizes a label indicating a log event included in log stream data which is a plurality of pieces of log data output consecutively, comprising:
storing, in a storage, importance information indicating an importance of a plurality of labels for each of a plurality of action sets each including one or more actions included in log data, and storing action set information identifying a first action set used for calculating a long-term similarity which is similarity in a long-term perspective, which is greater than a short-term perspective, with each label of a first log including one or more pieces of log data at a predetermined time point;
calculating the long-term similarity with each of the labels of the first log based on the importance information, an action set included in the first log at the predetermined time point, and the action set identified by the action set information;
detecting label candidates corresponding to the first log based on the long-term similarity; and
determining a second action set which is an action set used for calculating a long-term similarity with each label of a second log at a time point next to the predetermined time point based on the long-term similarity and storing action set information identifying the second action set in the storage,
wherein the similarity calculation is configured to calculate a long-term similarity with each of the labels of the first log on the basis of a first value based on importance corresponding to each action set included in the first log of the importance information on each of the labels and each action set that matches the first action set of the action set information, and a second value based on importance corresponding to each action set included in the first log of the importance information on each of the labels and each action set that does not match the first action set of the action set information.

11. A non-transitory computer readable medium storing a log analysis program to be executed by a computer implementing a log analysis apparatus that recognizes a label indicating a log event included in log stream data which is a plurality of pieces of log data output consecutively, wherein
in a state in which a storage has stored importance information indicating an importance of a plurality of labels for each of a plurality of action sets each including one or more actions included in log data for a plurality of labels, and has stored action set information identifying a first action set used for calculating a long-term similarity which is similarity in a long-term perspective, which is greater than a short-term perspective, with each label of a first log including one or more pieces of log data at a predetermined time point;
the log analysis program is configured to cause the computer to execute the steps of:
calculating the long-term similarity with each of the labels of the first log based on the importance information, an action set included in the first log at the predetermined time point, and the action set identified by the action set information;
detecting label candidates corresponding to the first log based on the long-term similarity;
determining a second action set whic

Assignees

• Hitachi Ltd

Inventors

• Backhus Jana
• Himura Yosuke
• Masuda Mineyoshi

Classifications

• G06F16/2379

  Updates performed during online database operations; commit processing · CPC title

• G06F16/2358 Primary

  Change logging, detection, and notification (replication G06F16/27) · CPC title

• G06F16/24568

  Data stream processing; Continuous queries · CPC title