Method and system for implementing target model configuration metadata for a log analytics system

Patent metadata

Field	Value
Publication number	US-2016294649-A1
Application number	US-201615088943-A
Country	US
Kind code	A1
Filing date	Apr 1, 2016
Priority date	Apr 3, 2015
Publication date	Oct 6, 2016
Grant date	—

Abstract

Disclosed is a system, method, and computer program product for implementing a log analytics method and system that can configure, collect, and analyze log records in an efficient manner. The log analytics system, method, and computer program product provide target-based configuration of log monitoring metadata. The log analytics system, method, and computer program product can be implemented in a cloud-based/SaaS-based architecture.

First claim

What is claimed is:

1 . A method, for configuring log monitoring to be performed in a system that implements a log analytics system, wherein the log monitoring comprises storing log data according to an association between log monitoring configuration metadata and a target, the target representing a component in a computing environment for which log data is generated, the method comprising:
generating the log monitoring configuration metadata by:
identifying a log source corresponding to a target, the log source identifying a location to monitor for the log data generated for the target;
identifying a parser for the log source;
configuring log monitoring configuration metadata comprising target-side log monitoring configuration metadata and server-side log monitoring configuration metadata, the target-side log monitoring configuration metadata including information regarding the log source and the server-side log monitoring configuration metadata including information regarding the parser for the log source;
distributing the target-side log monitoring configuration metadata to a host for the target, wherein the target is configured according to the target-side log monitoring configuration metadata;
configuring a log analytics server according to the server-side log monitoring configuration metadata;
gathering log data for the target from the host according to the target-side log monitoring configuration metadata; and performing log processing on the log data at the log analytics server according to the server-side log monitoring configuration metadata.

2 . The method of claim 1 , wherein the log monitoring configuration metadata comprises at least one of a base parser, a log rule that is associated with the target, an identification of the target, or a type identification for the target.

3 . The method of claim 1 , wherein the parser comprises a regular expression to define how to parse the log data for the target.

4 . The method of claim 1 , wherein for a single host having multiple targets, the log monitoring configuration metadata corresponds to a single file having metadata portions that pertain to the multiple targets.

5 . The method of claim 1 , wherein the log monitoring configuration metadata corresponds to an XML file.

6 . The method of claim 1 , wherein the log monitoring configuration metadata is generated, modified, or deleted upon a change to the association between the target and the log source, a change to the target, or a change to a status of a log collection agent on the host.

7 . The method of claim 1 , wherein the log monitoring configuration metadata is periodically synchronized to one or more master metadata databases.

8 . A computer readable medium having stored thereon a sequence of instructions which, when executed by a processor causes the processor to execute a method for configuring log monitoring to be performed in a system that implements a log analytics system, wherein the log monitoring comprises storing log data according to an association between log monitoring configuration metadata and a target, the target representing a component in a computing environment for which log data is generated, the method comprising:
generating the log monitoring configuration metadata by:
identifying a log source corresponding to a target, the log source identifying a location to monitor for the log data generated for the target;
identifying a parser for the log source;
configuring log monitoring configuration metadata comprising target-side log monitoring configuration metadata and server-side log monitoring configuration metadata, the target-side log monitoring configuration metadata including information regarding the log source and the server-side log monitoring configuration metadata including information regarding the parser for the log source;
distributing the target-side log monitoring configuration metadata to a host for the target, wherein the target is configured according to the target-side log monitoring configuration metadata;
configuring a log analytics server according to the server-side log monitoring configuration metadata;
gathering log data for the target from the host according to the target-side log monitoring configuration metadata; and performing log processing on the log data at the log analytics server according to the server-side log monitoring configuration metadata.

9 . The computer readable medium of claim 8 , wherein the log monitoring configuration metadata comprises at least one of a base parser, a log rule that is associated with the target, an identification of the target, or a type identification for the target.

10 . The computer readable medium of claim 8 , wherein the parser comprises a regular expression to define how to parse the log data for the target.

11 . The computer readable medium of claim 8 , wherein for a single host having multiple targets, the log monitoring configuration metadata corresponds to a single file having metadata portions that pertain to the multiple targets.

12 . The computer readable medium of claim 8 , wherein the log monitoring configuration metadata corresponds to an XML, file.

13 . The computer readable medium of claim 8 , wherein the log monitoring configuration metadata is generated, modified, or deleted upon a change to the association between the target and the log source, a change to the target, or a change to a status of a log collection agent on the host.

14 . The computer readable medium of claim 8 , wherein the log monitoring configuration metadata is periodically synchronized to one or more master metadata databases.

15 . A log analytics system implemented in a cloud or software-as-a-service architecture, the log analytics system comprising:
a configuration mechanism to configure log monitoring comprising a memory having stored thereon a sequence of instructions which, when executed by a processor, causes the processor to execute operations comprising by: identifying a log source corresponding to a target, the log source identifying a location to monitor for log data generated for the target; identifying a parser for the log source; configuring log monitoring configuration metadata comprising target-side log monitoring configuration metadata and server-side log monitoring configuration metadata, the target-side log monitoring configuration metadata including information regarding the log source and the server-side log monitoring configuration metadata including information regarding the parser for the log source; distributing the target-side log monitoring configuration metadata to a host for the target, wherein the target is configured according to the target-side log monitoring configuration metadata; configuring a log analytics pipeline at a log analytics server according to the server-side log monitoring configuration metadata; the log analytics pipeline receiving log data gathered from the host for the target according to the target-side log monitoring configuration metadata, wherein the log analytics pipeline performs log processing at the log analytics server according to the server-side log monitoring configuration metadata.

16 . The log analytics system of claim 15 , wherein the log monitoring configuration metadata comprises at least one of a base parser, a log rule that is associated with the target, an identification of the target, or a type identification for the target.

17 . The log analytics system of claim 15 , wherein the server-side log monitoring configuration metadata corresponds to a parser having a regular expression

Assignees

• Oracle Int Corp

Inventors

• Russell Jerry Paul
• Zhu Shushuai

Classifications

• G06F16/21

  Design, administration or maintenance of databases · CPC title

• G06F40/205

  Parsing · CPC title

• H04L43/04 Primary

  Processing captured monitoring data, e.g. for logfile generation · CPC title

• G06N20/00

  Machine learning · CPC title

• G06F16/353

  into predefined classes · CPC title