Automatic generation of regular expression based on log line data

Patent metadata

Field	Value
Publication number	US-10775751-B2
Application number	US-201615011040-A
Country	US
Kind code	B2
Filing date	Jan 29, 2016
Priority date	Jan 29, 2016
Publication date	Sep 15, 2020
Grant date	Sep 15, 2020

Abstract

In one aspect, a regular expression is automatically generated based on user input for fields that are desired to be extracted from log lines. The input may be received by user through an interface provided by a machine such as a controller. The input may identify one or more fields within a log line that should be extracted. Multiple instances of potential regular expression portions may be generated based on the user input, and different portions are combined together to determine if they achieve the desired extraction. Once a complete regular expression is generated based on user input, a user may provide additional input to identify examples or counterexamples of log line fields that satisfy or don't satisfy the user's intended extraction.

First claim

What is claimed is:

1. A method for automatically generating a regular expression from log line data, comprising:
identifying, by an application on a server, a portion of a log line in the log line data;
automatically generating, by the application on the server, multiple instances of multiple portions of a regular expression based on the identified portion, the multiple instances of multiple portions of the regular expression including:
at least one regular expression generated that represents a field within the portion of the log line, and
at least one regular expression that represents a gap within the portion of the log line, where the at least one regular expression that represents the gap is generated by identifying starting patterns and ending patterns of the gap, a particular pattern for a string immediately after the gap, or a count of delimiter occurrences defining the gap;

automatically generating, by the application on the server, a complete regular expression from one or more of the generated multiple instances; and
retrieving, by the application on the server, subsequent log lines based on the automatically generated complete regular expression.

2. The method of claim 1 , wherein identifying includes receiving user input indicative of the portion of a log line.

3. The method of claim 2 , wherein the received user input identifies a field in the indicated portion of the log line.

4. The method of claim 1 , wherein generating multiple instances includes:
generating at least one of the multiple instances of the multiple portions of the regular expressions for each of a plurality of gaps in the log line based on the identified portion; and
generating at least one of the multiple instances of the multiple portions of the regular expressions for each of a plurality of fields in the log line based on the identified portion.

5. The method of claim 4 , wherein automatically generating a regular expression includes generating a regular expression based on multiple instances of the regular expression portion for a single gap and multiple instances of the regular expression portion for a single field.

6. The method of claim 5 , further comprising receiving user input to modify the automatically generated regular expression based on receipt of a subsequent user input.

7. The method of claim 1 , further comprising receiving user input to modify the automatically generated regular expression based on receipt of a subsequent user input.

8. A non-transitory computer readable storage medium having embodied thereon a program, the program being executable by a processor to perform a method for automatically generating a regular expression from log line data, the method comprising:
identifying, by an application on a server, a portion of a log line in the log line data;
automatically generating, by the application on the server, multiple instances of multiple portions of a regular expression based on the identified portion, the multiple instances of multiple portions of the regular expression including:
at least one regular expression generated that represents a field within the portion of the log line, and
at least one regular expression that represents a gap within the portion of the log line, where the at least one regular expression that represents the gap is generated by identifying starting patterns and ending patterns of the gap, a particular pattern for a string immediately after the gap, or a count of delimiter occurrences defining the gap;

automatically generating, by the application on the server, a complete regular expression from one or more of the generated multiple instances; and
retrieving, by the application on the server, subsequent log lines based on the automatically generated complete regular expression.

9. The non-transitory computer readable storage medium of claim 8 , wherein identifying includes receiving user input indicative of the portion of a log line.

10. The non-transitory computer readable storage medium of claim 9 , wherein the received user input identifies a field in the indicated portion of the log line.

11. The non-transitory computer readable storage medium of claim 8 , wherein generating multiple instances includes:
generating at least one of the multiple instances of the multiple portions of the regular expressions for each of a plurality of gaps in the log line based on the identified portion; and
generating at least one of the multiple instances of the multiple portions of the regular expressions for each of a plurality of fields in the log line based on the identified portion.

12. The non-transitory computer readable storage medium of claim 11 , wherein automatically generating a regular expression includes generating a regular expression based on multiple instances of the regular expression portion for a single gap and multiple instances of the regular expression portion for a single field.

13. The non-transitory computer readable storage medium of claim 12 , further comprising receiving user input to modify the automatically generated regular expression based on receipt of a subsequent user input.

14. The non-transitory computer readable storage medium of claim 8 , further comprising receiving user input to modify the automatically generated regular expression based on receipt of a subsequent user input.

15. A system for automatically generating a regular expression from log line data, comprising:
a server including a memory and a processor; and
one or more modules stored in the memory and executed by the processor to:
identify a portion of a log line in the log line data,
automatically generate multiple instances of multiple portions of a regular expression based on the identified portion, the multiple instances of multiple portions of the regular expression including:
at least one regular expression generated that represents a field within the portion of the log line, and
at least one regular expression that represents a gap within the portion of the log line, where the at least one regular expression that represents the gap is generated by identifying starting patterns and ending patterns of the gap, a particular pattern for a string immediately after the gap, or a count of delimiter occurrences defining the gap,

automatically generate a complete regular expression from one or more of the generated multiple instances, and
retrieve subsequent log lines based on the automatically generated complete regular expression.

16. The system of claim 15 , wherein identifying includes receiving user input indicative of the portion of a log line.

17. The system of claim 16 , wherein the received user input identifies a field in the indicated portion of the log line.

18. The system of claim 15 , wherein the one or more modules are further executable to generate at least one of the multiple instances of the multiple portions of the regular expressions for each of a plurality of gaps in the log line based on the identified portion, and generate at least one of the multiple instances of the multiple portions of the regular expressions for each of a plurality of fields in the log line based on the identified portion.

19. The system of claim 18 , wherein generating a regular expression includes generating a regular expression based on multiple instances of the regular expression portion for a single gap and multiple instances of the regular expression portion for a single field.

20. The system of claim 19 , wherein the one or more modules are further executable to rec

Assignees

• Appdynamics Llc
• Cisco Tech Inc

Inventors

• Zhao Yuchen
• Haddadkaveh Nima
• Iyer Arjun

Classifications

• G06F17/40 Primary

  Data acquisition and logging (for input to computer G06F3/00) · CPC title

• G06F11/3003

  Monitoring arrangements specially adapted to the computing system or computing system component being monitored · CPC title

• G06F11/302

  where the computing system component is a software system · CPC title

• G06F16/95

  Retrieval from the web · CPC title

• G06F2201/875

  Monitoring of systems including the internet · CPC title