Techniques for detecting atypical events in event logs

Patent metadata

Field	Value
Publication number	US-11537498-B2
Application number	US-202016903069-A
Country	US
Kind code	B2
Filing date	Jun 16, 2020
Priority date	Jun 16, 2020
Publication date	Dec 27, 2022
Grant date	Dec 27, 2022

Abstract

Examples described herein generally relate to processing event logs where, for each of multiple events in an event log of the one or more event logs, a table of logged event instances can be generated for the event. For each of the multiple events, the table can be processed using an autoencoder to identify one or more of the logged event instances as anomalies, and an indication of at least a portion of the anomalies can be output. In addition, the event logs and/or corresponding tables of events can be used to train models for the autoencoders.

First claim

What is claimed is:

1. A computer-implemented method for processing event logs, comprising:
obtaining one or more event logs generated by a process executing on one or more computing devices;
for each event of multiple events in the one or more event logs:
generating a table of logged event instances for the event, wherein each logged event instance includes a value for a parameter, wherein the parameter is a feature for anomaly detection based on a numeric count of distinct values for the parameter being within a threshold numeric range;
identifying a logged event instance as an anomaly in the process by processing the table using an autoencoder, the identifying being based on the value for the parameter in the logged event instance; and
outputting an indication of the anomaly in the process.

2. The computer-implemented method of claim 1 , further comprising training a model for the autoencoder using the logged event instance that includes the value for the parameter.

3. The computer-implemented method of claim 2 , wherein training the model for the autoencoder using the logged event instance that includes the value for the parameter includes at least one of generating the value as a numeric value and/or normalizing the numeric value.

4. The computer-implemented method of claim 2 , wherein the autoencoder identifies the logged event instance as the anomaly based on subtracting output of the autoencoder from the table to determine an error value for the parameter.

5. The computer-implemented method of claim 1 , further comprising identifying the parameter for anomaly detection based on the numeric count of distinct values for the parameter being within the threshold numeric range.

6. The computer-implemented method of claim 5 , wherein the identifying includes:
generating a histogram for the numeric count of distinct values for the parameter; and
determining that the numeric count of distinct values for the parameter is within the threshold numeric range is based on the histogram.

7. The computer-implemented method of claim 1 , wherein the indication includes information for locating the anomaly in the one or more event logs.

8. The computer-implemented method of claim 1 , further comprising generating, based on the anomaly, a signature for the one or more event logs.

9. A computing device for processing event logs, comprising:
a memory storing one or more parameters or instructions for executing an operating system and one or more processes; and
at least one processor coupled to the memory, wherein the at least one processor is configured to:
obtain one or more event logs generated by a process executing on the computing device;
for each event of multiple events in the one or more event logs:
generate a table of logged event instances for the event, wherein each logged event instance includes a value for a parameter, wherein the parameter is a feature for anomaly detection based on a numeric count of distinct values for the parameter being within a threshold numeric range;
identify a logged event instance as an anomaly in the process by processing the table using a trained autoencoder, the identifying being based on the value for the parameter in the logged event instance; and
output an indication of the anomaly in the process.

10. The computing device of claim 9 , wherein the trained autoencoder identifies the logged event instance as the anomaly based on subtracting output of the trained autoencoder from the table to determine an error value for the parameter.

11. The computing device of claim 9 , wherein the at least one processor is configured to identify the parameter for anomaly detection based on the numeric count of distinct values for the parameter being within the threshold numeric range.

12. The computing device of claim 11 , wherein the at least one processor is configured to:
generate a histogram for the numeric count of distinct values for the parameter; and
determine that the numeric count of distinct values for the parameter is within the threshold numeric range based on the histogram.

13. The computing device of claim 9 , wherein the indication includes information for locating the anomaly in the one or more event logs.

14. The computing device of claim 9 , the at least one processor is further configured to generate, based on the anomaly, a signature for the one or more event logs.

15. A non-transitory computer-readable medium, comprising code executable by one or more processors for processing event logs, the code comprising code for:
obtaining one or more event logs generated by a process executing on one or more computing devices;
for each event of multiple events in the one or more event logs:
generating a table of logged event instances for the event, wherein each logged event instance includes a value for a parameter, wherein the parameter is a feature for anomaly detection based on a numeric count of distinct values for the parameter being within a threshold numeric range;
identifying a logged event instance as an anomaly in the process by processing the table using an autoencoder, the identifying being based on the value for the parameter in the logged event instance; and
outputting an indication of the anomaly in the process.

16. The non-transitory computer-readable medium of claim 15 , further comprising code for training a model for the autoencoder using the logged event instance that includes the value for the parameter.

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Ibrahim Ziyad Ahmad
• Paul Grenier Alexander Robert
• Mccaffrey James David
• Kamra Dharmanshu
• Prabhu Sudhakar Visweswara
• Carpenter Daniel James

Classifications

• G06F11/3476 Primary

  Data logging (G06F11/14, G06F11/2205 take precedence) · CPC title

• G06F21/552 Primary

  involving long-term monitoring or reporting · CPC title

• G06F2201/86

  Event-based monitoring · CPC title

• G06F2201/81

  Threshold · CPC title

• G06F21/57

  Certifying or maintaining trusted computer platforms, e.g. secure boots or power-downs, version controls, system software checks, secure updates or assessing vulnerabilities · CPC title