Automated data-generation for event-based system

Patent metadata

Field	Value
Publication number	US-2018032861-A1
Application number	US-201615224489-A
Country	US
Kind code	A1
Filing date	Jul 29, 2016
Priority date	Jul 29, 2016
Publication date	Feb 1, 2018
Grant date	—

Abstract

Described herein is a technology that facilitates the production of and the use of automated datagens for event-based. A datagen (i.e., data-generator or data generation system) is a component, module, or subsystem of computer systems that searches, monitors, and analyzes machine data. A datagen produces events that are further processed in various ways for subsequent use (such as searching, monitoring, and analysis).

First claim

What is claimed is:

1 . A computer-implemented method comprising:
obtaining a training corpus that contains example structured events, each structured event having at least one data field configured to contain data derived from machine-generated data; training a datagen that is configured to generate new structured events in accordance with the example structured events of the training corpus, the training includes employment of deep-learning technique to train the datagen using the training corpus.

2 . A method as recited in claim 1 , wherein the deep-learning technique includes a character-based recurrent neural network.

3 . A method as recited in claim 1 further comprising producing new events by the trained datagen.

4 . A method as recited in claim 1 , wherein the example structured events of the training corpus is a sequence of textual characters and the training of the datagen includes calculating statistical predictions based upon the sequence of the textual characters of the example structured events of the training corpus.

5 . A method as recited in claim 1 , wherein the example structured events of the training corpus is a sequence of textual characters and the training of the datagen includes calculating statistical predictions based upon the sequence of the textual characters of the example structured events of the training corpus, the method further comprising:
producing new events by the trained datagen, the producing includes generating a sequence of characters of the new events based upon the calculated statistical predictions of the sequence of the textual characters of the example structured events of the training corpus.

6 . A method as recited in claim 1 , wherein the example structured events of the training corpus is a sequence of textual characters and the training of the datagen includes calculating statistical predictions based upon the sequence of the textual characters of the example structured events of the training corpus, the method further comprising:
producing new events by the trained datagen, the producing includes generating a sequence of characters of the new events based upon the calculated statistical predictions of the sequence of the textual characters of the example structured events of the training corpus, wherein the generating includes selecting a next character of a new event based upon greatest likelihood of the selected character appearing next in the generated sequence as determined by the calculated statistical predictions of the sequence of the textual characters of the example structured events of the training corpus.

7 . A method as recited in claim 1 further comprising:
obtaining a dataset of machine-generated data;
generating, by the trained datagen and in accordance with the example structured events of the training corpus, multiple events from the obtained dataset of machine-generated data.

8 . A method as recited in claim 1 further comprising:
obtaining a dataset of machine-generated data, wherein the dataset of machine-generated data includes a sequence of textual characters;
processing the obtained dataset of machine-generated data by the trained datagen in order of the sequence of textual characters;
during the processing of the obtained dataset, calculating a statistical prediction of a next yet-to-be-processed group of one or more textual characters of the sequence of the textual characters;
generating, by the trained datagen, an event from the obtained dataset of machine-generated data one textual character at a time, wherein each generated character is generated based on the calculated statistical prediction of the next yet-to-be-processed group of one or more textual characters.

9 . One or more computer-readable media storing instructions thereon that, when executed by one or more processors, direct the one or more processors to perform operations comprising:
obtaining a training corpus that contains example structured events, each structured event having at least one data field configured to contain data derived from machine-generated data; training a datagen that is configured to generate new structured events in accordance with the example structured events of the training corpus, the training includes employment of deep-learning technique to train the datagen using the training corpus.

10 . One or more computer-readable media as recited in claim 9 , wherein the deep-learning technique includes a character-based recurrent neural network.

11 . One or more computer-readable media as recited in claim 9 , wherein the operations further comprise producing new events by the trained datagen.

12 . One or more computer-readable media as recited in claim 9 , wherein the example structured events of the training corpus is a sequence of textual characters and the training of the datagen includes calculating statistical predictions based upon the sequence of the textual characters of the example structured events of the training corpus, the operations further comprise:
producing new events by the trained datagen, the producing includes generating a sequence of characters of the new events based upon the calculated statistical predictions of the sequence of the textual characters of the example structured events of the training corpus.

13 . One or more computer-readable media as recited in claim 9 , wherein the example structured events of the training corpus is a sequence of textual characters and the training of the datagen includes calculating statistical predictions based upon the sequence of the textual characters of the example structured events of the training corpus, the operations further comprise:
producing new events by the trained datagen, the producing includes generating a sequence of characters of the new events based upon the calculated statistical predictions of the sequence of the textual characters of the example structured events of the training corpus, wherein the generating includes selecting a next character of a new event based upon greatest likelihood of the selected character appearing next in the generated sequence as determined by the calculated statistical predictions of the sequence of the textual characters of the example structured events of the training corpus.

14 . One or more computer-readable media as recited in claim 9 , the operations further comprise:
obtaining a dataset of machine-generated data; generating, by the trained datagen and in accordance with the example structured events of the training corpus, multiple events from the obtained dataset of machine-generated data.

15 . One or more computer-readable media as recited in claim 9 , the operations further comprise:
obtaining a dataset of machine-generated data, wherein the dataset of machine-generated data includes a sequence of textual characters; processing the obtained dataset of machine-generated data by the trained datagen in order of the sequence of textual characters; during the processing of the obtained dataset, calculating a statistical prediction of a next yet-to-be-processed group of one or more textual characters of the sequence of the textual characters; generating, by the trained datagen, an event from the obtained dataset of machine-generated data one textual character at a time, wherein each generated character is generated based on the calculated statistical prediction of the next yet-to-be-processed group of one or more textual characters.

16 . An automated data generation system comprising:
a training data handler configured to obtain a training corpus that contains exampl

Assignees

• Splunk Inc

Inventors

• Oliner Adam
• Yang Zidong
• Sreshta Sinduja

Classifications

• G06N3/044

  Recurrent networks, e.g. Hopfield networks · CPC title

• G06Q10/0637 Primary

  Strategic management or analysis, e.g. setting a goal or target of an organisation; Planning actions based on goals; Analysis or evaluation of effectiveness of goals · CPC title

• G06F40/274 Primary

  Converting codes to words; Guess-ahead of partial word inputs · CPC title

• G06N3/0442

  characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] · CPC title

• G06N3/09

  Supervised learning · CPC title