Machine learning based software correction

Patent metadata

Field	Value
Publication number	US-10558554-B2
Application number	US-201815907988-A
Country	US
Kind code	B2
Filing date	Feb 28, 2018
Priority date	Feb 28, 2018
Publication date	Feb 11, 2020
Grant date	Feb 11, 2020

Abstract

Techniques and solutions are described for training and using a classifier based on a machine learning model to analyze performance information to assist in correcting a software bug or performance issue. The performance information can be processed prior to submission to a trained classifier, such as to remove, modify, or format data. A classification result provided by the classifier can be compared with a database to determine whether a solution or target is associated with the classification result. User feedback can be used to provide more accurate suggestions of solutions or targets, as well as to improve the accuracy of the classifier.

First claim

What is claimed is:

1. One or more non-transitory computer-readable storage media storing computer-executable instructions for causing a computing system to perform processing to analyze a software performance issue or bug, the processing comprising:
receiving performance information associated with software associated with the performance issue or bug and associated with software execution during the occurrence of the performance issue or bug, the performance information comprising at least one performance parameter and at least one software feature associated with the at least one performance parameter;
formatting at least a portion of the performance information, comprising the at least one software feature, for submission to a classifier using a machine learning model;
providing the formatted performance information to the classifier;
receiving a classification result for the formatted performance information;
determining if a solution is associated with the classification result; and
based on the determining, returning at least one identified solution or an indication that a solution was not identified.

2. The one or more non-transitory computer-readable storage media of claim 1 , the processing further comprising:
determining at least one type associated with the performance information;
retrieving an extraction rule for the determined at least on type; and
according to the extraction rule, removing at least a portion of the performance information prior to providing the formatted performance information to the classifier.

3. The one or more non-transitory computer-readable storage media of claim 2 , wherein the extraction rule specifies at least one performance criterion to determine whether performance information should be removed.

4. The one or more non-transitory computer-readable storage media of claim 1 , wherein the formatting comprises instantiating one or more word vectors comprising at least some of the at least a portion of the performance information.

5. The one or more non-transitory computer-readable storage media of claim 1 , wherein the formatting comprises replacing specific identifiers of the at least a portion of the performance information with generic identifiers.

6. The one or more non-transitory computer-readable storage media of claim 5 , wherein the specific information comprises a table identifier or a field identifier.

7. The one or more non-transitory computer-readable storage media of claim 1 , wherein the machine learning model comprises a Continuous Bag of Words or Skip-Gram model.

8. The one or more non-transitory computer-readable storage media of claim 1 , wherein the machine learning model uses word embeddings.

9. The one or more non-transitory computer-readable storage media of claim 1 , the processing further comprising:
monitoring performance of the software;
determining that a performance issue has occurred; and
triggering generation of the performance information.

10. The one or more non-transitory computer-readable storage media of claim 1 , the processing further comprising:
determining a plurality of solutions as potentially relevant to the performance issue or bug based on the classification result;
ranking the plurality of solutions, wherein the returned at least one identified solution is the highest ranked solution.

11. The one or more non-transitory computer-readable storage media of claim 1 , the processing further comprising:
determining a first plurality of solutions as potentially relevant to the performance issue or bug based on the classification result; and
ranking the first plurality of solutions, wherein returning at least one identified solution comprises returning a ranked list comprising a plurality of the ranked solutions.

12. The one or more non-transitory computer-readable storage media of claim 1 , the processing further comprising:
receiving user input of a solution to a performance issue or bug corresponding to the classification result; and
adding the solution to a solution database, wherein the added solution is available for association with later classification results.

13. The one or more non-transitory computer-readable storage media of claim 1 , the processing further comprising:
receiving user input identifying the solution as responsive or not responsive to the performance issue or bug; and
adjusting an association between the classification result and the solution based on the user input.

14. The one or more non-transitory computer-readable storage media of claim 1 , the processing further comprising:
categorizing the performance information; and
selecting a classifier to which the formatted performance information will be provided based on the categorization.

15. The one or more non-transitory computer-readable storage media of claim 1 , the processing further comprising:
categorizing the performance information;
determining a formatting rule for a category assigned to the performance information; and
selecting performance information associated with one or more software operations based on the formatting rule.

16. The one or more non-transitory computer-readable storage media of claim 15 , the processing further comprising, for each of a plurality of software operations of the performance information:
comparing at least one performance parameter associated with the software operation with a threshold value; and
selecting the operation for formatting if the at least one performance parameter violates a threshold established for the at least one performance parameter and the software operation.

17. The one or more non-transitory computer readable storage media of claim 1 , wherein returning an indication that a solution was not identified comprising returning an investigation target, wherein the investigation target comprises a program or program component that may be associated with the performance issue or bug.

18. The one or more non-transitory computer-readable storage media of claim 17 , the processing further comprising:
receiving user input identifying the target as responsive or not responsive to the performance issue or bug; and
adjusting an association between the classification result and the target based on the user input.

19. A computing system that implements a performance analysis and recommendation system, the computing system comprising:
memory;
one or more processing units coupled to the memory; and
one or more non-transitory computer readable storage media storing instructions that, when loaded into the memory, cause the one or more processing units to perform operations for:
monitoring at least one software performance metric;
determining that the at least one metric violates a threshold;
causing performance information to be gathered, the performance information comprising at least one performance parameter and at least one software feature associated with the at least one performance parameter;
determining a type associated with the performance information;
retrieving an extraction rule associated with the type;
extracting the at least one performance parameter using the extraction rule;
formatting performance information comprising the at least one performance parameter for submission to a classifier using a machine learning model or a clustering engine;
providing the formatted performance information to the classifier or the clustering engine;
receiving a classification or clustering result for the formatted performance information;
querying a solutions database to de

Assignees

• Sap Se

Inventors

• Bhandarkar Dinesh
• Kalleppilli Biju V

Classifications

• G06N20/00

  Machine learning · CPC title

• G06F11/3409

  for performance assessment · CPC title

• G06F11/3466 Primary

  Performance evaluation by tracing or monitoring · CPC title

• G06F11/079

  Root cause analysis, i.e. error or fault diagnosis (in a hardware test environment G06F11/22; in a software test environment G06F11/36) · CPC title

• G06F11/3644 Primary

  by instrumenting at runtime · CPC title