Test result triage for a failed code validation

Patent metadata

Field	Value
Publication number	US-10970200-B2
Application number	US-201916245687-A
Country	US
Kind code	B2
Filing date	Jan 11, 2019
Priority date	Jan 11, 2019
Publication date	Apr 6, 2021
Grant date	Apr 6, 2021

Abstract

Systems, methods, and software are disclosed herein for triaging test results of a failed code validation. Test results are received indicative of a failed code validation performed in a client device. Failure data is generated from the test results indicating one or more failure modes which occurred during the failed code validation. A follow-up action associated with each of the one or more failure modes is then determined by comparing each of the one or more failure modes to previously identified failure modes. Instructions are communicated to the client device based on the determined follow-up action associated with each of the one or more failure modes.

First claim

The invention claimed is:

1. A computer apparatus comprising:
one or more computer readable storage media;
one or more processors operatively coupled to the one or more computer readable storage media; and
program instructions stored on the one or more computer readable storage media that, upon execution by the one or more processors, direct the one or more processors to at least:
receive test results indicative of a failed code validation performed in a client device;
generate, from the test results, failure data indicating one or more failure modes which occurred during the failed code validation;
determine a follow-up action associated with each of the one or more failure modes by comparing each of the one or more failure modes to previously identified failure modes, wherein the program instructions to determine the follow-up action comprise program instructions that, upon execution by the one or more processors, direct the one or more processors to at least:
classify each of the one or more failure modes as known or unknown by comparing each of the one or more failure modes to known failure modes; and

communicate instructions to the client device based on the determined follow-up action associated with each of the one or more failure modes, wherein the program instructions to communicate instructions to the client device comprise program instructions that, upon execution by the one or more processors, direct the one or more processors to at least:
block a retry of the failed code validation performed in the client device based on a determination that one or more of the one or more failure modes is classified as unknown.

2. The computer apparatus of claim 1 wherein the program instructions to generate the failure data comprise program instructions that, upon execution by the one or more processors, direct the one or more processors to at least:
normalize the test results.

3. The computer apparatus of claim 2 wherein the program instructions to normalize the test results comprise further instructions that, upon execution by the one or more processors, direct the one or more processors to:
sanitize the test results by removing specific details from the test results to generate a normalized code string.

4. The computer apparatus of claim 3 wherein the program instructions to normalize the test results comprise further instructions that, upon execution by the one or more processors, direct the one or more processors to at least:
hash the normalized test results to provide the failure data indicating the one or more failure modes which occurred during the failed code validation.

5. The computer apparatus of claim 1 wherein the program instructions to communicate instructions to the client device comprise program instructions that, upon execution by the one or more processors, direct the one or more processors to at least:
retry the failed code validation performed in the client device based on a determination that one or more of the one or more failure modes are known.

6. The computer apparatus of claim 1 wherein the program instructions comprise further program instructions that, upon execution by the one or more processors, direct the one or more processors to at least:
in response to a determination that a duration of the failed code validation exceeds a code validation timer or a number of failure modes exceeds a failure mode threshold, communicate instructions to block a retry of the failed code validation performed in the client device.

7. The computer apparatus of claim 1 wherein the program instructions to determine the follow-up action comprise program instructions that, upon execution by the one or more processors, direct the one or more processors to at least:
compare each of the one or more failure modes to previously identified failure modes having a corresponding Operating System (OS) type to determine the follow-up action associated with each of the one or more failure modes.

8. The computer apparatus of claim 1 wherein the program instructions to determine the follow-up action comprise program instructions that, upon execution by the one or more processors, direct the one or more processors to at least:
compare each of the one or more failure modes to previously identified failure modes having a corresponding Central Processing Unit (CPU) type to determine the follow-up action associated with each of the one or more failure modes.

9. The computer apparatus of claim 1 wherein the program instructions to determine the follow-up action comprise program instructions that, upon execution by the one or more processors, direct the one or more processors to at least:
compare each of the one or more failure modes to previously identified failure modes having a corresponding web-based platform type to determine the follow-up action associated with each of the one or more failure modes.

10. A method comprising:
receiving test results indicative of a failed code validation performed in a client device;
generating, from the test results, failure data indicating one or more failure modes which occurred during the failed code validation;
determining a follow-up action associated with each of the one or more failure modes by comparing each of the one or more failure modes to previously identified failure modes, wherein the determining the follow-up action comprises classifying each of the one or more failure modes as known or unknown by comparing each of the one or more failure modes to known failure modes; and
communicating instructions to the client device based on the determined follow-up action associated with each of the one or more failure modes, wherein communicating instructions to the client device comprises blocking a retry of the failed code validation performed in the client device based on a determination that one or more of the one or more failure modes is classified as unknown.

11. The method of claim 10 wherein generating the failure data comprises normalizing the test results.

12. The method of claim 11 wherein normalizing the test results comprises sanitizing the test results by removing specific details from the test results to generate a normalized code string.

13. The method of claim 12 wherein normalizing the test results further comprises hashing the normalized test results to provide the failure data indicating the one or more failure modes which occurred during the failed code validation.

14. The method of claim 10 wherein the instructions to the client device comprise instructions to retry the failed code validation performed in the client device based on a determination that one or more of the one or more failure modes are known.

15. The method of claim 10 further comprising:
in response to determining that a duration of the failed code validation exceeds a code validation timer or a number of failure modes exceeds a failure mode threshold, communicating instructions to block a retry of the failed code validation performed in the client device.

16. The method of claim 10 wherein determining the follow-up action comprises comparing each of the one or more failure modes to previously identified failure modes having a corresponding Operating System (OS) type to determine the follow-up action associated with each of the one or more failure modes.

17. The method of claim 10 wherein determining the follow-up action comprises comparing each of the one or more failure modes to previously identified failure modes having a corresponding Central Processing Unit (CPU) type to determine the follow-up action associated with

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Haubold Jeremy
• De Sousa Santos Jessica Priscila

Classifications

• G06F11/3692 Primary

  for test results analysis · CPC title

• G06F11/366

  using diagnostics (G06F11/0703 takes precedence) · CPC title

• G06F11/0793

  Remedial or corrective actions (recovery from an exception in an instruction pipeline G06F9/3861; by retry G06F11/1402; for recovering from a failure of a protocol instance or entity H04L69/40) · CPC title

• G06F11/3688

  for test execution, e.g. scheduling of test suites · CPC title