Crowd-sourced QA with trusted compute model

Patent metadata

Field	Value
Publication number	US-11748246-B2
Application number	US-202117242711-A
Country	US
Kind code	B2
Filing date	Apr 28, 2021
Priority date	Apr 28, 2021
Publication date	Sep 5, 2023
Grant date	Sep 5, 2023

Abstract

An approach is provided in which the approach collects distributed test results from multiple users, wherein at least one of the multiple users is an untrusted user. The approach aggregates the distributed test results in response to determining that each one of the distributed test results corresponds to the same system under test. The approach then presents the aggregated test results as a trusted test result.

First claim

The invention claimed is:

1. A method comprising:
collecting a plurality of distributed test results from a plurality of users, wherein at least one of the plurality of users is an untrusted user;
selecting one of the plurality of distributed test results corresponding to a selected one of the plurality of users;
determining a verification level of the selected user;
in response to determining that the selected user requires verification, verifying the selected user by verifying a token sent with the selected distributed test results;
in response to verifying the selected user, verifying the selected distributed test results;
in response to verifying the selected distributed test results, aggregating the selected distributed tests results based on both a system under test ID and a test ID;
aggregating the plurality of distributed test results in response to determining that each one of the plurality of distributed test results corresponds to a same system under test; and
presenting the aggregated test results as a trusted test result.

2. The method of claim 1 further comprising:
utilizing a trust service to verify the selected user, wherein the trust service implements a plurality of verification levels; and
adding a new verification level to the trust service in response to verifying the selected distributed test results.

3. The method of claim 1 further comprising:
adjusting the verification level of the selected user in response to determining that an amount of times the selected user provides truthful distributed test results reaches a threshold.

4. The method of claim 1 further comprising:
determining that a first one of the plurality of distributed test results from a first one of the plurality of users corresponds to a software test target;
determining that a second one of the plurality of distributed test results from a second one of the plurality of users corresponds to the software test target; and
combining the first distributed test results with the second distributed test results in response to determining that they both correspond to the same software test target.

5. The method of claim 3 further comprising:
providing a graphical user interface to the plurality of users that comprises a plurality of software test target indicators;
identifying a selected one of the plurality of test target indicators that corresponds to the software test target; and
marking the software test target indicator as complete in response to receiving the first distributed test results.

6. The method of claim 1 further comprising:
prior to collecting the plurality of distributed test results, receiving a registration request from the untrusted user and sending a public key to the untrusted user in response to receiving the registration request; and
verifying the untrusted user based on verifying the public key included in a set of the plurality of distributed test results sent from the untrusted user.

7. An information handling system comprising:
one or more processors;
a memory coupled to at least one of the processors;
a set of computer program instructions stored in the memory and executed by at least one of the processors in order to perform actions of:
collecting a plurality of distributed test results from a plurality of users, wherein at least one of the plurality of users is an untrusted user;
selecting one of the plurality of distributed test results corresponding to a selected one of the plurality of users;
determining a verification level of the selected user;
in response to determining that the selected user requires verification, verifying the selected user by verifying a token sent with the selected distributed test results;
in response to verifying the selected user, verifying the selected distributed test results;
in response to verifying the selected distributed test results, aggregating the selected distributed tests results based on both a system under test ID and a test ID;
aggregating the plurality of distributed test results in response to determining that each one of the plurality of distributed test results corresponds to a same system under test; and
presenting the aggregated test results as a trusted test result.

8. The information handling system of claim 7 wherein the processors perform additional actions comprising:
utilizing a trust service to verify the selected user, wherein the trust service implements a plurality of verification levels; and
adding a new verification level to the trust service in response to verifying the selected distributed test results.

9. The information handling system of claim 7 wherein the processors perform additional actions comprising:
adjusting the verification level of the selected user in response to determining that an amount of times the selected user provides truthful distributed test results reaches a threshold.

10. The information handling system of claim 7 wherein the processors perform additional actions comprising:
determining that a first one of the plurality of distributed test results from a first one of the plurality of users corresponds to a software test target;
determining that a second one of the plurality of distributed test results from a second one of the plurality of users corresponds to the software test target; and
combining the first distributed test results with the second distributed test results in response to determining that they both correspond to the same software test target.

11. The information handling system of claim 10 wherein the processors perform additional actions comprising:
providing a graphical user interface to the plurality of users that comprises a plurality of software test target indicators;
identifying a selected one of the plurality of test target indicators that corresponds to the software test target; and
marking the software test target indicator as complete in response to receiving the first distributed test results.

12. The information handling system of claim 7 wherein the processors perform additional actions comprising:
prior to collecting the plurality of distributed test results, receiving a registration request from the untrusted user and sending a public key to the untrusted user in response to receiving the registration request; and
verifying the untrusted user based on verifying the public key included in a set of the plurality of distributed test results sent from the untrusted user.

13. A computer program product stored in a computer readable storage medium, comprising computer program code that, when executed by an information handling system, causes the information handling system to perform actions comprising:
collecting a plurality of distributed test results from a plurality of users, wherein at least one of the plurality of users is an untrusted user;
selecting one of the plurality of distributed test results corresponding to a selected one of the plurality of users;
determining a verification level of the selected user;
in response to determining that the selected user requires verification, verifying the selected user by verifying a token sent with the selected distributed test results;
in response to verifying the selected user, verifying the selected distributed test results;
in response to verifying the selected distributed test results, aggregating the selected distributed tests results based on both a system under test ID and a test ID;
aggregating the plurality of distributed test results in response to determining that each one of the plurality of distributed test results corresponds to a same system under test; and
presenting the aggregated

Assignees

• IBM

Inventors

• Lambert Shelley
• Ellison Timothy Peter
• Shin Haechul
• Cuomo Gennaro A
• Mosakowski Barry
• Parzygnat Mark Anthony

Classifications

• G06F11/3698

  Environments for analysis, debugging or testing of software · CPC title

• G06F11/3692 Primary

  for test results analysis · CPC title

• G06F11/3664

  Physics · mapped topic

• H04L9/3239

  involving non-keyed hash functions, e.g. modification detection codes [MDCs], MD5, SHA or RIPEMD · CPC title

• G06F2221/033

  Test or assess software · CPC title