Test device selection using multi-pass scoring

Patent metadata

Field	Value
Publication number	US-9684579-B1
Application number	US-201414562343-A
Country	US
Kind code	B1
Filing date	Dec 5, 2014
Priority date	Dec 5, 2014
Publication date	Jun 20, 2017
Grant date	Jun 20, 2017

Abstract

A method for selecting test devices in a service provider environment may include receiving a request for performing a test on a device, the request specifying at least one parameter for the test. A list of available devices may be obtained. The list of available devices may be filtered to generate a list of candidate hardware devices that meet the at least one parameter for the test. Using a first set of operations, current state information for each of the candidate devices may be obtained. Using a second set of operations, at least one weight may be assigned to the current state information to generate weighted state information for each of the candidate devices. Device scores may be generated using the weighted state information. Based on the scores, one of the candidate devices may be selected for performing the test.

First claim

What is claimed is:

1. A computer-readable storage medium including instructions that upon execution cause a computer system to:
receive a request for performing a test on a device, the request specifying at least one parameter for the test;
obtain a list of available devices;
filter the list of available devices to generate a list of candidate devices that meet the at least one parameter for the test;
obtain current state information for each of the candidate devices;
normalize the current state information so that minimum and maximum values are mapped within a numeric range;
assign at least one weight to the normalized current state information to generate normalized weighted state information for each of the candidate devices;
generate device scores based at least in part on the normalized weighted state information; and
based at least in part on the scores, select one of the candidate devices for performing the test.

2. The computer-readable storage medium according to claim 1 , wherein the instructions, upon execution further cause the computer system to:
assign each of the candidate devices to a separate test activity queue.

3. The computer-readable storage medium according to claim 2 , wherein the instructions, upon execution further cause the computer system to:
forward the test to an activity queue assigned to the selected one of the candidate devices.

4. The computer-readable storage medium according to claim 1 , wherein the at least one weight comprises a weight value for each of a plurality of criteria associated with the current state information.

5. The computer-readable storage medium according to claim 4 , wherein each of the plurality of criteria associated with the current state information for a selected device comprises:
device status information;
a number of pending tasks;
a number of idle devices with one or more characteristic that is common with a corresponding characteristic of the selected device;
a total number of devices with the one or more characteristic that is common with the corresponding characteristic of the selected device; and
a last executed test time.

6. A method for selecting test devices in a service provider environment, the method comprising:
using a test scheduling service (TSS) of a service provider:
obtaining a list of candidate devices that meet at least one pre-determined test parameter of a test;
using a first set of operations, obtain current state information for each of the candidate devices and normalize the current state information so that minimum or maximum values are mapped to numeric values;
using a second set of operations, generate device scores for each of the candidate devices based at least in part on applying a weighting to the normalized current state information; and
based at least in part on the device scores, select one of the candidate devices for performing the test.

7. The method according to claim 6 , further comprising:
receiving a request for performing the test on at least one of a plurality of available devices.

8. The method according to claim 7 , further comprising:
filtering the plurality of available devices to generate the list of candidate devices, the filtering based on at least one input criteria.

9. The method according to claim 8 , wherein the at least one input criteria comprises device build type, device battery life, device operating system revision number, device rooted state, device product code, device operating status, and/or time of last conducted device test.

10. The method according to claim 6 , wherein the current state information for a selected one of the candidate devices comprises:
device status information;
a number of pending tasks for the selected device;
a number of idle devices with one or more characteristic that is common with a corresponding characteristic of the selected device;
a total number of devices with the one or more characteristic that is common with the corresponding characteristic of the selected device; and
a last executed test time.

11. The method according to claim 6 , further comprising:
during the first set of operations, assigning numeric values to the current state information for each of the candidate devices.

12. The method according to claim 11 , further comprising:
during the second set of operations, normalizing the assigned numeric values to fit within a pre-determined numeric range.

13. The method according to claim 12 , further comprising:
retrieving at least one weight from a look-up table, the at least one weight associated with the current state information.

14. The method according to claim 13 , further comprising:
assign the at least one weight to the normalized numeric values to generate the device scores.

15. The method according to claim 6 , further comprising:
identifying an activity queue assigned to the selected one of the candidate devices.

16. The method according to claim 15 , further comprising:
queuing the test in the activity queue for execution at the selected one of the candidate devices.

17. A system for selecting test devices in a service provider environment, the system comprising:
a plurality of test devices attached to at least one of a plurality of server computers, the plurality of server computers coupled together through a network to form the service provider;
a queuing server coupled to the plurality of server computers; and
a test scheduling service coupled to the queuing server, the test scheduling service operable to:
generate a list of candidate devices from the plurality of test devices, the candidate devices meeting at least one pre-determined test parameter of a test;
obtain current state information for each of the candidate devices, the state information including a number of pending tasks;
generate device scores for each of the candidate devices based at least in part on the obtained current state information including the number of pending tasks;
based at least in part on the scores, select one of the candidate devices for performing the test; and
communicating via the queuing server, the test to the selected one of the candidate devices for execution.

18. The system according to claim 17 , wherein the queuing server is operable to queue the test to an activity queue associated with the candidate device.

19. The system according to claim 17 , wherein the test scheduling service is further operable to:
receive a request for performing the test on at least one of the plurality of test devices.

20. The system according to claim 19 , wherein the test scheduling service is further operable to:
filter the plurality of test hardware devices to generate the list of candidate devices, the filtering based on at least one input criteria.

Assignees

• Amazon Tech Inc

Inventors

• Adams Zoe
• Marinov Jordan Petrov
• Ganoo Chaitanya Avinash

Classifications

• G06F11/263 Primary

  Generation of test inputs, e.g. test vectors, patterns or sequences {; with adaptation of the tested hardware for testability with external testers} · CPC title

• G06F11/261 Primary

  by simulating additional hardware, e.g. fault simulation · CPC title