Managing and adapting monitoring programs

Patent metadata

Field	Value
Publication number	US-11296971-B1
Application number	US-202117378643-A
Country	US
Kind code	B1
Filing date	Jul 16, 2021
Priority date	Feb 3, 2021
Publication date	Apr 5, 2022
Grant date	Apr 5, 2022

Abstract

Systems, methods, and apparatus, including computer-readable media for managing and adapting monitoring programs. In some implementations, a system accesses data describing a monitoring program that involves collecting data over a period of time from a monitoring group comprising geographically distributed devices. The system classifies the devices in the monitoring group into different groups for the categories based on the attributes associated with the devices. The system monitoring performance of the devices in the different groups and determines a performance score for each group. The system adjusts administration of the monitoring program based on the performance scores, including performing one or more actions to improve diversity in the monitoring group.

First claim

The invention claimed is:

1. A method performed by one or more computers, the method comprising:
accessing, by the one or more computers, data describing a monitoring program that involves collecting data over a period of time from a monitoring group comprising geographically distributed devices involved in the monitoring program, wherein the distributed devices provide sensor measurements over a communication network, wherein the monitoring program has a corresponding diversity target that specifies a level of diversity to achieve among multiple categories, wherein the accessed data includes (i) data collection requirements indicating types of data for the devices in the monitoring group to provide as part of the monitoring program and (ii) attributes associated with the devices in the monitoring group;
based on the attributes associated with the devices, classifying, by the one or more computers, the devices in the monitoring group into different groups corresponding to the categories;
monitoring, by the one or more computers, performance of the devices in the different groups with respect to the data collection requirements for the monitoring program;
evaluating, by the one or more computers, the performance of the different groups to determine a performance score for each group, wherein the performance score for a group is based on levels of compliance with the data collection requirements by the devices in the group; and
adjusting, by the one or more computers, administration of the monitoring program for the devices based on the performance scores for the groups, the adjusting comprising performing one or more actions to increase diversity with respect to the categories for the monitoring performed for the monitoring program.

2. The method of claim 1 , comprising prioritizing the different groups based on the performance scores for the different groups; and
wherein the administration of the monitoring program is adjusted based on the prioritization of the groups.

3. The method of claim 2 , comprising evaluating a level of diversity present for the monitoring program including determining, for each group, at least one of (i) an effect that the group has on the level of diversity or (ii) a likelihood that the group will meet the requirements for the group that are specified by the diversity target,
wherein prioritizing the different groups comprises prioritizing the different groups based on at least one of (i) the determined effects that the respective groups have on the level of diversity present for the monitoring program or (ii) the determined likelihoods that the respective groups will meet corresponding requirements for the groups.

4. The method of claim 3 , wherein the monitoring program has a predetermined minimum number of members for each group;
wherein evaluating the level of diversity for the monitoring program comprises determining differences between current or predicted future amounts of enrolled, complying members of the groups and corresponding minimums for the groups, wherein the effect that each group has on the level of diversity or the likelihood that each group meets its requirements is based on the determined difference for the group; and
wherein prioritizing the different groups comprises prioritizing the different groups based on the differences.

5. The method of claim 4 , wherein determining the differences between amounts of members and the corresponding minimums comprises:
identifying, for each of the groups, a number of devices in the group; and
calculating, for each of the groups, a difference between the identified number of devices and the minimum number of devices.

6. The method of claim 4 , wherein determining the differences comprises:
using the performance scores to predict, for each of the groups, a number of devices in the group that will comply with the data collection requirements at a future time; and
calculating, for each of the groups, a difference between the predicted number of devices for the group at the future time and the minimum for the group.

7. The method of claim 3 , wherein the diversity target for the monitoring program has one or more predetermined ratios specifying desired composition characteristics of the monitoring group with respect to the different categories;
wherein evaluating the level of diversity for the monitoring program comprises comparing a ratio involving of the devices in one or more groups with the one or more predetermined ratios, wherein the effect that each group has on the level of diversity or the likelihood of each group meeting corresponding requirements is based on the comparison of the ratios for the group; and
wherein prioritizing the different groups comprises prioritizing the different groups based on the comparisons.

8. The method of claim 1 , comprising accessing profiles that define the categories;
wherein classifying the devices in the monitoring group into the different groups corresponding to the categories comprises:
comparing the attributes associated with the devices with attributes or attribute ranges specified by the profiles; and
based on the comparison, assigning each devices to at least one of the categories.

9. The method of claim 1 , wherein adjusting administration of the monitoring program comprises:
adjusting a composition of the devices in the monitoring group based on the performance scores; or
selectively adjusting operation of a subset of devices that is selected from the monitoring group based on the performance scores.

10. The method of claim 9 , wherein the adjusting is performed in response to determining that the monitoring program (i) currently fails to meet or (ii) is predicted by the one or more computers to not meet the diversity target in the future, wherein the determination comprises comparing a current or predicted level of diversity for the monitoring group to the level of diversity specified by the diversity target.

11. A system comprising:
one or more computers; and
one or more computer-readable media storing instructions that, when executed by the one or more computers, are operable to cause the one or more computers to perform operations comprising:
accessing, by the one or more computers, data describing a monitoring program that involves collecting data over a period of time from a monitoring group comprising geographically distributed devices involved in the monitoring program, wherein the distributed devices provide sensor measurements over a communication network, wherein the monitoring program has a corresponding diversity target that specifies a level of diversity to achieve among multiple categories, wherein the accessed data includes (i) data collection requirements indicating types of data for the devices in the monitoring group to provide as part of the monitoring program and (ii) attributes associated with the devices in the monitoring group;
based on the attributes associated with the devices, classifying, by the one or more computers, the devices in the monitoring group into different groups corresponding to the categories;
monitoring, by the one or more computers, performance of the devices in the different groups with respect to the data collection requirements for the monitoring program;
evaluating, by the one or more computers, the performance of the different groups to determine a performance score for each group, wherein the performance score for a group is based on levels of compliance with the data collection requirements by the devices in the group; and
adjusting, by the one or more computers, administration of the monitoring program for the devices based on the performance scores for the groups, the adjusting comprisi

Assignees

• Vignet Inc

Inventors

• Jain Praduman
• Schilling Josh
• Klein Dave

Classifications

• H04L43/50 Primary

  Testing arrangements · CPC title

• H04L43/065 Primary

  related to network devices · CPC title

• H04L43/04

  Processing captured monitoring data, e.g. for logfile generation · CPC title

• H04L41/22

  comprising specially adapted graphical user interfaces [GUI] · CPC title

• H04L41/0893

  Assignment of logical groups to network elements · CPC title