Evaluating user contribution in collaborative environments

Patent metadata

Field	Value
Publication number	US-10546509-B2
Application number	US-201715794439-A
Country	US
Kind code	B2
Filing date	Oct 26, 2017
Priority date	Oct 26, 2017
Publication date	Jan 28, 2020
Grant date	Jan 28, 2020

Abstract

Methods, systems and computer program products for evaluating contributions in collaborative environments are provided herein. A computer-implemented method includes obtaining indications of tasks of a given workload that are completed by respective ones of a plurality of client devices. The method also includes identifying first and second tasks completed by first and second client devices at first and second times, determining whether the first task provides a clue for the second task and whether the second time is within a designated time window starting after the first time, and analyzing eye-gaze information to determine whether a second user of the second client device observed a first user of the first client device performing the first task. The method further comprises modifying contribution scores for the first and second users responsive to the determinations.

First claim

What is claimed is:

1. A computer-implemented method, comprising:
obtaining, from collaborative applications running on a plurality of client devices, indications of tasks of a given workload that have been completed by respective ones of the plurality of client devices;
identifying (i) a first task of the given workload completed by a first one of the plurality of client devices at a first time and (ii) a second task of the given workload completed by a second one of the plurality of client devices at a second time subsequent to the first time;
determining whether the first task provides a clue for completing the second task;
determining whether the second time is within a designated time window starting after the first time;
analyzing eye-gaze pattern information to determine whether a second user of the second client device observed a first user of the first client device performing the first task; and
modifying contribution scores for the first user and the second user related to the given workload responsive to determining: (i) that the first task provides a clue for completing the second task; (ii) that the second time is within the designated time window; and (iii) that the second user observed the first user performing the first task;
wherein the steps are carried out by at least one computing device.

2. The method of claim 1 , wherein determining whether the first task provides a clue for the second comprises utilizing a clue dependency graph for the given workload, the clue dependency graph comprising nodes representing the tasks of the given workload and directed edges connecting the tasks of the given workload to one another.

3. The method of claim 2 , wherein a given directed edge connecting the first task to the second task indicates that the first task provides a clue for execution of the second task, and wherein a weight assigned to the given directed edge represents obviousness of the clue.

4. The method of claim 1 , wherein determining whether the second time is within the designated time window comprises utilizing a time dependency graph, the time dependency graph comprising nodes representing tasks and directed edges connecting the tasks of the given workload to one another.

5. The method of claim 4 , wherein a given directed edge connecting the first task to the second task indicates that the second task is performed subsequent to the first task, and wherein a weight assigned to the given directed edge represents the designated time window.

6. The method of claim 1 , comprising tracking user confusion state changes as the plurality of client devices perform the tasks of the given workload, wherein the user confusion state changes are utilized in determining whether the second user of the second client device observed the first user of the first client device performing the first task.

7. The method of claim 6 , wherein tracking user confusion state changes comprises analyzing interaction patterns of users of the plurality of client devices.

8. The method of claim 6 , wherein tracking user confusion state changes comprises analyzing facial expressions of users of the plurality of client devices.

9. The method of claim 1 , comprising capturing the eye-gaze pattern information via respective image sensors or cameras of the plurality of client devices.

10. The method of claim 1 , wherein modifying contribution scores for the first user and the second user related to the given workload comprises increasing a contribution score for the first user by a first designated amount and decreasing a contribution score for the second user by a second designated amount.

11. The method of claim 10 , wherein at least one of the first designated amount and the second designated amount is based on obviousness of the clue for the second task provided by the first task.

12. The method of claim 10 , wherein at least one of the first designated amount and the second designated amount is based on the length of time between performance of the first task and performance of the second task.

13. The method of claim 1 , wherein actions taken by a user in one of the collaborative applications running on one of the plurality of client devices are real-time replicated in the collaborative applications running on other ones of the plurality of client devices.

14. The method of claim 1 , wherein the plurality of client devices implement a collaborative learning environment, and the given workload comprises a problem to be completed by students utilizing the plurality of client devices.

15. The method of claim 1 , wherein the plurality of client devices implement a collaborative support environment, and the given workload comprises a support ticket to be completed by support representatives utilizing the plurality of client devices.

16. A computer program product, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a device to cause the device to perform steps of:
obtaining, from collaborative applications running on a plurality of client devices, indications of tasks of a given workload that have been completed by respective ones of the plurality of client devices;
identifying (i) a first task of the given workload completed by a first one of the plurality of client devices at a first time and (ii) a second task of the given workload completed by a second one of the plurality of client devices at a second time subsequent to the first time;
determining whether the first task provides a clue for completing the second task;
determining whether the second time is within a designated time window starting after the first time;
analyzing eye-gaze pattern information to determine whether a second user of the second client device observed a first user of the first client device performing the first task; and
modifying contribution scores for the first user and the second user related to the given workload responsive to determining: (i) that the first task provides a clue for completing the second task; (ii) that the second time is within the designated time window; and (iii) that the second user observed the first user performing the first task.

17. The computer program product of claim 16 , the program further causing the device to perform the step of tracking user confusion state changes as the plurality of client devices perform the tasks of the given workload, wherein the user confusion state changes are utilized in determining whether the second user of the second client device observed the first user of the first client device performing the first task.

18. A system comprising:
a memory; and
at least one processor coupled to the memory and configured for:
obtaining, from collaborative applications running on a plurality of client devices, indications of tasks of a given workload that have been completed by respective ones of the plurality of client devices;
identifying (i) a first task of the given workload completed by a first one of the plurality of client devices at a first time and (ii) a second task of the given workload completed by a second one of the plurality of client devices at a second time subsequent to the first time;
determining whether the first task provides a clue for completing the second task;
determining whether the second time is within a designated time window starting after the first time;
analyzing eye-gaze pattern information to determine whether a second user of the second client device observed a first user of the first client

Assignees

• IBM

Inventors

• Dey Prasenjit
• Ekambaram Vijay
• Sinha Aditya
• Sukhwani Mohak

Classifications

• G06F18/29

  Graphical models, e.g. Bayesian networks · CPC title

• G09B5/02

  with visual presentation of the material to be studied, e.g. using film strip · CPC title

• G09B19/0069

  Engineering, e.g. mechanical, electrical design (computer engineering G09B19/0053) · CPC title

• G09B7/02 Primary

  of the type wherein the student is expected to construct an answer to the question which is presented or wherein the machine gives an answer to the question presented by a student · CPC title

• H04L67/12

  specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks · CPC title