Cross-classroom and cross-institution item validation

Patent metadata

Field	Value
Publication number	US-9508266-B2
Application number	US-201313871455-A
Country	US
Kind code	B2
Filing date	Apr 26, 2013
Priority date	Apr 27, 2012
Publication date	Nov 29, 2016
Grant date	Nov 29, 2016

Abstract

Anonymous pretesting items for subsequent presentation to participants in a group enable an instructor to validate responses and revise the items accordingly.

First claim

What is claimed is:

1. A server for pretesting items for subsequent presentation to participants in a target class, the system comprising:
a processor; and
a computer readable medium comprising instructions to perform the operations comprising:
transmitting, by the server, an item to devices associated with a potential set of students, wherein the item is presented by the devices associated with the potential set of students;
receiving first student input data from at least some of the devices associated with the potential set of students, wherein the first student input data is received in response to presenting the item to the potential set of students;
receiving at least one value of a characteristic associated with students in the potential set of students, wherein the characteristics corresponds with a demographic characteristic, a behavioral characteristic, or a performance characteristic of the students;
pruning, by the server, at least one of the first student input data received in response to presenting the item from the potential set of students to form a composite class of remaining students, wherein the composite class of remaining students comprises the first student input data and the composite class of remaining students is associated with a value matching the at least one value of the characteristic;
transmitting a post task question to at least some of the devices associated with the composite class of remaining students;
receiving second student input data in response to the post task question;
creating guidance on revising the item based at least in part on the first student input data or second student input data received from the devices associated with the composite class of remaining students; and
transmitting, by the server, the guidance for revising the item to an instructor device associated with the target class.

2. The server of claim 1 , wherein success of the item is assessed by determining a number or percentage of members of the composite class of remaining students answered the item correctly.

3. The server of claim 1 , wherein success of the item is assessed by a number or percentage of received input that indicates at least some understanding of material covered by the item.

4. The server of claim 1 , the operations further comprising:
determining, by the server, that the characteristics associated with the composite class of remaining students correspond with the characteristics of the target class.

5. The server of claim 1 , wherein the revised item is automatically transmitted to the instructor device associated with the target class.

6. The server of claim 1 , wherein the pruning implements a nonparametric matching algorithm.

7. The server of claim 1 , wherein the pruning implements a coarsened exact matching algorithm.

8. A method for pretesting items for subsequent presentation to participants in a target class, the method comprising:
transmitting, by a computer system, an item to devices associated with a potential set of students, wherein the item is presented by the devices associated with the potential set of students;
receiving first student input data from at least some of the devices associated with the potential set of students, wherein the first student input data is received in response to presenting the item to the potential set of students;
receiving at least one value of a characteristic associated with students in the potential set of students, wherein the characteristics corresponds with a demographic characteristic, a behavioral characteristic, or a performance characteristic of the students;
pruning, by the computer system, at least one of the first student input data received in response to presenting the item from the potential set of students to form a composite class of remaining students, wherein the composite class of remaining students comprises the first student input data and the composite class of remaining students is associated with a value matching the at least one value of the characteristic;
transmitting a post task question to at least some of the devices associated with the composite class of remaining students;
receiving second student input data in response to the post task question;
creating guidance on revising the item based at least in part on the first student input data or second student input data received from the devices associated with the composite class of remaining students; and
transmitting, by the computer system, the guidance for revising the item to an instructor device associated with the target class.

9. The method of claim 8 , wherein success of the item is assessed by determining a number or percentage of members of the composite class of remaining students answered the item correctly.

10. The method of claim 8 , wherein success of the item is assessed by a number or percentage of received input that indicates at least some understanding of material covered by the item.

11. The method of claim 8 , further comprising:
determining, by the server, that the characteristics associated with the composite class of remaining students correspond with the characteristics of the target class.

12. The method of claim 8 , wherein the revised item is automatically transmitted to the instructor device associated with the target class.

13. The method of claim 8 , wherein the pruning implements a nonparametric matching algorithm.

14. The method of claim 8 , wherein the pruning implements a coarsened exact matching algorithm.

15. A non-transitory computer readable medium storing a plurality of instructions that when executed control a computer system to pretest items for subsequent presentation to participants in a target class, the instructions comprising:
transmitting, by a computer system, an item to devices associated with a potential set of students, wherein the item is presented by the devices associated with the potential set of students;
receiving first student input data from at least some of the devices associated with the potential set of students, wherein the first student input data is received in response to presenting the item to the potential set of students;
receiving at least one value of a characteristic associated with students in the potential set of students, wherein the characteristics corresponds with a demographic characteristic, a behavioral characteristic, or a performance characteristic of the students;
pruning, by the computer system, at least one of the first student input data received in response to presenting the item from the potential set of students to form a composite class of remaining students, wherein the composite class of remaining students comprises the first student input data and the composite class of remaining students is associated with a value matching the at least one value of the characteristic;
transmitting a post task question to at least some of the devices associated with the composite class of remaining students;
receiving second student input data in response to the post task question;
creating guidance on revising the item based at least in part on the first student input data or second student input data received from the devices associated with the composite class of remaining students; and
transmitting, by the computer system, the guidance for revising the item to an instructor device associated with the target class.

16. The computer readable medium of claim 15 , wherein success of the item is assessed by determining a number or percentage of members of the composite class of remaining students answered the item correctly.

Assignees

• Harvard College

Inventors

• King Gary
• Lukoff Brian
• Mazur Eric

Classifications

• G09B7/02

  of the type wherein the student is expected to construct an answer to the question which is presented or wherein the machine gives an answer to the question presented by a student · CPC title

• G09B5/02

  with visual presentation of the material to be studied, e.g. using film strip · CPC title

• G09B5/00

  Electrically-operated educational appliances (working with questions and answers G09B7/00; simulators G09B9/00; advertising or displaying in general G09F) · CPC title

• G09B7/00 Primary

  Electrically-operated teaching apparatus or devices working with questions and answers (mechanically operated G09B3/00; computing arrangements G06F) · CPC title

• G09B5/04

  with audible presentation of the material to be studied (sound recording or reproducing G11B) · CPC title