Systems and methods for identifying anomalous test item renderings

Patent metadata

Field	Value
Publication number	US-9619900-B2
Application number	US-201514928064-A
Country	US
Kind code	B2
Filing date	Oct 30, 2015
Priority date	Oct 31, 2014
Publication date	Apr 11, 2017
Grant date	Apr 11, 2017

Abstract

Systems and methods are provided for determining a test item rendering anomaly. A test item is provided to a first test item rendering system that is configured to generate a first graphical representation of the test item. The test item is provided to a second test item rendering system that is configured to generate a second graphical representation of the test item. A first digital pixel image of the first graphical representation is captured, and a second digital pixel image of the second graphical representation is captured. Pixels of the digital pixel images are processed, where the processing includes comparing the digital pixel images to determine an extent to which the second digital pixel image differs from the first digital pixel image. An indication for the test item is provided when the second digital pixel image differs from the first digital pixel image by more than a threshold amount.

First claim

It is claimed:

1. A computer-implemented method of determining a test item rendering anomaly, comprising:
providing a test item to a processor-based first test item rendering system, the first test item rendering system including a graphics rendering engine and being configured to generate a first graphical representation of the test item;
providing the test item to a second processor-based test item rendering system, the second test item rendering system including a graphics rendering engine and being configured to generate a second graphical representation of the test item;
capturing, using a computer processing system, a first digital pixel image of the first graphical representation and a second digital pixel image of the second graphical representation;
processing pixels of the first and second digital pixel images using the computer processing system, wherein the processing includes using an automated digital image analysis algorithm, the processing including comparing the first digital pixel image to the second digital pixel image, the algorithm generating a numerical metric that indicates an extent to which the second digital pixel image differs from the first digital pixel image; and
providing an anomaly indication for the test item when the numerical metric indicates that the second digital pixel image differs from the first digital pixel image by more than a threshold amount;
storing metadata for each of the first and second digital pixel images in one or more data structures, the metadata including data that identifies the test item, the rendering system used in generating the digital pixel image, and a date on which the digital pixel image was captured; and
automatically selecting the first and second digital pixel images for comparison based on the metadata.

2. The method of claim 1 , wherein the comparing of the first digital pixel image to the second digital pixel image comprises performing a pixel-by-pixel comparison of the first digital pixel image and the second digital pixel image to determine whether the second test item rendering system is rendering the test item differently than the first test item rendering system, the anomaly indication being provided when more than a threshold number of pixels differ between the first digital pixel image and the second digital pixel image.

3. The method of claim 1 , wherein the first and second test item rendering systems are configured to (i) process the test item, and (ii) convert the test item into a form suitable for display on a computer monitor or for printing on paper.

4. The method of claim 1 , wherein the test item is certified as a usable test item based on the first digital pixel image, and wherein a determination of whether the test item retains a certified status is based on the comparing of the first digital pixel image to the second digital pixel image.

5. The method of claim 1 , wherein the anomaly indication is provided via a report, the report indicating that the test item requires adjustment.

6. The method of claim 1 , wherein the anomaly indication is provided via a report, the report indicating that the second test item rendering system requires adjustment.

7. A computer-implemented system of determining a test item rendering anomaly, the system comprising:
one or more processing systems; and
one or more non-transitory computer-readable storage mediums in communication with the one or more processing systems and encoded with instructions for commanding the one or more processing systems to execute steps comprising:
providing a test item to a processor-based first test item rendering system, the first test item rendering system including a graphics rendering engine and being configured to generate a first graphical representation of the test item;
providing the test item to a second processor-based test item rendering system, the second test item rendering system including a graphics rendering engine and being configured to generate a second graphical representation of the test item;
capturing a first digital pixel image of the first graphical representation and a second digital pixel image of the second graphical representation;
processing pixels of the first and second digital pixel images, wherein the processing includes using an automated digital image analysis algorithm, the processing including comparing the first digital pixel image to the second digital pixel image, the algorithm generating a numerical metric that indicates an extent to which the second digital pixel image differs from the first digital pixel image; and
providing an anomaly indication for the test item when the numerical metric indicates that the second digital pixel image differs from the first digital pixel image by more than a threshold amount;
storing metadata for each of the first and second digital pixel images in one or more data structures, the metadata including data that identifies the test item, the rendering system used in generating the digital pixel image, and a date on which the digital pixel image was captured; and
automatically selecting the first and second digital pixel images for comparison based on the metadata.

8. The system of claim 7 , wherein the comparing of the first digital pixel image to the second digital pixel image comprises performing a pixel-by-pixel comparison of the first digital pixel image and the second digital pixel image to determine whether the second test item rendering system is rendering the test item differently than the first test item rendering system, the anomaly indication being provided when more than a threshold number of pixels differ between the first digital pixel image and the second digital pixel image.

9. The system of claim 7 , wherein the first and second test item rendering systems are configured to (i) process the test item, and (ii) convert the test item into a form suitable for display on a computer monitor or for printing on paper.

10. The system of claim 7 , wherein the test item is certified as a usable test item based on the first digital pixel image, and wherein a determination of whether the test item retains a certified status is based on the comparing of the first digital pixel image to the second digital pixel image.

11. The system of claim 7 , wherein the anomaly indication is provided via a report, the report indicating that the test item requires adjustment.

12. The system of claim 7 , wherein the anomaly indication is provided via a report, the report indicating that the second test item rendering system requires adjustment.

13. A non-transitory computer-readable storage medium for determining a test item rendering anomaly, the computer-readable storage medium comprising computer executable instructions which, when executed, cause a processing system to execute steps comprising:
providing a test item to a processor-based first test item rendering system, the first test item rendering system including a graphics rendering engine and being configured to generate a first graphical representation of the test item;
providing the test item to a second processor-based test item rendering system, the second test item rendering system including a graphics rendering engine and being configured to generate a second graphical representation of the test item;
capturing a first digital pixel image of the first graphical representation and a second digital pixel image of the second graphical representation;
processing pixels of the first and second digital pixel images, wherein the processing includes using an automated digital image analysis algorithm, the processing including comparing the first digital pixel image to the second digi

Assignees

• Educational Testing Service

Inventors

• Lee Mankit
• Merkushen Roman
• Thakker Ketan

Classifications

• G06T11/00 Primary

  Two-dimensional [2D] image generation · CPC title

• G06T7/0002 Primary

  Inspection of images, e.g. flaw detection · CPC title

• G06T2207/30168

  Image quality inspection · CPC title