System and method for automated testing of user interface software for visual responsiveness

Patent metadata

Field	Value
Publication number	US-10114733-B1
Application number	US-201615249974-A
Country	US
Kind code	B1
Filing date	Aug 29, 2016
Priority date	Aug 29, 2016
Publication date	Oct 30, 2018
Grant date	Oct 30, 2018

Abstract

A benchmark test system captures and records root, or input, behavior from a user input device as one or more time-displaced samples of input. The system also separately captures and records the canvas, or visual, behavior of a user interface in response to the captured input as a series of time-displaced image frames. The image frames are analyzed for visual prompts occurring responsive to the input, and parameters of the image frames are determined. A parametric difference between corresponding ones of the root events and canvas responses is thereby computed, in order to determine a degree of visual responsiveness for the user interface software respective to the root input.

First claim

What is claimed is:

1. A system for testing user interface software for time lag in actuating a visual prompt responsive to user manipulation of a user input device, the system comprising:
a display unit defining a canvas for displaying image frames;
a root capturing unit executable to capture user actuation of the user input device as at least one root event at a root software level, an operating system operating at the root software level, each root event being captured as a series of time-displaced samples of input device actuation;
a canvas capturing unit executable to capture processing of the root event by the user interface software as a canvas response at a canvas software level, the user interface software operating at the canvas software level for user interaction with an application, the canvas response being captured as a series of time-displaced image frames; and,
a test analysis unit coupled to said root and canvas capturing units, said test analysis unit executable to determine a parametric difference between corresponding ones of the root events and canvas responses, and to determine a degree of visual responsiveness for the user interface software based thereon, said test analysis unit thereby discriminating portions of the time lag introduced at the canvas software level from portions of the time lag introduced at the root software level.

2. The system of claim 1 , wherein the parametric difference corresponds to a response time difference between a start time of the input device actuation and a visual start time of the canvas response.

3. The system of claim 2 , wherein:
each of the canvas image frames includes a time parameter,
the test analysis unit is further executable to determine a pixel intensity of an image frame, and
the test analysis unit determines the visual start time of the canvas response to be the time of a changed canvas image frame having a difference in pixel intensity from that of an initial canvas image frame exceeding a predefined intensity difference threshold, the initial canvas image frame having a time corresponding in time to the start time of the input device actuation.

4. The system of claim 2 , wherein:
each of the canvas image frames includes a time parameter,
the test analysis unit is further executable to detect a predefined response image template within an image frame, and
the test analysis unit determines the visual start time of the canvas response to be the time of a responding canvas image frame containing the predefined response image template.

5. The system of claim 2 , wherein:
each of the canvas image frames includes a time parameter,
the canvas capturing unit captures the processing of the root event for the duration of a predefined time threshold; and
the degree of visual responsiveness is determined to be below a benchmark responsiveness if the visual start time of the canvas response is determined not to be a time of any image frame of the series of time-displaced image frames.

6. The system of claim 1 , wherein:
each of the root event samples includes a time parameter and a coordinate location parameter,
each of the canvas image frames includes a time parameter, thereby corresponding in time with one of the root event samples, and
the parametric difference corresponds to at least one response distance between the coordinate location of a root event sample and a coordinate location of a visual response in a canvas image frame corresponding in time to the root event sample.

7. The system of claim 6 , wherein:
the test analysis unit is further executable to detect a coordinate location of a predefined response image template within an image frame, and
the test analysis unit determines the coordinate location of the visual response in the image frame to be the image template coordinate location.

8. The system of claim 6 , wherein:
the test analysis unit includes a composite image generator executable to generate composite image frames each displaying one of the canvas image frames and a root sample signifier, the root sample signifier displayed at the coordinate location of the root event sample corresponding in time with the canvas image frame, the composite image frame having a time parameter corresponding with the time parameter of the canvas image frame,
the test analysis unit is further executable to detect a predefined response image template within an image frame, and to detect the root sample signifier within an image frame, and
the test analysis unit determines the response distance of a root event sample as a pixel distance, in the composite image frame corresponding in time with the root event sample, between the predefined response image template and the root sample signifier.

9. The system of claim 6 , wherein the degree of visual responsiveness is determined to be below a benchmark responsiveness if the response distance exceeds a predefined threshold distance for at least a predefined threshold number of image frames.

10. The system of claim 6 , wherein the degree of visual responsiveness is determined to be below a benchmark responsiveness if the response distance exceeds a predefined threshold distance in each canvas image frame of a consecutive subseries of the series of canvas image frames where the consecutive subseries is longer than a predefined threshold number.

11. The system of claim 1 , wherein:
each of the root event samples includes a time parameter and a coordinate location parameter,
each of the canvas image frames includes a time parameter, thereby corresponding in time with one of the root event samples, and
the parametric difference corresponds to a number or ratio of canvas image frames for which an expected visual response of a root event sample was not present in a canvas image frame corresponding in time to the root event sample, for at least a subset of the captured root event samples.

12. A benchmark test system for testing user interface software for time lag in actuating a visual prompt responsive to user manipulation of a user input device, the system comprising:
a display unit defining a canvas for displaying image frames;
an input event recorder executable to record user actuation of the user input device as at least one input event at a root software level, an operating system operating at the root software level, each input event being recorded as a series of time-displaced samples of input device actuation;
an interface response recorder executable to record a processing of the root input event by the user interface software as an interface response at a canvas software level, the user interface software operating at the canvas software level for user interaction with an application, the interface response being recorded as a series of time-displaced image frames; and,
a test analysis unit coupled to said input event and interface response recorders, said test analysis unit executable to compute at least one of a response time difference and a response distance between corresponding ones of the input event samples and interface response image frames, and to determine a degree of visual responsiveness for the user interface software based thereon, said test analysis unit thereby discriminating portions of the time lag introduced at the canvas software level from portions of the time lag introduced at the root software level.

13. A method of testing user interface software for time lag in actuating a visual prompt responsive to user manipulation of a user input device, the method comprising:
recording user actuation of the user input device as at least one root event at a root software level, an operating system operat

Assignees

• Cadence Design Systems Inc

Inventors

• Varghese David
• Saxena Mohit
• Sharma Anshul
• Ginetti Arnold Jean-Marie Gustave

Classifications

• G06T7/90

  Determination of colour characteristics · CPC title

• G06F9/451

  Execution arrangements for user interfaces · CPC title

• G06T2200/24

  involving graphical user interfaces [GUIs] · CPC title

• G06F11/3672

  Test management · CPC title

• G06T2207/10024

  Color image · CPC title