Automatically capturing user interactions and evaluating user interfaces in software programs using field testing

Patent metadata

Field	Value
Publication number	US-9836193-B2
Application number	US-201313968872-A
Country	US
Kind code	B2
Filing date	Aug 16, 2013
Priority date	Aug 16, 2013
Publication date	Dec 5, 2017
Grant date	Dec 5, 2017

Abstract

A method includes analyzing, on a first computing device, data from second computing device(s) of user interaction with a user interface of an application previously executed on the second computing device(s). The data corresponds to events caused by the user interaction with the user interface of the application. The first computing device generates representation(s) of the analyzed data and outputs the representation(s) of the user interaction. Another method includes capturing and logging, by a computing device, events caused by user interaction with a user interface of an application when the application is executed on the computing device. In response to a trigger, data comprising the captured and logged events is sent toward another computing device. Another method includes instrumenting a measurement library into an application to create an instrumented version of the application, and sending the instrumented application to computing device(s). Methods, apparatus, software, and computer program products are disclosed.

First claim

What is claimed is:

1. A method, comprising:
analyzing, on a first computing device, data from at least one second computing device of user interaction with a plurality of views of a user interface of an application, the data corresponding to events caused by the user interaction with the views of the user interface of the application and comprising at least one screenshot for each of the views of the application captured by the at least one second computing device, wherein the data was previously collected by the at least one second computing device in response to the at least one second computing device executing an instrumented version of the application, wherein the analyzing further comprises determining tracked touch interaction with specific user interface controls for the views of the user interface of the application that correspond to each of the screenshots, and determining paths of interaction between the screenshots of the user interface and a frequency of each of the paths of interaction, and wherein the first computing device and the at least one second computing devices are different;
generating by the first computing device one or more representations of the analyzed data, wherein the generating comprises generating an indication for each of the determined paths of interaction between the screenshots of the user interface and an indication of the frequency of each of the paths, and wherein the generating comprises generating a set of the screenshots, wherein each respective screenshot in the set comprises an indication of the tracked touch interaction for the view corresponding to the respective screenshot; and
outputting by the first computing device the one or more representations of the user interaction with the user interface of the application,
wherein at least one of the events corresponds to a user interaction in an area of the user interface having no user interface control.

2. The method of claim 1 , wherein the events comprise one or more of short presses on a touchscreen, long presses on a touchscreen, or swipes on the touchscreen.

3. The method of claim 1 , wherein:
the screenshots are determined to exist in a timeline; and
generating further comprises generating the set of the screenshots to follow the timeline.

4. The method of claim 1 , wherein:
the screenshots are determined to have been performed relative to a single view of the user interface of the application; and
generating further comprises generating the set of the screenshots that emanate from the single view.

5. The method of claim 1 , wherein:
analyzing further comprises determining quantitatively measured usage interaction for an individual component within a given view of the user interface; and
generating further comprises generating indications of the quantitatively measured usage interaction for the individual component within the given view of the user interface.

6. The method of claim 1 , wherein the instrumented version of the application comprises the application and a measurement library for collecting the data corresponding to the events caused by the user interaction with the user interface of the application.

7. The method of claim 1 , wherein the data comprises a plurality of entries, each entry corresponding to a given one of the events and comprising at least:
a timestamp for the given event;
either the captured screenshot or a reference to the captured screenshot corresponding to the given event; and
absolute coordinates associated with the given event.

8. A computing device, comprising:
a memory comprising computer-readable code; and
one or more processing units, wherein the one or more processing units and the memory are configured in response to execution of the computer-readable code to cause the computing device to perform the following:
analyzing, on the computing device, data from at least one other computing device of user interaction with a plurality of views of a user interface of an application, the data corresponding to events caused by the user interaction with the views of the user interface of the application and comprising at least one screenshot for each of the views of the application captured by the at least one other computing device, wherein the data was previously collected by the at least one other computing device in response to the at least one other computing device executing an instrumented version of the application, wherein the computing device and the at least one other computing devices are different, wherein the analyzing further comprises determining tracked touch interaction with specific user interface controls for the views of the user interface of the application, and determining paths of interaction between the screenshots of the user interface and a frequency of each of the paths of interaction;
generating by the computing device one or more representations of the analyzed data, wherein the generating comprises generating an indication for each of the determined paths of interaction between the screenshots of the user interface and an indication of the frequency of each of the paths, and wherein the generating comprises generating a set of the screenshots, wherein each respective screenshot in the set comprises an indication of the tracked touch interaction for the view corresponding to the respective screenshot; and
outputting by the computing device the one or more representations of the user interaction with the user interface of the application,
wherein at least one of the events corresponds to a user interaction in an area of the user interface having no user interface control.

9. The method of claim 7 , wherein a first one of the entries corresponds to a first event and comprises a first screenshot, and a second one of the entries corresponds to a second event that occurred later than the first event and comprising a reference to the first screenshot, such that the first event corresponds to the user interaction in the area of the user interface having no user interface control.

Assignees

• IBM

Inventors

• Ligman Joseph W
• Pistoia Marco
• Ponzo John
• Thomas Gegi

Classifications

• G06F11/3672 Primary

  Test management · CPC title

• G06F3/04842 Primary

  Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence) · CPC title

• G06F3/0488

  using a touch-screen or digitiser, e.g. input of commands through traced gestures · CPC title

• G06F3/0484

  for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range · CPC title

• G06F11/3664

  Physics · mapped topic