Systems and methods for automated web performance testing for cloud apps in use-case scenarios

Patent metadata

Field	Value
Publication number	US-2018253373-A1
Application number	US-201715446152-A
Country	US
Kind code	A1
Filing date	Mar 1, 2017
Priority date	Mar 1, 2017
Publication date	Sep 6, 2018
Grant date	—

Abstract

Systems and methods for measuring performance metrics of apps where a controller schedules performance testing of a plurality of apps to generate a set of performance metrics from a client, server and device relating to performance of each app wherein the generated set of performance metrics comprises processing times and requests of the app. The scheduled performance testing is executed by a combination of the client, server, and device includes different networks, operating systems, and browsers. A performance engine captures the set of performance metrics of each app from the different client, server and device, and organizes the app metrics into categories based on an instrumentation and profile of each app. The categories include clusters comprising performance metrics of the client, server, and device. A user interface renders the set of performance metrics to facilitate comparisons between each cluster and category of the set of performance metrics.

First claim

1 . An automated testing system for measuring performance metrics of apps, comprising:
at least one processor deployed on a server being programmed to implement the measuring of metrics for the performance testing by the server of a plurality of app coupled to the server; the at least one processor scheduling performance testing of a plurality of apps to generate a set of performance metrics from a client, server and device relating to performance of each app wherein the performance metrics comprises processing times and requests associated with the app, wherein the scheduling performance testing of each app is executed by a combination of the client, server, and device comprising different networks, operating systems, and browsers; the at least one processor having a performance engine to capture the set of performance metrics of each app from the client, server and device, and further for organizing the set of performance metrics into categories based on an instrumentation and profile of each app wherein the categories comprise clusters of the performance metrics; the at least one processor having a graphic user interface for rendering the set of performance metrics in a manner to facilitate comparisons between each cluster and category of the set of performance metrics; and the at least one processor having an input to receive data from processors of the client, server, and network to present in a first instance in a waterfall graph of event logging and processing time for events across the client, server and network; and in a second instance to present in a flame graph for discrete events of creation time for each component on a page whereby a bottleneck in an event and component processing is identified visually at an event level across the client, server and network and at a component level across components of the page related to the client, server and network by a particular display of the waterfall and flame graphs.

2 . The system of claim 1 , wherein the generated set of performance metrics from the networks comprise performance metrics related to a plurality of different network protocols.

3 . The system of claim 1 , wherein the controller is configured to implement a set of instructions to download the plurality of apps, load pages and capture performance metrics for each page of the app across a plurality of scenarios from the server to each client using different combinations of the network, operating system and browser.

4 . The system of claim 1 , wherein the performance testing comprises testing on different sets of at least two browsers.

5 . The system of claim 1 , wherein the performance testing comprises testing on sets of at least two operating systems.

6 . The system of claim 1 , further comprising:
a database which stores the set of performance metrics which is captured and enables the controller to retrieve the set of performance metrics for automated analysis and trending by the performance engine to determine an improvement or regression of a feature of an app through an app development cycle.

7 . The system of claim 1 , further comprising:
a plurality of types of performance metrics provided of the client, server and network at least comprising: client metrics of page loading and rendering time, component loading and rendering time, and client processor profiles; server metrics of server processing time, request time of each layer of a server, number of calls and latency time for app services to a server, application program interfaces (APIs) executions at a server, and server database structured query language (SQL) statements; and network metrics of network requests, time in a network layer, and network navigation timing API metrics.

8 . A method for measuring performance metrics of apps during an app development cycle, the method comprising:
implementing performance testing by a server of a plurality of apps coupled to the server by programming a processor of the server to schedule performance testing of a plurality of apps to generate a set of performance metrics from a client, server and network relating to performance of each app wherein the set of performance metrics comprises processing and request times associated with the app, wherein the scheduling performance testing of each app is executed from a processor of the server for a combination of clients, servers, and networks comprising different devices, operating systems, and browsers; capturing by a performance engine the set of performance metrics of each app from the clients, servers and networks, and further organizing the set of performance metrics into categories based on an instrumentation and profile of each app wherein the categories comprise clusters of the performance metrics; and rendering by a user interface the set of performance metrics in a manner to facilitate comparisons between each cluster and category of the set of performance metrics for automated analysis and trending by the performance engine to determine an improvement or regression of a feature of an app through the app development cycle.

9 . The method of claim 8 , wherein the generated set of metrics from the networks comprises performance metrics related to a plurality of different network protocols.

10 . The method of claim 8 , wherein the controller is configured to implement a set of instructions to download the app, load pages and capture performance metrics for each page of the app across a plurality of scenarios from the server to each client using a different combinations of the network, operating system and browser.

11 . The method of claim 8 , wherein the performance testing comprises testing on different sets of at least two browsers.

12 . The method of claim 8 , wherein the performance testing comprises testing on sets of at least two operating systems.

13 . The method of claim 8 , further comprising:
storing by a database the set of performance metrics which is captured and enabling the controller to retrieve the set of performance metrics for automated analysis and trending by the performance engine to determine an improvement or regression of a feature of an app through an app development cycle.

14 . The method of claim 8 , further comprising:
providing a plurality of types of performance metrics of the client, server and network at least comprising: client metrics of page loading and rendering time, component loading and rendering time, and client processor profiles; server metrics of server processing time, request time of each layer of a server, and number of calls and time taken for app services, APIs executions at a server, and server database SQL statements; and network metrics of requests, time in a network layer, and network navigation timing API metrics.

15 . A system comprising:
at least one processor; and at least one computer-readable storage device comprising instructions that when executed causes execution of a method of cloud automated testing for measuring performance metrics of apps during an app development cycle, the method comprising: configuring a first module to connect a plurality of clients to an app server running the app; configuring a second module to implement a plurality of operating systems; configuring a third module to implement a plurality of web browsers; configuring a fourth module to implement a plurality of network connections; downloading the app, and loading pages and capturing performance metrics for each page of the downloaded app across a plurality of scenarios from the app server to each client using a different combinations of the plurality of operating systems, web brows

Assignees

• Salesforce Com Inc

Inventors

• Mathur Ashraya R

Classifications

• G06F11/3409

  for performance assessment · CPC title

• G06F11/32

  with visual {or acoustical} indication of the functioning of the machine · CPC title

• G06F11/3404

  for parallel or distributed programming · CPC title

• G06F11/3476

  Data logging (G06F11/14, G06F11/2205 take precedence) · CPC title

• G06F11/34

  Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation {; Recording or statistical evaluation of user activity, e.g. usability assessment} · CPC title