Performance testing metrics provide quantitative results which helps in analysis and resolution of the performance bottlenecks. For the benefit of the performance engineers, the complete web application performance testing picture is sliced into smaller pieces for detailed and discreet monitoring and analysis in AgileLoad. The performance indicators are categorised as - Virtual Users, Transactions , Web Resources, System Resources, Web Servers, Application Servers, Database Servers, Network Components and the Client Side of the App, which map directly to the logical entities involved in accessing and servicing a web application.
AgileLoad supported components include:
- Operating Systems - Microsoft Windows XP/2000/2003 Server, Unix AIX/SUN Solaris/HP True64/Linux RedHat
- Web and Application Servers - IIS, Apache, BEA WebLogic, IBM WebSphere, JBOSS In case of server monitoring the metrics depend upon the type of web server.
- Database Servers - Oracle, MySQL, Microsoft SQL
- Network Equipment - SNMP-compatible components, as well as ERP including Siebel and SAP
- Client Side - Rich Internet Application based on
, Flex, Silverlight, web-services and more. Ajax
Some of the most useful performance indicators under each category, as captured by AgileLoad are listed below
- Running VUser – No of VUsers currently accessing a system and their state.
- VUser with Error Graph – Indicating the number of users which were unable to complete their actions and errors they faced.
- Average Transaction Response Time - Gives details of the response time of each executed transaction.
- Passed/Failed Transactions – Indicator of how many transactions passed and failed during the performance test.
Web resource metrics provide the web server parameters information captured during the performance test
- Hits per Second - Information of number of HTTP requests sent to the web server during specific time period of the performance test.
- Throughput - Information on server response during specific time period of the performance test.
- HTTP Responses per Seconds - Information of all the HTTP status codes generated per second during the performance test execution.
- Errors per Second - The number different types of errors generated per second.
- Pages Downloaded per Second - Web pages downloaded in every second of the test from the web server.
- Connections per Second - Information on number of new TCP/IP connections opened and shut down during each second of the performance test.
Monitoring of all the machines (Servers and load injectors) and their operating system resources which are being used in performance testing of the application are listed below.
- CPU Usage - Information on machine CPU utilization.
- Memory Usage - Information on amount of free disk space, virtual memory pages moved between main memory and disk storage and percent memory used during the scenario execution.
- Disk Space Usage - Information on percent disk space used.
- Service Monitors - Information on running processes on the machine and the amount of system resources they are using.
List of performance metrics monitored during performance test varies with the type of web server; following is the list of performance counters for Microsoft IIS web server.
- Bytes Sent per Second
- Bytes Received per Second
- Get Requests per Second
- Post Requests per Second
- Maximum Connections
- Not Found (404) Error per Second
- Private Bytes
- Percent of Processor Time
- System Processor Queue Length
- Percent Disk Time
- Queue Length
- Cache Hit Percent
- Request Wait Time
- Request Queued
- Transactions per Second
Every type of application server has its performance counters based on its model. Some of the important Web Sphere application performance counters being monitored by AgileLoad are:
- Memory Free
- Total Memory
- Memory Use
- Bean Destroys
- Active Threads
- Total Threads
- Percent Time Maxed
- Connection Wait Time
- Connection Time
- Connection Percent Used
- Active Transactions
- Transactions Suspended
- Rolled Back
- Servlet Errors
- Sessions Invalidated
Some of the common metrics monitored for all database servers are following,
- Database Transaction Summary - Information of volume of data sent and received by the database server.
- Database Connection Summary - Information of total number of open and close connections on database server during the test.
- Database Thread Summary - Information of number of new threads created, connected and used.
- Database Query Summary - Information of total number of queries read, wrote and deleted by the database server.
Some of the most important performance counters of Microsoft SQL Server are following,
- Transactions per Second
- Log Cache Hit Ratio
- Log Cache Reads per Second
- Log Bytes Flushed per Second
- Log Flush Wait Time
- Log Flush Waits per Second
- Log Flushes per Second
- Percent Log Used
Some of the important network metrics within a network segment which AgileLoad monitors are,
- Bytes per Second Sent to Network Interface Card
- Bytes per Second Received by NIC
- TCP Segments Sent per Second
- TCP Segments Received per Second
- TCP Segments per Second
- TCP Segments Retransmitted per Second
- TCP Connections Failures
- TCP Connections Reset
- TCP Connections Established
Client Side of the App
Some of the important client side metrics being monitored by AgileLoad include
- Images Load Time
- CSS Files Load Time
- HTML Resources Load Time
- DNS Lookup Time
- TCP Connection Time
- SSL Handshake Time
- Time to First Byte
- Content Download Time
- Content Caching Policy
- Availability Content Delivery Network (CDN)
- Resources Rendering Time
- Number of HTTP Requests
- HTTP Response Status
- HTTP Response Size
Each of these performance indicators can be viewed as a list or graph. The performance indicators can be co-related with each other by dragging and dropping them onto a single graph.
The complete review of AgileLoad can be found here.
[Image Credits: AgileLoad]