Jason Gorman describes in his article 6 requirements for the practical use of dashboards in performance related testing.
As a performance tester, I find that I often need to provide snapshot summaries of systems performance ala dashboards. The key requirement imposed is normally timeliness and relevance of data being presented.
So sticking with Jason’s requirements, how do you implement good dashboard design?
Read MoreIIS logs if setup properly can be an extremely useful source of information during load testing and for other activities such as capacity planning. Another load tester sings its praise here and provides good justification of why you should use it.
The log parser comes for free with the IIS resource pack but you can also get it individually here.
Read MoreI often use a combination of cygwin with ssh to remotely manage windows servers, as I find this to be a quicker (and hopefully safer) method of access to my windows boxes. To that end, I often need to check windows application event logs. Typically you’d use the mmc, but all I want to do mostly is check the last 10 or 100 entries for things like break in attempts, or application warnings/failures etc.
Read MoreWhen load testing MQ you are no doubt going to need to be able to monitor queue statistics in terms of how many messages have been enqueued and dequeued within a given timeframe. You can use native runmqsc commands to query queues in order to find current queue depths but this is just a snapshot in time. In order to be able to monitor how many messages have been or gone, you need to sample the queue over time. Here is a good synopsis of options…
A good way to achieve this is my using the PCF support package provided by IBM.
The WebSphere MQ Programmable Command Formats (PCF) provide the capability to perform administration tasks on a queue manager by sending and receiving WebSphere MQ messages of a special format. PCF request messages are sent to the queue manager’s command queue, where they are processed by the command server and replies returned to the designated reply-to queue.
This SupportPac contains a set of Java classes representing PCF header structures as well as an agent that can be used to simplify the task of communicating with a target queue manager and thus enable the use of WebSphere MQ Programmable Command Formats for queue manager administration.
Using a simple java object as documented in sample code, you can query queue stats of your choice. My implementation simply prints out the queue you are connected to, along with enqueue, dequeue and total queue size stats over an arbitrary time frame (eg. last 2 seconds). This can easily be called from a simple shell script.
Read MoreThe screenshot you see is an example of the graphs you can output in my descriptive statistics Excel template
without using any macros. I often find moving averages and boxplots / histograms to be extremely useful in analyzing raw data for stress and volume testing …
So what are my favourite descriptive statistics I like to analyse data with?
1. Box plots and histograms are great for displaying distribution and give you a good insight to characteristics of the data you are analyzing. A standard box-whisker chart plots up to 5 data sets using box-whisker symbols. Variants are box plots without the whiskers. The 5 data sets are sometimes called the maximum, 3rd quartile, median, 1st quartile and minimum, although they can represent any kind of quantities.
2. Trend line charts. A trend line is a straight line that fits a number of data points computed using linear regression (the least square method). This is excellent for predictive analysis of data, especially if you get roped into capacity planning activities. You do however need a signficant empirical baseline from which to draw your results, which usually means you have to execute many test runs in order to establish the data.
3. Good old line charts. A line chart with thinned results (for easier presentation) can give immediate insights to where your transaction response times went wrong. This is good for coarse analysis of rows of data. Some variations of the line chart is the moving average as provided by excel. I’m not sure on the accuracy of moving averages though. Can be useful when trying to smooth the curve.
4. In a spline charts, the data points are connected together using cardinal spline curves (as opposed to straight lines). The “tension” of the curve can normally be configured.
