Untangling Performance Test Terminology

Confused about the subtle differences between a stress test, performance test or a load test? Don’t get lost in the technobabble! Altentee uses the following terminology to help describe the different types of tests we conduct:

  1. Shakeout – single script, single user, low volume scenarios whose main purpose is to confirm a correctly configured test harness (script, parameters, data, correlation, iterations) and environment.
  2. Baseline – arbitrary user/volumes (typically low with multiple iterations) used to establish a baseline or reference point for further testing. Can also be used to explore/set SLAs if they have not been defined by the business using minimum sample sizes appropriate to establish a baseline.
  3. Load / Volume – often expressed in terms of intended production load at 100% or variations of depending on the desired test outcome. Can also incorporate growth scenarios e.g. 200%. Can also be structured to incrementally increase and sustain load up to a target so that utilization metrics can be collected for capacity modelling at each increment (e.g. ramp up from 100 to 200 users at increments of 20 additional users, with sustained execution of 20 minutes at each increment).
  4. Stress / Stress to Break – loads greater than intended production loads or with the specific purpose to identify component failure or ‘break’ points. Other variants include Surge / Spike testing simulating peak demands over shorter timeframes.
  5. Soak / Endurance – long running tests to establish performance over longer timeframes and any anomalies over time.
  6. Failover / Availability – targeted tests under load in failover conditions. Can also be used to test disaster recovery or availability scenarios.
  7. Component Based – Targeted tests designed to isolate and examine components (functional and/or specific technologies/platforms) under load.
  8. Penetration / Security – with the increasing inclusion of security related non-functional requirements, targeted load tests which focus on application level and/or hardware level security NFRs.
  9. Tuning – cyclic / iterative performance testing using any number of variables in a process of test, tune, retest and compare (for improvement).

Still confused? Contact Altentee and we’ll help explain our approach.

Read More

Using bees to find bottlenecks

On a basic level, the honeybee’s dilemma is a tale of two flower patches. If one patch is yielding better nectar than the other, how can the hive use its workforce most efficiently to retrieve the best supply at the moment? The solution, which earned Austrian zoologist Karl von Frisch a Nobel Prize, is a communication system called the waggle dance.

Saw this on Slashdot … The waggle dance

Read More

Risk management for dummies

This video is a play on Pascal’s Wager which really gets you thinking on possible outcomes for a given scenario being climate change … Is it a better bet to accept doomsday predictions for climate changes, or rest on our haunches and let the world pass by? If you haven’t thought about it yet, get onto it. Good find Ted, pass the word and do your bit … :)

Read More

Understanding the effect of MQ persistence on disk performance

Recently I have been trying to determine what the impact of using MQ message persistence is on disk subsystem performance. There is alot of literature from IBM recommending ideal configurations to support MQ persistence, so I won’t turn this into a post that recommends ideal settings. What I did want to achieve though, was to get a better understanding of how MQ interacts with the disk subsystem when using message persistence, so that I could plan for capacity and make better recommendations for tuning on a production SAN environment.

Read More

Problem solving in general and MQ 2195 reason codes

So I spent the best part of a day assisting our sys admin and developer resolve MQ errors that a COBOL client was throwing when opening more than one concurrent connection to a clustered queue manager.

Because the architecture had recently changed to a cluster, and my load harness was no longer getting the desired response times, we all went down the track of investigating issues with clustering as this was the new kid on the block. Whilst the developer and sys admin revealed many other interesting tid bits of information regarding correct configuration and the like, we still could not collectively resolve the problem: the COBOL code was throwing a 2195 reason code on multiple MQCONN calls to the queue manager.

Read More