The BrowserMob interface has a handy findRegexMatches method you can call as follows:

It takes a string input and [...]]]> If you’re using an RBU or VU you may need to extract content from the previous response. For example, enumerate a link to a PDF file for subsequent download.

The BrowserMob interface has a handy findRegexMatches method you can call as follows:

var link = browserMob.findRegexMatches(selenium.getHtmlSource(), 'href="(.+?pdf)"');
browserMob.beginStep("16_download_content");
  browser.get(link[0], 200); 
browserMob.endStep();

It takes a string input and regex pattern parameters. For the string input I am simply passing in the HTML source from the previous request based on a browser object:
var browser = browserMob.getActiveHttpClient();

The regex pattern is self explanatory. The link object will be an array of matches; in this case I’m issuing a get request for the first link in the match array. You might want to add some more code to make this more robust e.g. if no links are found etc.

Pretty simple hey!

Think Time Think time, is normally defined as the amount of time a virtual user ‘thinks’ between individual steps within a transaction. This time is usually excluded from response time measurements [...]]]> If you come from the commercial toolset mindset, you might be interested in how we achieve think time and pacing when using alternatives such as BrowserMob.

Think Time
Think time, is normally defined as the amount of time a virtual user ‘thinks’ between individual steps within a transaction. This time is usually excluded from response time measurements and is an important inclusion in terms of script behaviour. Having no think time means the virtual user will race through transactions more quickly than a normal user would which potentially creates unrealistic load. So how do we emulate think time behaviour in a BrowserMob script?

The BrowserMob replay engine drives Selenium scripts, so you will need to play in the JavaScript sandbox so to speak, in order to emulate think time. There’s some well documented ways to implement this on the BrowserMob blog here.

The BrowserMob interface (API documentation here) has a pause method as follows:
browserMob.pause(15000);

This effectively pauses the user for a period of 15 seconds in the previous example.

The Selenium interface (API documentation here) also has a similar setSpeed method which will set the number of milliseconds to pause after each step elminating the need to specify individual pause statements between each step as follows:
selenium.setSpeed(2000);

If you would like to emulate some common think time settings as you might see in a LoadRunner script you can implement the following at the top of your RBU or VU scripts:

// Multiply think time by:
var think_time_multiple = 1;
 
// Use random percentage for think time e.g. 50 - 100:
var think_time_percentage = 50 + (Math.random() * 100);
 
// Limit think time to:
var think_time_limit = 0; 
 
// Set think time to:
var think_time = 3; //seconds
 
if(think_time_limit > 0) {
  selenium.setSpeed(think_time_limit * 1000);
} else {
  selenium.setSpeed(think_time * think_time_multiple * (think_time_percentage/100) * 1000);
}

If you need additional think time after steps then you can add the following statement where required:
browserMob.pause(3000 * think_time_multiple * (think_time_percentage/100));

Pacing
Some people like to control transaction volume/throughput by messing about with think time. I don’t favour this approach because you need to first be cognisant of how long each iteration will take including think time then adjust think time accordingly. Once it’s set (or hardcoded into your script) then the vusers are stuck with that setting. I much prefer the pacing concept, which determines how long a vuser should ‘wait’ before starting the next iteration. This wait time is determined by elapsed time including any server side processing, not just user think time. So how do we emulate pacing in a BrowserMob script?

First I like to set some targets such as the transaction rate (per hour per user) and then calculate the pacing from this target. I also set a variable iterate to true, which comes into play in the main body of the script.

// Run logic
var transaction_rate = 10;  // Target transaction rate per hour per user
var pacing  = 60*60*1000 / transaction_rate;
var iterate = true;

Now we’ve got the target and pacing set, we can get into the main logic. Let’s start an iteration.

while (iterate) {
  var start = new Date();
  browserMob.beginTransaction();
  ...

Notice I’ve used a while loop which checks for the iterate boolean and creates a start date time stamp. I also initiate a BrowserMob transaction.

After we complete the iteration (which is essentially a bunch of BrowserMob steps) we can close the while loop with some pacing logic.

  ...
  // Calculate Pacing
  var finish  = new Date(); 
  var elapsed = finish.getTime() - start.getTime();
 
  if (browserMob.isValidation()) {
    browserMob.log("Adjusted pacing would be "+ (pacing - elapsed) + " msecs");
    browserMob.log("or " + Math.round(3600000 / (pacing - elapsed) ) + " trans/hour");
    iterate = false;
  } else {
    if(pacing > elapsed)
      browserMob.pause(pacing - elapsed);
  }
 
  browserMob.endTransaction();
}

This will calculate the finish date time stamp and then determine the difference between the previously calculated target pacing and the actual time elapsed. If there is a difference then the user will pause, otherwise it will just continue on to the next iteration. This basically allows you to target a set number of transactions per hour. When validating the script it will output these calculations to the log file.

Google AdWords page quality criteria are many, a prominent check being your landing page load time. A slow load time can lead to a lower quality score.

Make sure you’re not wasting effort on advertising campaigns. Altentee uses BrowserMob to help check website performance for clients from multiple locations. You [...]]]> Can you afford a slow landing page?

Google AdWords page quality criteria are many, a prominent check being your landing page load time. A slow load time can lead to a lower quality score.

Make sure you’re not wasting effort on advertising campaigns. Altentee uses BrowserMob to help check website performance for clients from multiple locations. You can run a free check using the form below.

If you need help optimizing your landing page, or help with performance testing in general feel free to contact Altentee today.

I’m using MySQL’s AES which is better than older standards. Basically 128 bit key encryption …

To begin, let’s create a [...]]]> There’s been a bit of buzz lately where MySQL databases have been compromised and user passwords being stored in plain text. Bad fu. It’s pretty simple to encrypt passwords in MySQL, here’s how you do it.

I’m using MySQL’s AES which is better than older standards. Basically 128 bit key encryption …

To begin, let’s create a demonstration table. e.g.
CREATE TABLE users ( user_name VARCHAR(20) NOT NULL, user_password BLOB );

So now we have a simple table with a user_name field and user_password field. e.g.

mysql> DESCRIBE users;
+---------------+-------------+------+-----+---------+-------+
| FIELD         | Type        | NULL | KEY | DEFAULT | Extra |
+---------------+-------------+------+-----+---------+-------+
| user_name     | varchar(20) | NO   |     | NULL    |       |
| user_password | blob        | YES  |     | NULL    |       |
+---------------+-------------+------+-----+---------+-------+

If we were creating a new user we would insert using the AES_ENCRYPT function. e.g.

INSERT INTO users VALUES ('tim.koopmans', AES_ENCRYPT('mypassword', 'Q4QjOyKANXj3qO'));

In this case I’ve used AES_ENCRYPT which encrypts a string and returns a binary string. The AES (Advanced Encryption Standard) algorithm is encoded with a 128-bit key. The second parameter is the key string, important to keep that secure!

On its own, this offers a good level of security. If the database was compromised the hacker will see information like this. e.g.

mysql> SELECT * FROM users;
+--------------+------------------+
| user_name    | user_password    |
+--------------+------------------+
| tim.koopmans | ??iu?hQ:???Ti?} |
+--------------+------------------+

The problem with this, is that the user_password field can still be ‘guessed’ via brute force methods. Every password that is the same i.e. ‘mypassword’ will also have the same binary string.

You can add a salt to this by prefixing random digits to the plain text password before you encrypt. For example, we can use the first 12 digits of a UUID. e.g.

INSERT INTO users VALUES ('joe.blow', AES_ENCRYPT(CONCAT(LEFT(UUID(),12), 'mypassword'), 'Q4QjOyKANXj3qO'));

The result set now looks like this if compromised.

mysql> SELECT * FROM users;
+--------------+----------------------------------+
| user_name    | user_password                    |
+--------------+----------------------------------+
| tim.koopmans | ??iu?hQ:???Ti?}                  |
| joe.blow     | ?M}69?H?0?H}?/{.?we|?A?P7ZD?R?   |
+--------------+----------------------------------+

Every password that is encrypted should look different which will further complicate dictionary attacks.

To determine a password from a stolen hash, an attacker cannot simply try common passwords (such as English language words or names). Rather, they must calculate the hashes of random characters (at least for the portion of the input they know is the salt), which is much slower.

For best results, the value/length of your salt should be kept secret (and separate from your database). You should also keep the key secret of course!

Now if we ever need to extract the password we use the AES_DECRYPT function. e.g.

mysql> SELECT SUBSTR(AES_DECRYPT(user_password, 'Q4QjOyKANXj3qO'),13) AS password FROM users WHERE user_name = 'joe.blow';
+------------+
| password   |
+------------+
| mypassword |
+------------+

In this case we know the first 12 digits were the salt so I used SUBSTR after decrypting the hash to get the original password.

A couple of caveats, encrypting passwords in this fashion is reversible i.e. If your key gets compromised, then the hacker has access to all your passwords. Another option is to reverse the plain text and key parameters of AES_ENCRYPT, where the user’s password now becomes the key, and the key becomes the plain text. Even if the database and key is compromised, it is hard to crack user passwords. You would not be able to recover lost passwords of course … Security through obscurity!

In this [...]]]> A common requirement when load testing is to populate subsequent requests with dynamic data from previous requests. If you’re thinking about using a database to store static test data, such as usernames, paswords, credit card numbers etc then read on to find out how to extract this data at runtime and populate JMeter variables.

In this example we’ll need to add a JDBC Connection Configuration to our test plan. This will define and control the way in which JMeter will access your database.

Notice I’ve left the defaults for connection pool properties, although you may need to tweak this depending on how you’ll access the data. I’ve used a JDBC connection URL that accesses a local mysql server listening on port 3306. The database in use is called watirgrid

I’ve also defined the JDBC Driver class and specified the username / password to connect with.

Now we can make a JDBC request using the JDBC Request sampler. It will look something like this:

The Variable Name matches the JDBC Connection Config Variable Name i.e. mysql

I’m using a select statement that will print out results in a JSON-like format. That query is as follows:

SELECT
CONCAT("[",
GROUP_CONCAT(
CONCAT("{email:'",email,"'"),
CONCAT(",login_count:'",login_count),"'}")
,"]")
AS json FROM users

In doing this, it will return a result set that looks like this:

+-------------------------------------------------+
| json                                            |
+-------------------------------------------------+
| [{email:'tim.koops@gmail.com',login_count:'1'}] |
+-------------------------------------------------+
1 row IN SET (0.00 sec)

Now we need to add a Post Processor Regex Extractor to the JDBC request as follows:

The reason we do this is so we can extract the results from the JSON string returned by the database query. For example, to extract and populate a variable called ${email} our PPRE would look like this:

Now we’re cooking! Add a Debug Sampler to confirm everything works as you’d expect. You should see results like this in the response data

email=tim.koops@gmail.com
email_g=1
email_g0=email:'tim.koops@gmail.com'
email_g1=tim.koops@gmail.com
mysql=org.apache.jmeter.protocol.jdbc.config.DataSourceElement$DataSourceComponentImpl@10d69502

Attached is a demo test plan: jdbc_example.jmx

We’re [...]]]> No, not the song but my adage for 2010 in my approach to performance testing in general. Readers of my old blog 90kts.com may have noticed the recent change to altentee.com. In 2009 I helped setup a boutique testing company called Altentee Pty Ltd. Read on for a quick advertisement about this new initiative …

We’re an Australian software performance and test automation specialist company based in Melbourne. We already have a global presence and employ some of the most experienced performance engineers available.

We pride ourselves on quality and cost effective solutions . Altentee offers alternative performance and test automation solutions using viable open source tool sets. This eliminates exorbitant license costs commonly associated with commercial tool sets. We’ve partnered with the team from BrowserMob.com for clients that want real-browser based performance testing. We’re also big fans of Hyperic for performance monitoring, Watir for browser automation and JMeter for protocol level performance tests.

Altentee’s mission is to deliver affordable and reliable performance and test automation alternatives to more traditional solutions.

Our goal is to deliver high quality, timely solutions whilst remaining affordable for any size organisation. Pay for the skills, not the tools and bring the focus back to your outcomes: delivering a fast, reliable system.

If you’re interested in what we do or know someone looking for performance and test automation solutions that won’t break the bank feel free to contact us via info@altentee.com

Essentially the EC2 snapshot is triggered like this:

This method is based on advice from here

Notice I [...]]]> So I’ve been working on a system where the MySQL instance on EC2 sporadically locks up (mysqld is zombied) during a snapshot process =)

Essentially the EC2 snapshot is triggered like this:

system("xfs_freeze -f /vol")                        and die;
system("ec2-create-snapshot $vol -K $key -C $crt ") and die;
system("xfs_freeze -u /vol")                        and die;

This method is based on advice from here

Notice I am doing an xfs_freeze of the entire volume on which the mysql data sits. It is intended to be used with volume managers and hardware RAID devices that support the creation of snapshots such as EBS.

The -f flag requests the specified XFS filesystem to be frozen from new modifications. When this is selected, all ongoing transactions in the filesystem are allowed to complete, new write system calls are halted, other calls which modify the filesystem are halted, and all dirty data, metadata, and log information are written to disk. Any process attempting to write to the frozen filesystem will block waiting for the filesystem to be unfrozen.

Note that even after freezing, the on-disk filesystem can contain information on files that are still in the process of unlinking. These files will not be unlinked until the filesystem is unfrozen or a clean mount of the snapshot is complete.

The -u flag is used to un-freeze the filesystem and allow operations to continue. Any filesystem modifications that were blocked by the freeze are unblocked and allowed to complete.

EC2 recommends this for ec2-create-snapshot:

When taking a snapshot of a file system, we recommend unmounting it first. This ensures the file system metadata is in a consistent state, that the ‘mounted indicator’ is cleared, and that all applications using that file system are stopped and in a consistent state. Some file systems, such as xfs, can freeze and unfreeze activity so a snapshot can be made without unmounting.

Clearly we don’t want to unmount the whole drive, so freezing the XFS volume is our best bet.

On a couple of occassions, we’ve seen the whole mysql server crash when a backup was taking place. The first symptoms displayed in logs is a long semaphore wait. This message is repeated for different threads:

mysqld[26852]: InnoDB: Warning: a long semaphore wait:

mysqld[26852]: --Thread 46912644213072 has waited at ./../include/trx0sys.ic line 101 for 241.00 seconds the semaphore:

mysqld[26852]: X-lock on RW-latch at 0x2aaaaf01cc08 created in file buf0buf.c line 497

mysqld[26852]: a writer (thread id 46912644213072) has reserved it in mode  wait exclusive

After about 10 minutes of this behaviour the server crashes. This is because 101 threads have made 101 connections where the max_connections=100. This only occurs whenever we are doing a snapshot in the background which makes me thing its a deadlock condition on the underlying filesystem.

mysqld[26852]: InnoDB: We intentionally crash the server, because it appears to be hung.

mysqld[26852]: 091207 20:16:41InnoDB: Assertion failure in thread 46912625965392 in file srv0srv.c line 2097

mysqld[26852]: InnoDB: We intentionally generate a memory trap.

mysqld[26852]: InnoDB: Submit a detailed bug report to http://bugs.mysql.com.

mysqld[26852]: InnoDB: If you get repeated assertion failures or crashes, even

mysqld[26852]: InnoDB: immediately after the mysqld startup, there may be

mysqld[26852]: InnoDB: corruption in the InnoDB tablespace. Please refer to

mysqld[26852]: InnoDB: http://dev.mysql.com/doc/refman/5.0/en/forcing-recovery.html

mysqld[26852]: InnoDB: about forcing recovery.

mysqld[26852]: 091207 20:16:41 - mysqld got signal 11 ;

mysqld[26852]: This could be because you hit a bug. It is also possible that this binary

mysqld[26852]: or one of the libraries it was linked against is corrupt, improperly built,

mysqld[26852]: or misconfigured. This error can also be caused by malfunctioning hardware.

mysqld[26852]: We will try our best to scrape up some info that will hopefully help diagnose

mysqld[26852]: the problem, but since we have already crashed, something is definitely wrong

mysqld[26852]: and this may fail.

In panic mode, we restarted the whole server which was able to then recover from this error. But is there a more graceful recovery open to us?

Alestic have released a revised version of the ec2-create-snapshot called ec2-consistent-snapshot

Here are some of the ways in which the ec2-consistent-snapshot program has improved over the original:

Command line options for passing in AWS keys, MySQL access information, and more.
Can be run with or without a MySQL database on the file system. This lets you use the command to initiate snapshots for any EBS volume.
Can be used with or without XFS file systems, though if you don’t use XFS, you run the risk of not having a consistent file system on EBS volume restore.
Instead of using the painfully slow ec2-create-snapshot command written in Java, this Perl program accesses the EC2 API directly with orders of magnitude speed improvement.
A preliminary FLUSH is performed on the MySQL database before the FLUSH WITH READ LOCK. This preparation reduces the total time the tables are locked.
A preliminary sync is performed on the XFS file system before the xfs_freeze. This preparation reduces the total time the file system is locked.
The MySQL LOCK now has timeouts and retries around it. This prevents horrible blocking interactions between the database lock, long running queries, and normal transactions. The length of the timeout and the number of retries are configurable with command line options.
The MySQL FLUSH is done in such a way that the statement does not propagate through to slave databases, negatively impacting their performance and/or causing negative blocking interactions with long running queries.
Cleans up MySQL and XFS locks if it is interrupted, if a timeout happens, or if other errors occur. This prevents a number of serious locking issues when things go wrong with the environment or EC2 API.
Can snapshot EBS volumes in a region other than the default (e.g., eu-west-1).
Can initiate snapshots of multiple EBS volumes at the same time while everything is consistently locked. This has been used to create consistent snapshots of RAIDed EBS volumes.

Here’s hoping that this release fixes the problems with locking of the filesystem! Notice we don’t attempt to flush and lock since we are using the InnoDB engine.

Revised snapshot code looks like this:

# create snapshot
system("xfs_freeze -f /vol")                        and die;
system("ec2-consistent-snapshot --aws-access-key-id $key --aws-secret-access-key $secret --xfs-filesystem /vol $vol") and die;
system("xfs_freeze -u /vol")                        and die;

# auto complete word autocompleteword.automatic=1

# advanced find and replace find.replace.advanced=1

# force [...]]]> Well the subject goes without saying. I’ve been working on WinXP again lately and when I’m not around my favourite text editor on the mac (TextMate) I opt for SCiTE on windows. The following settings I find useful for this simple but powerful text editor. YMMV.


# auto complete word
autocompleteword.automatic=1

# advanced find and replace
find.replace.advanced=1

# force monospaced fonts
font.base=$(font.monospace)
font.small=$(font.monospace)
font.comment=$(font.monospace)
font.text=$(font.monospace)
font.text.comment=$(font.monospace)
font.embedded.base=$(font.monospace)
font.embedded.comment=$(font.monospace)
font.vbs=$(font.monospace)
font.monospace=font:DejaVu Sans Mono,size:8

# show the status bar
statusbar.visible=1


Part of my grievance for that was based on frustration in dealing with different load testing platforms. Not frustration born out of incompetence, more the frustration born out writing tests for increasingly [...]]]> In my last post I made a tongue-in-cheek observation that load testing really doesn’t work by today’s perceived standards for web automation and testing.

Part of my grievance for that was based on frustration in dealing with different load testing platforms. Not frustration born out of incompetence, more the frustration born out writing tests for increasingly complex browsers with an antiquated toolset.

That’s not to say that protocol based application load testing (or the languages they sit on) aren’t powerful in their own right. LoadRunner, JMeter et al. has their place in the load testing world. But I would like to relegate them to the background as useful, sometimes expensive tools to spin up the wheels so to speak. Nothing like a bit of JMeter infused fun to get the wheels smoking on your system under test. But for the real part, measuring user experience on the web platform?

If you can’t wait to get started then I’d like to introduce BrowserMob.com. Been around since mid 2008 and offers real browser testing using Selenium scripts distributed across a grid of EC2 computers. Reasonably priced, well documented and supported by the legendary Patrick Lightbody, founder of things like OpenQA and Selenium Remote Control.

But hang on a second, you don’t like Selenium right? Have zero budget? Love Watir? Well I’m working on a similar construct aptly called WatirGrid which also allows for distributed testing across a grid network using Watir. The engine (gem) itself is open source and you can get it as follows:

gem install watirgrid

If you’d like to contribute please follow the WatirGrid project on github.

The Basics
WatirGrid is built on Rinda which implements the Linda distributed computing paradigm in Ruby. According to Wikipedia: “Linda is a model of coordination and communication among several parallel processes operating upon objects stored in and retrieved from shared, virtual, associative memory.”

In other words, WatirGrid allows multiple parallel processes to provide remote Watir objects in the form of tuple spaces across a grid network. This grid network is controlled by a ring server and looks a bit like this:

Key Terminology
The controller implements a repository of tuples (tuple space) that can be accessed concurrently. The controller hosts the ring server which advertises these tuples across a grid network. Typically you will host one controller on a central machine.

The providers make remote Watir objects available to the tuple space hosted by the ring server. Typically you will host one or many providers on your grid network, for example, each PC may become a single provider of a Watir tuple in the form of an Internet Explorer, Firefox or Safari browser object.

Simple Example
Pick a server to host the controller for the ring server and execute the following:

  $ controller

This should find your external facing IP and start the corresponding DRb and Ring Servers:

  DRb server started on : druby://143.238.105.61:11235
  Ring server started on: druby://143.238.105.61:12358

On each client PC, host the provider for the distributed (DRb) Watir objects

  $  provider

This should find the recently started Ring Server and register a tuple for the Watir object:

  DRb server started on : druby://143.238.105.61:11236
  Ring server found on  : druby://143.238.105.61:12358
  New tuple registered  : druby://143.238.105.61:12358

You will now be able to execute commands across remote browsers on your grid network.
e.g.

  grid = Watir::Grid.new(:ring_server_port => 12358)
  grid.start(:quantity => 1, :read_all => true)
  threads = []
  grid.browsers.each do |browser|
    threads << Thread.new do
      b = browser[:object].new_browser
      b.goto("http://www.google.com")
      b.text_field(:name, 'q').set("watirgrid")
      b.button(:name, "btnI").click
    end
  end
  threads.each {|thread| thread.join}

You may wish to host the controller and providers on different machines in a real scenario. You can specify things like server hostnames, ports and access control lists for each of the different types of servers. For more help see:

  $  controller --help
  Usage: controller [options]
  Specific options:
      -H, --drb-server-host HOST       Specify DRb Server interface to host on
      -d, --drb-server-port PORT       Specify DRb Server port to listen on
      -h, --ring-server-host HOST      Specify Ring Server interface to host on
      -r, --ring-server-port PORT      Specify Ring Server port to listen on
      -a, --access-control-list ACLS   Specify a comma separated Access Control List
      -l, --log-level LEVEL            Specify log level {DEBUG|INFO|ERROR}
          --help                       Show this message
 
  $  provider --help
  Usage: provider [options]
  Specific options:
      -H, --drb-server-host HOST       Specify DRb Server interface to host on
      -d, --drb-server-port PORT       Specify DRb Server port to listen on
      -h, --ring-server-host HOST      Specify Ring Server host to connect to
      -r, --ring-server-port PORT      Specify Ring Server port to broadcast on
      -b, --browser-type TYPE          Specify browser type to register {ie|firefox|safari}
      -a, --access-control-list ACLS   Specify a comma separated Access Control List
      -l, --log-level LEVEL            Specify log level {DEBUG|INFO|ERROR}
          --help                       Show this message

Need more help?

Keep an eye on the wiki and the rspec test cases!

Load testing does not work for web as it is today primarily because… 1. Load testing tool [...]]]> I was going to write why LoadRunner does not work for the web as it is today and into the foreseeable future, but decided against that because I don’t think the following points are necessarily limited to a specific toolset.

Load testing does not work for web as it is today primarily because…

1. Load testing tool languages are not flexible enough.
Load testing tools are in need of a major overhaul. Consider LoadRunner, the majority of your scripts will be specified in C or proprietary API calls such as lr_ statements. String manipulation for example, whilst possible is extremely clunky and requires oodles of C code. Lack of proper regex or xpath support limits script flexibility in terms of parsing the necessary contents of web page and ajax calls. Don’t even get me started on ajax! Similar complaints for JMeter. Although better support for regex you can really only edit test plans via the GUI. You’ll go cross-eyed looking at test plans in xml. I think the Grinder is the closest product I’ve seen that supports flexibility as you can write tests in Jython which provides more flexibility in terms of scripting.

2. We’re still stuck in record and playback mode.
We really haven’t advanced like our automation brethren. Part of the big sell by commercial load testing companies is “Look it’s easy, just hit the red button and play it back when you’re finished!”. Bollocks to that, we know that never works. A dream load testing tool would support the ability to hand-code your statements and be editable in a text editor! In a similar fashion to watir.

The dream tool would look like:

1
2
3
4

vusers.start(100)
  browser.goto('url')
  browser.button(:text, /Continue/).click
end

Instead load testing scripts in LoadRunner look something like this:

1
2
3
4
5
6
7
8
9
10

web_submit_data("transaction",
"Action=https://someurl/I/have/to/keep/correlating/damn/it",
"Method=POST"
"RecContentType="text/html",
"Mode=HTML",
"ITEMDATA,
"Name=L_3_0_0_1_0_1", "Value={another_value_I_have_to_correlate}",
"Name=L_3_0_0_2_0_1", "Value={maybe_paramatize_this_one}",
ENDITEM,
LAST);

We are of course talking about very different technologies. Watir automation scripts driving real browsers with win32ole, whereas most load testing tools are injecting traffic at the protocol layer. If a typical end to end scenario has 30 steps, this can amount to some 2000+ lines of code thanks to the extra verbosity of having to specify all the form values and so forth. This also means you have oodles of code to correlate and parametize. It’s not something you want to hand code, the quickest path is to record (unfortunately). BrowserMob is the closest online provider I’ve seen that overcomes this, supporting load tests written as selenium scripts. Problem is, we need a solution for all those non-public facing companies, the bulk of where I do my work at!

3. No real code versioning.
Changes to the UI will break your script! One can implement their own code versioning as I have recently begun to do with git but there really should be built in support for this. That aside, you revision your code but effectively blast it away every time you hit the red record button. Reason? Each recording can look vastly different to the last. It only takes one form value L_3_0_0_1_0_1 to change to L_3_0_0_1_0_2 or one block of code executed by an ajax call to shift to cause mass confusion. Versioning your code *may* pick this change up, but only if you got the power of the diff! Diffing just got a little harder. Shoot back 10 years ago and diffing for correlation would have been a cinch. Wind forward to today with hundreds of ajax or json calls and correlation can be a nightmare. Windiff struggles. Daisydiff does a much better job highlighting differences by line and by column. This approach kind of works in a waterfall test environment. Throw in a twice weekly code drop in your performance test environment and watch your scripts break. Agile anyone?

So if I was an uber-elite programmer what would I do to move loadtesting into the current decade?
1. Develop a load-testing tool that is based on a flexible scripting language such as Ruby.
2. Get rid of the record button and provide free text editors to the world!
3. Provide a framework that can be versioned and easily detect changes in the system under test.