Regex in BrowserMob Scripts

If you’re using an RBU or VU you may need to extract content from the previous response. For example, enumerate a link to a PDF file for subsequent download.

The BrowserMob interface has a handy findRegexMatches method you can call as follows:

var link = browserMob.findRegexMatches(selenium.getHtmlSource(), 'href="(.+?pdf)"');
browserMob.beginStep("16_download_content");
  browser.get(link[0], 200); 
browserMob.endStep();

It takes a string input and regex pattern parameters. For the string input I am simply passing in the HTML source from the previous request based on a browser object:
var browser = browserMob.getActiveHttpClient();

The regex pattern is self explanatory. The link object will be an array of matches; in this case I’m issuing a get request for the first link in the match array. You might want to add some more code to make this more robust e.g. if no links are found etc.

Pretty simple hey!

Emulating Think Time and Pacing with BrowserMob

If you come from the commercial toolset mindset, you might be interested in how we achieve think time and pacing when using alternatives such as BrowserMob.

Think Time
Think time, is normally defined as the amount of time a virtual user ‘thinks’ between individual steps within a transaction. This time is usually excluded from response time measurements and is an important inclusion in terms of script behaviour. Having no think time means the virtual user will race through transactions more quickly than a normal user would which potentially creates unrealistic load. So how do we emulate think time behaviour in a BrowserMob script?

The BrowserMob replay engine drives Selenium scripts, so you will need to play in the JavaScript sandbox so to speak, in order to emulate think time. There’s some well documented ways to implement this on the BrowserMob blog here.

The BrowserMob interface (API documentation here) has a pause method as follows:
browserMob.pause(15000);

This effectively pauses the user for a period of 15 seconds in the previous example.

The Selenium interface (API documentation here) also has a similar setSpeed method which will set the number of milliseconds to pause after each step elminating the need to specify individual pause statements between each step as follows:
selenium.setSpeed(2000);

If you would like to emulate some common think time settings as you might see in a LoadRunner script you can implement the following at the top of your RBU or VU scripts:

// Multiply think time by:
var think_time_multiple = 1;
 
// Use random percentage for think time e.g. 50 - 100:
var think_time_percentage = 50 + (Math.random() * 100);
 
// Limit think time to:
var think_time_limit = 0; 
 
// Set think time to:
var think_time = 3; //seconds
 
if(think_time_limit > 0) {
  selenium.setSpeed(think_time_limit * 1000);
} else {
  selenium.setSpeed(think_time * think_time_multiple * (think_time_percentage/100) * 1000);
}

If you need additional think time after steps then you can add the following statement where required:
browserMob.pause(3000 * think_time_multiple * (think_time_percentage/100));

Pacing
Some people like to control transaction volume/throughput by messing about with think time. I don’t favour this approach because you need to first be cognisant of how long each iteration will take including think time then adjust think time accordingly. Once it’s set (or hardcoded into your script) then the vusers are stuck with that setting. I much prefer the pacing concept, which determines how long a vuser should ‘wait’ before starting the next iteration. This wait time is determined by elapsed time including any server side processing, not just user think time. So how do we emulate pacing in a BrowserMob script?

First I like to set some targets such as the transaction rate (per hour per user) and then calculate the pacing from this target. I also set a variable iterate to true, which comes into play in the main body of the script.

// Run logic
var transaction_rate = 10;  // Target transaction rate per hour per user
var pacing  = 60*60*1000 / transaction_rate;
var iterate = true;

Now we’ve got the target and pacing set, we can get into the main logic. Let’s start an iteration.

while (iterate) {
  var start = new Date();
  browserMob.beginTransaction();
  ...

Notice I’ve used a while loop which checks for the iterate boolean and creates a start date time stamp. I also initiate a BrowserMob transaction.

After we complete the iteration (which is essentially a bunch of BrowserMob steps) we can close the while loop with some pacing logic.

  ...
  // Calculate Pacing
  var finish  = new Date(); 
  var elapsed = finish.getTime() - start.getTime();
 
  if (browserMob.isValidation()) {
    browserMob.log("Adjusted pacing would be "+ (pacing - elapsed) + " msecs");
    browserMob.log("or " + Math.round(3600000 / (pacing - elapsed) ) + " trans/hour");
    iterate = false;
  } else {
    if(pacing > elapsed)
      browserMob.pause(pacing - elapsed);
  }
 
  browserMob.endTransaction();
}

This will calculate the finish date time stamp and then determine the difference between the previously calculated target pacing and the actual time elapsed. If there is a difference then the user will pause, otherwise it will just continue on to the next iteration. This basically allows you to target a set number of transactions per hour. When validating the script it will output these calculations to the log file.

Instant Website Tests with BrowserMob

Can you afford a slow landing page?

Google AdWords page quality criteria are many, a prominent check being your landing page load time. A slow load time can lead to a lower quality score.

Make sure you’re not wasting effort on advertising campaigns. Altentee uses BrowserMob to help check website performance for clients from multiple locations. You can run a free check using the form below.

(redirects to BrowserMob)

If you need help optimizing your landing page, or help with performance testing in general feel free to contact Altentee today.

Securing MySQL Passwords

There’s been a bit of buzz lately where MySQL databases have been compromised and user passwords being stored in plain text. Bad fu. It’s pretty simple to encrypt passwords in MySQL, here’s how you do it.

Continue reading Securing MySQL Passwords

Dynamic Data from JMeter JDBC Requests

A common requirement when load testing is to populate subsequent requests with dynamic data from previous requests. If you’re thinking about using a database to store static test data, such as usernames, paswords, credit card numbers etc then read on to find out how to extract this data at runtime and populate JMeter variables.

Continue reading Dynamic Data from JMeter JDBC Requests

Harder, Better, Faster, Stronger [and Cheaper!]

No, not the song but my adage for 2010 in my approach to performance testing in general. Readers of my old blog 90kts.com may have noticed the recent change to altentee.com. In 2009 I helped setup a boutique testing company called Altentee Pty Ltd. Read on for a quick advertisement about this new initiative …

Continue reading Harder, Better, Faster, Stronger [and Cheaper!]

Improved EBS snapshots on Amazon EC2

So I’ve been working on a system where the MySQL instance on EC2 sporadically locks up (mysqld is zombied) during a snapshot process =)

Essentially the EC2 snapshot is triggered like this:

system("xfs_freeze -f /vol")                        and die;
system("ec2-create-snapshot $vol -K $key -C $crt ") and die;
system("xfs_freeze -u /vol")                        and die;

This method is based on advice from here

Notice I am doing an xfs_freeze of the entire volume on which the mysql data sits. It is intended to be used with volume managers and hardware RAID devices that support the creation of snapshots such as EBS.
Continue reading Improved EBS snapshots on Amazon EC2

My Favourite SCiTE settings …

Well the subject goes without saying. I’ve been working on WinXP again lately and when I’m not around my favourite text editor on the mac (TextMate) I opt for SCiTE on windows. The following settings I find useful for this simple but powerful text editor. YMMV.

Continue reading My Favourite SCiTE settings …

Going Parallel … distributed testing across a grid network using Watir

In my last post I made a tongue-in-cheek observation that load testing really doesn’t work by today’s perceived standards for web automation and testing.

Part of my grievance for that was based on frustration in dealing with different load testing platforms. Not frustration born out of incompetence, more the frustration born out writing tests for increasingly complex browsers with an antiquated toolset.

That’s not to say that protocol based application load testing (or the languages they sit on) aren’t powerful in their own right. LoadRunner, JMeter et al. has their place in the load testing world. But I would like to relegate them to the background as useful, sometimes expensive tools to spin up the wheels so to speak. Nothing like a bit of JMeter infused fun to get the wheels smoking on your system under test. But for the real part, measuring user experience on the web platform?
Continue reading Going Parallel … distributed testing across a grid network using Watir

Why Load Testing Does Not Work for the Web `Date::today >> 36`

I was going to write why LoadRunner does not work for the web as it is today and into the foreseeable future, but decided against that because I don’t think the following points are necessarily limited to a specific toolset.

Load testing does not work for web as it is today primarily because…
Continue reading Why Load Testing Does Not Work for the Web `Date::today >> 36`