The solution as it turns [...]]]> Recently a client asked Altentee to generate some load against their subscription to Google Analytics (GA). Calls to GA and other 3rd party monitoring tools are typically requests we blacklist or consider out of scope. So it was unusual to think the other way around, how would you performance test GA?

The solution as it turns out is quite simple. Google Analytics is well documented here in terms of what query parameters constitute the GIF request that registers traffic.

The key parameters which need to be catered for when spoofing traffic are:
utmn => Unique ID generated for each GIF request to prevent caching of the GIF image.
utmhid => Another unique ID not documented but I think is related to AdSense
plus all the GA cookies including utma, utmb, utmc and utmz

The best way to parametize this is to observe real traffic with a tool like FireBug in the Net(work) panel. That way you can make a copy of a real GIF request. Essentially you are going to fake the call to __utm.gif. An example request looks like this:

http://www.google-analytics.com/__utm.gif?utmwv=4.7.2&utmn=123456789&utmhn=www.altentee.com&utmcs=UTF-8&utmsr=1920×1200&utmsc=24-bit&utmul=en-us&utmje=1&utmfl=10.0%20r45&utmdt=AlteneeBlog&utmhid=1697634062&utmr=-&utmp=%2F&utmac=UA-12345678-3&utmcc=__utma%3D199946558.28502083.1280297456.1280637882.1281056979.6%3B%2B__utmz%3D199946558.1280297456.1.1.utmcsr%3D(direct)%7Cutmccn%3D(direct)%7Cutmcmd%3D(none)%3B&gaq=1

All you need to do to spoof traffic or generate page views to your GA account is make repeated calls to this URL after parametizing key fields previously mentioned. An example request in JMeter with corresponding parametized functions in bold (random and time) looks like this:

/__utm.gif?utmwv=4.7.2&utmn=${__Random(10000000,60000000,utmn)}&utmhn=www.altentee.com&utmcs=UTF-8&utmsr=1366×768&utmsc=24-bit&utmul=en-us&utmje=1&utmfl=10.1%20r53&utmdt=PerformanceTest&utmhid=${__Random(100000000,900000000,utmhid)}&utmr=-&utmp=%2F&utmac=UA-12345678-3&utmcc=__utma%3D199946558.1448353202.${__time()}.${__time()}.${__time()}.4%3B%2B__utmz%3D199946558.${__time()}.1.1.utmcsr%3D(direct)%7Cutmccn%3D(direct)%7Cutmcmd%3D(none)%3B&gaq=1

You could also build this URL from scratch although it’s much easier to copy and modify the target URL for the site you’re actually testing. Please don’t use the URLs above as I’ve faked other things like the Google Analytics account code (utmac). It’s also worth changing the page title (utmdt) so you can separate real traffic from the fake traffic generated during performance testing.

Build a simple test plan in JMeter with a HTTP sampler generating requests to the aforementioned URL, put in a constant timer (or similar) so you don’t spam GA’s servers and away you go. You can then measure response time to this service for as many threads as you like. I generated 10K page views in 5 minutes which was more than sufficient to satisfy the client’s requirements.

The problem here is that you cannot auto authenticate to a proxy server using Chrome –proxy-server=my.proxy.com:3128 or using Firefox.

Chrome will always prompt you for a user name and password on an authenticating proxy when it starts up [...]]]> A question came up on stack overflow about how to route Watir through an authenticating proxy within your script.

The problem here is that you cannot auto authenticate to a proxy server using Chrome –proxy-server=my.proxy.com:3128 or using Firefox.

Chrome will always prompt you for a user name and password on an authenticating proxy when it starts up (via Webdriver or manually). Firefox will not prompt (when launched via WebDriver) and will silently fail.

I *almost* got this to work by modifying headers on the fly with a Firefox add on (Modify Headers). Basic authentication is just base-64 encoded so you can force each request to authorize itself by modifying the Proxy-Authorization request header e.g.:
Proxy-Authorization Basic asHJksaKHs87akhkjah7

However whilst this works for manually launched instances of Firefox, Webdriver launched instances would complain that the add on is not compatible with FF3.6.3 and so once again would fail. The other problem with this approach is that I couldn’t find a similar plugin for Chrome.

Eventually I gave in and went with a slightly more complicated workaround. That is, chaining an authenticated proxy (with squid) running on localhost with a cache_peer relationship to the authenticated proxy running out there on the ‘tubes’. I had thought about using Webrick as a proxy (as per previous posts) but lack of SSL support is a gotcha for me. Squid seemed most appropriate in this case:

To install squid from source:
cd /opt/src
sudo curl -O http://www.squid-cache.org/Versions/v2/2.7/squid-2.7.STABLE9.tar.gz
sudo tar xjf squid-2.7.STABLE9.tar.gz
cd squid-2.7.STABLE9
# I wanted SSL support for this
./configure --enable-ssl
make
sudo make install

Edit the squid.conf (location is normally /etc/squid/squid.conf on linux):
sudo vim /usr/local/squid/etc/squid.conf

# Don't want to run squid as root
# TAG: cache_effective_user
cache_effective_user nobody
# TAG: cache_effective_group
cache_effective_group wheel
# Point this proxy to a peer cache, and provide the logon credentials
# TAG: cache_peer
cache_peer proxy.altentee.com parent 3128 0 default no-query login=username:mypassword
never_direct allow all
# Allow access from my local private network
# TAG: cache_peer
acl localnet src 10.0.0.0/0

Save changes and change ownership as appropriate:
sudo chown -R nobody:wheel /usr/local/squid

Initialize the cache and startup squid:
sudo /usr/local/squid/sbin/squid -z
sudo /usr/local/squid/sbin/squid -D

Now you can configure firefox (using -ProfileManager) or chrome (using –proxy-server=127.0.0.1) to point to your local proxy server. This proxy will then authenticate on your behalf and peer all requests via the original parent.

A little long winded but achieves the outcome. Added side benefit is you’ve now got a local proxy (with access logs) that you can start doing interesting things with in your automation / performance tests like modifying headers and so forth.

Hope this helps!

The BrowserMob interface has a handy findRegexMatches method you can call as follows:

It takes a string input and [...]]]> If you’re using an RBU or VU you may need to extract content from the previous response. For example, enumerate a link to a PDF file for subsequent download.

The BrowserMob interface has a handy findRegexMatches method you can call as follows:

var link = browserMob.findRegexMatches(selenium.getHtmlSource(), 'href="(.+?pdf)"');
browserMob.beginStep("16_download_content");
  browser.get(link[0], 200); 
browserMob.endStep();

It takes a string input and regex pattern parameters. For the string input I am simply passing in the HTML source from the previous request based on a browser object:
var browser = browserMob.getActiveHttpClient();

The regex pattern is self explanatory. The link object will be an array of matches; in this case I’m issuing a get request for the first link in the match array. You might want to add some more code to make this more robust e.g. if no links are found etc.

Pretty simple hey!

Think Time Think time, is normally defined as the amount of time a virtual user ‘thinks’ between individual steps within a transaction. This time is usually excluded from response time measurements [...]]]> If you come from the commercial toolset mindset, you might be interested in how we achieve think time and pacing when using alternatives such as BrowserMob.

Think Time
Think time, is normally defined as the amount of time a virtual user ‘thinks’ between individual steps within a transaction. This time is usually excluded from response time measurements and is an important inclusion in terms of script behaviour. Having no think time means the virtual user will race through transactions more quickly than a normal user would which potentially creates unrealistic load. So how do we emulate think time behaviour in a BrowserMob script?

The BrowserMob replay engine drives Selenium scripts, so you will need to play in the JavaScript sandbox so to speak, in order to emulate think time. There’s some well documented ways to implement this on the BrowserMob blog here.

The BrowserMob interface (API documentation here) has a pause method as follows:
browserMob.pause(15000);

This effectively pauses the user for a period of 15 seconds in the previous example.

The Selenium interface (API documentation here) also has a similar setSpeed method which will set the number of milliseconds to pause after each step elminating the need to specify individual pause statements between each step as follows:
selenium.setSpeed(2000);

If you would like to emulate some common think time settings as you might see in a LoadRunner script you can implement the following at the top of your RBU or VU scripts:

// Multiply think time by:
var think_time_multiple = 1;
 
// Use random percentage for think time e.g. 50 - 100:
var think_time_percentage = 50 + (Math.random() * 100);
 
// Limit think time to:
var think_time_limit = 0; 
 
// Set think time to:
var think_time = 3; //seconds
 
if(think_time_limit > 0) {
  selenium.setSpeed(think_time_limit * 1000);
} else {
  selenium.setSpeed(think_time * think_time_multiple * (think_time_percentage/100) * 1000);
}

If you need additional think time after steps then you can add the following statement where required:
browserMob.pause(3000 * think_time_multiple * (think_time_percentage/100));

Pacing
Some people like to control transaction volume/throughput by messing about with think time. I don’t favour this approach because you need to first be cognisant of how long each iteration will take including think time then adjust think time accordingly. Once it’s set (or hardcoded into your script) then the vusers are stuck with that setting. I much prefer the pacing concept, which determines how long a vuser should ‘wait’ before starting the next iteration. This wait time is determined by elapsed time including any server side processing, not just user think time. So how do we emulate pacing in a BrowserMob script?

First I like to set some targets such as the transaction rate (per hour per user) and then calculate the pacing from this target. I also set a variable iterate to true, which comes into play in the main body of the script.

// Run logic
var transaction_rate = 10;  // Target transaction rate per hour per user
var pacing  = 60*60*1000 / transaction_rate;
var iterate = true;

Now we’ve got the target and pacing set, we can get into the main logic. Let’s start an iteration.

while (iterate) {
  var start = new Date();
  browserMob.beginTransaction();
  ...

Notice I’ve used a while loop which checks for the iterate boolean and creates a start date time stamp. I also initiate a BrowserMob transaction.

After we complete the iteration (which is essentially a bunch of BrowserMob steps) we can close the while loop with some pacing logic.

  ...
  // Calculate Pacing
  var finish  = new Date(); 
  var elapsed = finish.getTime() - start.getTime();
 
  if (browserMob.isValidation()) {
    browserMob.log("Adjusted pacing would be "+ (pacing - elapsed) + " msecs");
    browserMob.log("or " + Math.round(3600000 / (pacing - elapsed) ) + " trans/hour");
    iterate = false;
  } else {
    if(pacing > elapsed)
      browserMob.pause(pacing - elapsed);
  }
 
  browserMob.endTransaction();
}

This will calculate the finish date time stamp and then determine the difference between the previously calculated target pacing and the actual time elapsed. If there is a difference then the user will pause, otherwise it will just continue on to the next iteration. This basically allows you to target a set number of transactions per hour. When validating the script it will output these calculations to the log file.

Google AdWords page quality criteria are many, a prominent check being your landing page load time. A slow load time can lead to a lower quality score.

Make sure you’re not wasting effort on advertising campaigns. Altentee uses BrowserMob to help check website performance for clients from multiple locations. You [...]]]> Can you afford a slow landing page?

Google AdWords page quality criteria are many, a prominent check being your landing page load time. A slow load time can lead to a lower quality score.

Make sure you’re not wasting effort on advertising campaigns. Altentee uses BrowserMob to help check website performance for clients from multiple locations. You can run a free check using the form below.

If you need help optimizing your landing page, or help with performance testing in general feel free to contact Altentee today.

We’re [...]]]> No, not the song but my adage for 2010 in my approach to performance testing in general. Readers of my old blog 90kts.com may have noticed the recent change to altentee.com. In 2009 I helped setup a boutique testing company called Altentee Pty Ltd. Read on for a quick advertisement about this new initiative …

We’re an Australian software performance and test automation specialist company based in Melbourne. We already have a global presence and employ some of the most experienced performance engineers available.

We pride ourselves on quality and cost effective solutions . Altentee offers alternative performance and test automation solutions using viable open source tool sets. This eliminates exorbitant license costs commonly associated with commercial tool sets. We’ve partnered with the team from BrowserMob.com for clients that want real-browser based performance testing. We’re also big fans of Hyperic for performance monitoring, Watir for browser automation and JMeter for protocol level performance tests.

Altentee’s mission is to deliver affordable and reliable performance and test automation alternatives to more traditional solutions.

Our goal is to deliver high quality, timely solutions whilst remaining affordable for any size organisation. Pay for the skills, not the tools and bring the focus back to your outcomes: delivering a fast, reliable system.

If you’re interested in what we do or know someone looking for performance and test automation solutions that won’t break the bank feel free to contact us via info@altentee.com

# auto complete word autocompleteword.automatic=1

# advanced find and replace find.replace.advanced=1

# force [...]]]> Well the subject goes without saying. I’ve been working on WinXP again lately and when I’m not around my favourite text editor on the mac (TextMate) I opt for SCiTE on windows. The following settings I find useful for this simple but powerful text editor. YMMV.


# auto complete word
autocompleteword.automatic=1

# advanced find and replace
find.replace.advanced=1

# force monospaced fonts
font.base=$(font.monospace)
font.small=$(font.monospace)
font.comment=$(font.monospace)
font.text=$(font.monospace)
font.text.comment=$(font.monospace)
font.embedded.base=$(font.monospace)
font.embedded.comment=$(font.monospace)
font.vbs=$(font.monospace)
font.monospace=font:DejaVu Sans Mono,size:8

# show the status bar
statusbar.visible=1


Part of my grievance for that was based on frustration in dealing with different load testing platforms. Not frustration born out of incompetence, more the frustration born out writing tests for increasingly [...]]]> In my last post I made a tongue-in-cheek observation that load testing really doesn’t work by today’s perceived standards for web automation and testing.

Part of my grievance for that was based on frustration in dealing with different load testing platforms. Not frustration born out of incompetence, more the frustration born out writing tests for increasingly complex browsers with an antiquated toolset.

That’s not to say that protocol based application load testing (or the languages they sit on) aren’t powerful in their own right. LoadRunner, JMeter et al. has their place in the load testing world. But I would like to relegate them to the background as useful, sometimes expensive tools to spin up the wheels so to speak. Nothing like a bit of JMeter infused fun to get the wheels smoking on your system under test. But for the real part, measuring user experience on the web platform?

If you can’t wait to get started then I’d like to introduce BrowserMob.com. Been around since mid 2008 and offers real browser testing using Selenium scripts distributed across a grid of EC2 computers. Reasonably priced, well documented and supported by the legendary Patrick Lightbody, founder of things like OpenQA and Selenium Remote Control.

But hang on a second, you don’t like Selenium right? Have zero budget? Love Watir? Well I’m working on a similar construct aptly called WatirGrid which also allows for distributed testing across a grid network using Watir. The engine (gem) itself is open source and you can get it as follows:

gem install watirgrid

If you’d like to contribute please follow the WatirGrid project on github.

The Basics
WatirGrid is built on Rinda which implements the Linda distributed computing paradigm in Ruby. According to Wikipedia: “Linda is a model of coordination and communication among several parallel processes operating upon objects stored in and retrieved from shared, virtual, associative memory.”

In other words, WatirGrid allows multiple parallel processes to provide remote Watir objects in the form of tuple spaces across a grid network. This grid network is controlled by a ring server and looks a bit like this:

Key Terminology
The controller implements a repository of tuples (tuple space) that can be accessed concurrently. The controller hosts the ring server which advertises these tuples across a grid network. Typically you will host one controller on a central machine.

The providers make remote Watir objects available to the tuple space hosted by the ring server. Typically you will host one or many providers on your grid network, for example, each PC may become a single provider of a Watir tuple in the form of an Internet Explorer, Firefox or Safari browser object.

Simple Example
Pick a server to host the controller for the ring server and execute the following:

  $ controller

This should find your external facing IP and start the corresponding DRb and Ring Servers:

  DRb server started on : druby://143.238.105.61:11235
  Ring server started on: druby://143.238.105.61:12358

On each client PC, host the provider for the distributed (DRb) Watir objects

  $  provider

This should find the recently started Ring Server and register a tuple for the Watir object:

  DRb server started on : druby://143.238.105.61:11236
  Ring server found on  : druby://143.238.105.61:12358
  New tuple registered  : druby://143.238.105.61:12358

You will now be able to execute commands across remote browsers on your grid network.
e.g.

  grid = Watir::Grid.new(:ring_server_port => 12358)
  grid.start(:quantity => 1, :read_all => true)
  threads = []
  grid.browsers.each do |browser|
    threads << Thread.new do
      b = browser[:object].new_browser
      b.goto("http://www.google.com")
      b.text_field(:name, 'q').set("watirgrid")
      b.button(:name, "btnI").click
    end
  end
  threads.each {|thread| thread.join}

You may wish to host the controller and providers on different machines in a real scenario. You can specify things like server hostnames, ports and access control lists for each of the different types of servers. For more help see:

  $  controller --help
  Usage: controller [options]
  Specific options:
      -H, --drb-server-host HOST       Specify DRb Server interface to host on
      -d, --drb-server-port PORT       Specify DRb Server port to listen on
      -h, --ring-server-host HOST      Specify Ring Server interface to host on
      -r, --ring-server-port PORT      Specify Ring Server port to listen on
      -a, --access-control-list ACLS   Specify a comma separated Access Control List
      -l, --log-level LEVEL            Specify log level {DEBUG|INFO|ERROR}
          --help                       Show this message
 
  $  provider --help
  Usage: provider [options]
  Specific options:
      -H, --drb-server-host HOST       Specify DRb Server interface to host on
      -d, --drb-server-port PORT       Specify DRb Server port to listen on
      -h, --ring-server-host HOST      Specify Ring Server host to connect to
      -r, --ring-server-port PORT      Specify Ring Server port to broadcast on
      -b, --browser-type TYPE          Specify browser type to register {ie|firefox|safari}
      -a, --access-control-list ACLS   Specify a comma separated Access Control List
      -l, --log-level LEVEL            Specify log level {DEBUG|INFO|ERROR}
          --help                       Show this message

Need more help?

Keep an eye on the wiki and the rspec test cases!

Load testing does not work for web as it is today primarily because… 1. Load testing tool [...]]]> I was going to write why LoadRunner does not work for the web as it is today and into the foreseeable future, but decided against that because I don’t think the following points are necessarily limited to a specific toolset.

Load testing does not work for web as it is today primarily because…

1. Load testing tool languages are not flexible enough.
Load testing tools are in need of a major overhaul. Consider LoadRunner, the majority of your scripts will be specified in C or proprietary API calls such as lr_ statements. String manipulation for example, whilst possible is extremely clunky and requires oodles of C code. Lack of proper regex or xpath support limits script flexibility in terms of parsing the necessary contents of web page and ajax calls. Don’t even get me started on ajax! Similar complaints for JMeter. Although better support for regex you can really only edit test plans via the GUI. You’ll go cross-eyed looking at test plans in xml. I think the Grinder is the closest product I’ve seen that supports flexibility as you can write tests in Jython which provides more flexibility in terms of scripting.

2. We’re still stuck in record and playback mode.
We really haven’t advanced like our automation brethren. Part of the big sell by commercial load testing companies is “Look it’s easy, just hit the red button and play it back when you’re finished!”. Bollocks to that, we know that never works. A dream load testing tool would support the ability to hand-code your statements and be editable in a text editor! In a similar fashion to watir.

The dream tool would look like:

1
2
3
4

vusers.start(100)
  browser.goto('url')
  browser.button(:text, /Continue/).click
end

Instead load testing scripts in LoadRunner look something like this:

1
2
3
4
5
6
7
8
9
10

web_submit_data("transaction",
"Action=https://someurl/I/have/to/keep/correlating/damn/it",
"Method=POST"
"RecContentType="text/html",
"Mode=HTML",
"ITEMDATA,
"Name=L_3_0_0_1_0_1", "Value={another_value_I_have_to_correlate}",
"Name=L_3_0_0_2_0_1", "Value={maybe_paramatize_this_one}",
ENDITEM,
LAST);

We are of course talking about very different technologies. Watir automation scripts driving real browsers with win32ole, whereas most load testing tools are injecting traffic at the protocol layer. If a typical end to end scenario has 30 steps, this can amount to some 2000+ lines of code thanks to the extra verbosity of having to specify all the form values and so forth. This also means you have oodles of code to correlate and parametize. It’s not something you want to hand code, the quickest path is to record (unfortunately). BrowserMob is the closest online provider I’ve seen that overcomes this, supporting load tests written as selenium scripts. Problem is, we need a solution for all those non-public facing companies, the bulk of where I do my work at!

3. No real code versioning.
Changes to the UI will break your script! One can implement their own code versioning as I have recently begun to do with git but there really should be built in support for this. That aside, you revision your code but effectively blast it away every time you hit the red record button. Reason? Each recording can look vastly different to the last. It only takes one form value L_3_0_0_1_0_1 to change to L_3_0_0_1_0_2 or one block of code executed by an ajax call to shift to cause mass confusion. Versioning your code *may* pick this change up, but only if you got the power of the diff! Diffing just got a little harder. Shoot back 10 years ago and diffing for correlation would have been a cinch. Wind forward to today with hundreds of ajax or json calls and correlation can be a nightmare. Windiff struggles. Daisydiff does a much better job highlighting differences by line and by column. This approach kind of works in a waterfall test environment. Throw in a twice weekly code drop in your performance test environment and watch your scripts break. Agile anyone?

So if I was an uber-elite programmer what would I do to move loadtesting into the current decade?
1. Develop a load-testing tool that is based on a flexible scripting language such as Ruby.
2. Get rid of the record button and provide free text editors to the world!
3. Provide a framework that can be versioned and easily detect changes in the system under test.

Now I can tag emails in my inbox when I view them with the keyboard shortcut l->a for action, w for [...]]]> I really like the concept of Getting Things Done. I’m also a fan of Inbox Zero. Well, at least try to be. Here’s how I implement GTD using my gmail account…

First I create four labels:
label:g-action
label:g-waiting-on
label:g-some-day
label:g-finished

Now I can tag emails in my inbox when I view them with the keyboard shortcut l->a for action, w for waiting-on and so forth.

I make a habit of reading, deleting or tagging each email that comes through my inbox. Hence I get closer to Inbox Zero.

I also use the Quick Links feature available through GMail Labs.

I then create links to the 4 labels by searching on the syntax described previously.

There are of course plenty of apps and additional plugins for specific browsers that achieve the same thing. I just like this approach because it is generally OS/browser independent which means I can take it with me from workplace to workplace (provided they let me access my gmail account of course!)