Watirgrid Handshake

For some time now I’ve been working in the background on watirgrid which allows for distributed testing across a grid network using Watir. The engine (gem) itself is open source and you can get it as follows:

gem install watirgrid

One of the questions I often get asked (via email) are what ports / network communications are required by Watirgrid, especially in the setup and establishment of a grid. Watirgrid uses both UDP and TCP to establish comms between the providers and a controller as such. Consider the following bounce diagram.

  1. First we set up a controller listening on a public WAN IP of 150.133.123.51. We tell the controller that the ring server should listen on this IP with the -h flag. We also tell the controller to start its DRb server on the same IP with the -H flag. If we don’t specify a port for the ring server or the DRb server (using -r and -d flags) then the controller will use the defaults; 12358 and 11235 respectively.
  2. Next we start up a provider and point it to the Ring server IP of 150.133.123.51 with the -h flag. We also tell the provider to start its DRb server on its own public IP with the -H flag, which in this case is 150.133.123.8
  3. When the provider starts in the previous step, it will broadcast using UDP to all hosts on the controller’s subnet, on port 12358 asking if there are any Ring servers available.
  4. The controller will subsequently respond, letting the provider know which port its own DRb server can be be contacted on using TCP. In this case it will send a TCP packet to the provider’s port 11236 after which the provider will respond back via TCP to the controller’s port of 11235.

Essentially the ports you need open on the controller, are UDP (default 12358) which is the Ring server and TCP (default 11235) which is the controller’s DRb server. The ports you need open on the provider are TCP (default 11236) which is the provider’s DRb server.

All communications between controller and provider are first marshalled via the Ring server (using UDP) and then directed between controller and provider(s) via the DRb servers (using TCP). If you have established a grid out in the wild, or on your own intranet, you should be using the external WAN or LAN IP of your host machines (not local / loopback addresses).

I’ll be posting more information on using watirgrid, including how to execute tests via different testing frameworks such as cucumber in the near future. I’ll also make an effort to monitor watir-general for any questions related to watirgrid.

Happy bot’netting!

Read More

A year in review and a consolidated approach

Wow, wasn’t that long ago when I was promising things that were harder, better, faster and stronger. I think I got part of the way there. The core reason I set up Altentee was to provide reliable (and potentially cheaper) alternatives to traditional licensed performance testing tool sets. There’s no doubt Altentee can test at the limits of your typical web app using tools that cost zilch. The bigger challenge I’ve found this year is convincing potential clients that they really don’t need to spend that much. I’ve even offered a free load test on our homepage to help illustrate this point.

We were lucky to be selected by the development team at Cordelta to help them automate and performance test a high profile public website called MyHospitals. We were able to test millions of hits per hour from domestic and international locations in a wide variety of load scenarios. The success of this approach was underpinned by the following:

  1. An open minded project / development team not coupled to a ‘must-have-most-expensive-toolset-to-do-job’ mentality
  2. An open minded performance test analyst (me!) who believes Excel really is the grandpa of charting, R is the grandma of stats, Sparklines are the only way to present time series data to management, ANY tool can simulate load via HTTP/S and that there is no real distinction between good software testers or developers (only hard work separates the best from the worst).
  3. That 2010 come-no-doubt 2011 buzz word… Cloud

As I sit here pluggin’ our alternative approach at Altentee, I can see the rise of other more successful punters taking on the big kids. I sit and [continue to] chuckle at the reaction to LR pricing and LR zealots who will fall on their swords over LR itself. I have come to realise one thing though, it is not about the tool, or even the alternate tool like perhaps I first thought. It is more about the freedom of choice.

To tackle MyHospitals I was free to choose and implement the following tools:

  1. WatirGrid to orchestrate a small flotilla of IE and FireFox based browsers based on Watir
  2. JMeter to add more at the protocol level of performance testing
  3. httperf to do some basic benchmarking, similar to my front page
  4. numbrcrunchr to pull together system metrics and make for easier analysis
  5. A variety of Australian cloud providers and of course Amazon EC2 to host the test environment

It has been a great year. I’m not entirely free of the commercial chains just yet and am still needing the LR type work to prop up this approach, but I hope 2011 brings about some fresh thinking in performance and test automation with hopefully me somewhere amidst that space.

Read More

Watirgrid Support for Watir-Webdriver

With the growing popularity of watir-webdriver I’ve decided to uplift watirgrid to support this newer approach. For installation instructions of Watir on top of WebDriver there is a good post on the topic by Željko Filipin. There’s also some musings on the ‘why’ of watir-webdriver at Alister Scott’s blog, WatirMelon.

If you want to get on with the business of parallel testing across a grid network with Watir or Watir-WebDriver then have a look at the following code:

Firstly, update your gem installation if it’s been a while ;)

gem update --system

Now install the prerelease version of Watirgrid

gem install watirgrid --prerelease

At this stage I haven’t updated my gemspec with dependencies, so you’ll also need uuid

gem install uuid

To test a simple 1 controller / 1 provider scenario using IE

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
require 'rubygems'
require 'watirgrid'
 
# setup a controller on port 12351 for your new grid
controller = Controller.new(
        :ring_server_port => 12351,
  :loglevel => Logger::ERROR)
controller.start
 
# add a provider to your grid
# :browser_type => 'webdriver' if using webdriver or
# :browser_type => 'ie' if using watir...
provider = Provider.new(
        :ring_server_port => 12351,
  :loglevel => Logger::ERROR, :browser_type => 'webdriver')
provider.start
 
# connect to the grid and take all providers from it (this time only one)
grid = Watir::Grid.new(:ring_server_port => 12351,
    :ring_server_host => '127.0.0.1')
 
grid.start(:take_all => true)
 
# for each provider on the grid, launch a new thread to start multiple browsers
threads = []
grid.browsers.each do |browser|
        threads << Thread.new do
                p browser[:hostname]
                p browser[:architecture]
                p browser[:browser_type]
                # in this case we are starting a new IE browser
                b = browser[:object].new_browser(:ie)
                b.goto("http://www.google.com")
                b.text_field(:name, 'q').set("watirgrid")
                b.button(:name, "btnI").click
        end
end
threads.each {|thread| thread.join}

There’s some more examples of using watirgrid with watir-webdriver here.

Read More

Authenticating Proxy with Webdriver and Watir

A question came up on stack overflow about how to route Watir through an authenticating proxy within your script.

The problem here is that you cannot auto authenticate to a proxy server using Chrome –proxy-server=my.proxy.com:3128 or using Firefox.

Chrome will always prompt you for a user name and password on an authenticating proxy when it starts up (via Webdriver or manually). Firefox will not prompt (when launched via WebDriver) and will silently fail.

I *almost* got this to work by modifying headers on the fly with a Firefox add on (Modify Headers). Basic authentication is just base-64 encoded so you can force each request to authorize itself by modifying the Proxy-Authorization request header e.g.:
Proxy-Authorization Basic asHJksaKHs87akhkjah7

However whilst this works for manually launched instances of Firefox, Webdriver launched instances would complain that the add on is not compatible with FF3.6.3 and so once again would fail. The other problem with this approach is that I couldn’t find a similar plugin for Chrome.

Eventually I gave in and went with a slightly more complicated workaround. That is, chaining an authenticated proxy (with squid) running on localhost with a cache_peer relationship to the authenticated proxy running out there on the ‘tubes’. I had thought about using Webrick as a proxy (as per previous posts) but lack of SSL support is a gotcha for me. Squid seemed most appropriate in this case:

To install squid from source:
cd /opt/src
sudo curl -O http://www.squid-cache.org/Versions/v2/2.7/squid-2.7.STABLE9.tar.gz
sudo tar xjf squid-2.7.STABLE9.tar.gz
cd squid-2.7.STABLE9
# I wanted SSL support for this
./configure --enable-ssl
make
sudo make install

Edit the squid.conf (location is normally /etc/squid/squid.conf on linux):
sudo vim /usr/local/squid/etc/squid.conf

# Don't want to run squid as root
# TAG: cache_effective_user
cache_effective_user nobody
# TAG: cache_effective_group
cache_effective_group wheel
# Point this proxy to a peer cache, and provide the logon credentials
# TAG: cache_peer
cache_peer proxy.altentee.com parent 3128 0 default no-query login=username:mypassword
never_direct allow all
# Allow access from my local private network
# TAG: cache_peer
acl localnet src 10.0.0.0/0

Save changes and change ownership as appropriate:
sudo chown -R nobody:wheel /usr/local/squid

Initialize the cache and startup squid:
sudo /usr/local/squid/sbin/squid -z
sudo /usr/local/squid/sbin/squid -D

Now you can configure firefox (using -ProfileManager) or chrome (using –proxy-server=127.0.0.1) to point to your local proxy server. This proxy will then authenticate on your behalf and peer all requests via the original parent.

A little long winded but achieves the outcome. Added side benefit is you’ve now got a local proxy (with access logs) that you can start doing interesting things with in your automation / performance tests like modifying headers and so forth.

Hope this helps!

Read More

Going Parallel … distributed testing across a grid network using Watir

In my last post I made a tongue-in-cheek observation that load testing really doesn’t work by today’s perceived standards for web automation and testing.

Part of my grievance for that was based on frustration in dealing with different load testing platforms. Not frustration born out of incompetence, more the frustration born out writing tests for increasingly complex browsers with an antiquated toolset.

That’s not to say that protocol based application load testing (or the languages they sit on) aren’t powerful in their own right. LoadRunner, JMeter et al. has their place in the load testing world. But I would like to relegate them to the background as useful, sometimes expensive tools to spin up the wheels so to speak. Nothing like a bit of JMeter infused fun to get the wheels smoking on your system under test. But for the real part, measuring user experience on the web platform?

Read More