Wow, wasn’t that long ago when I was promising things that were harder, better, faster and stronger. I think I got part of the way there. The core reason I set up Altentee was to provide reliable (and potentially cheaper) alternatives to traditional licensed performance testing tool sets. There’s no doubt Altentee can test at the limits of your typical web app using tools that cost zilch. The bigger challenge I’ve found this year is convincing potential clients that they really don’t need to spend that much. I’ve even offered a free load test on our homepage to help illustrate this point.
We were lucky to be selected by the development team at Cordelta to help them automate and performance test a high profile public website called MyHospitals. We were able to test millions of hits per hour from domestic and international locations in a wide variety of load scenarios. The success of this approach was underpinned by the following:
- An open minded project / development team not coupled to a ‘must-have-most-expensive-toolset-to-do-job’ mentality
- An open minded performance test analyst (me!) who believes Excel really is the grandpa of charting, R is the grandma of stats, Sparklines are the only way to present time series data to management, ANY tool can simulate load via HTTP/S and that there is no real distinction between good software testers or developers (only hard work separates the best from the worst).
- That 2010 come-no-doubt 2011 buzz word… Cloud
As I sit here pluggin’ our alternative approach at Altentee, I can see the rise of other more successful punters taking on the big kids. I sit and [continue to] chuckle at the reaction to LR pricing and LR zealots who will fall on their swords over LR itself. I have come to realise one thing though, it is not about the tool, or even the alternate tool like perhaps I first thought. It is more about the freedom of choice.
To tackle MyHospitals I was free to choose and implement the following tools:
- WatirGrid to orchestrate a small flotilla of IE and FireFox based browsers based on Watir
- JMeter to add more at the protocol level of performance testing
- httperf to do some basic benchmarking, similar to my front page
- numbrcrunchr to pull together system metrics and make for easier analysis
- A variety of Australian cloud providers and of course Amazon EC2 to host the test environment
It has been a great year. I’m not entirely free of the commercial chains just yet and am still needing the LR type work to prop up this approach, but I hope 2011 brings about some fresh thinking in performance and test automation with hopefully me somewhere amidst that space.
Read More
Here’s an idea… Ever been in a LoadRunner contract where you’ve got a ton of scripts to write for business transactions that have many many design steps?
Even worse, have you had to write those scripts against a development environment with an unstable code base and shocking performance? Sound familiar? Ever felt the frustration of re-recording manual test cases back into LoadRunner in said environments. Why wouldn’t you just automate the script recording process?
Read More
Previously I identified how to setup pattern matching with Regex in LoadRunner.
Here’s how you implement search and replace functionality with the same POSIX libraries used in the previous example.
Create a replace function in your favourite headers file or wherever you keep your framework’y stuff… You also need the pcre3.dll and pcreposix.h header added as files to your script.
replace(const char *string, char *pattern, char *replace, char *match)
{
int length;
int status;
int eflag;
char buf[1024] = "";
char out[1024] = "";
regex_t re;
regmatch_t pmatch[128];
lr_load_dll("pcre3.dll");
if((status = regcomp(&re, pattern, REG_EXTENDED)) != 0){
regerror(status, &re, buf, 120);
lr_output_message("Match PCRE Exit 2");
return 2;
}
while(status = regexec( &re, string, 1, pmatch, eflag)== 0){
strncat(out, string, pmatch[0].rm_so);
strcat(out, replace);
string += pmatch[0].rm_eo;
eflag = REG_NOTBOL;
}
strcat(out, string);
lr_save_string(out, match);
}Then anywhere in your actions you can call on this function, passing it the string buffer you wish to operate on, a search value, a replace value and the name of the LoadRunner parameter you want the result saved into.
lr_save_string("FOO%20BAR%20DING", "buffer");
replace(lr_eval_string("{buffer}"),"%20"," ", "custom");What you’ll end up with is something like this:
Home.c(6): Notify: Saving Parameter "buffer = FOO%20BAR%20DING"
Home.c(7): Notify: Parameter Substitution: parameter "buffer" = "FOO%20BAR%20DING"
framework.h(135): Notify: Saving Parameter "custom = FOO BAR DING"
Enjoy.
Read More
Previously I identified a way in which to test SPNEGO or Kerberos authentication with LoadRunner. However this implementation was buggy in the sense that if you ran your load tests under reasonable load with the WinInet replay engine (instead of sockets) you were likely to encounter the following error:
Error -27492: "HttpSendRequest" failed, Windows error code=12057 (certificate revoked) and retry limit (0) exceeded for URL="
https://someplacesecure.com.au/secure.html", Snapshot Info [MSH 1 2]
This error occurs when using WinInet replay instead of sockets with Integrated Authentication enabled in run-time settings. The purpose of this was to allow vusers to use SSO with SPNEGO authentication in an IBM WebSEAL environment.
After spending some time with the mystical HP level 3 support, they identified an undocumented flag which helps out significantly in this. So, instead of using the WinInet replay engine (which is not encouraged by HP) you should do something similar to the following.
vuser_init()
{
// Preferred run-time settings
// Browser -> Browser Emulation
// [ ] Simulate a new user on each iteration
// Preferences -> Options
// Enable Integration Authentication [Yes]
web_set_sockets_option("INITIAL_BASIC_AUTH","1");
web_set_user("DOMAIN.LOCAL\\username",
"password",
"someplacesecure.com.au:443");
web_url("myportal",
"URL=https://someplacesecure.com.au/wps",
"Resource=0",
"Referer=",
"Mode=HTML",
LAST);
return 0;
}The magic is in the web_set_sockets_option("INITIAL_BASIC_AUTH","1") flag. Set that and you can then use LoadRunner in Sockets mode which as it turns out, is much more stable.
Enjoy.
Read More
When running multiple vuser groups with the same script i.e.
If that script references a data file (for parameters) then that data file is shared by *all* vuser groups.
e.g. my parameter called {values} has
When running, vuser group_b will error with these settings e.g.
i.e.
So 2 separate vuser groups still equals one shared parameter file (and settings).
Watch out for this!
Read More
