I’m using MySQL’s AES which is better than older standards. Basically 128 bit key encryption …

To begin, let’s create a [...]]]> There’s been a bit of buzz lately where MySQL databases have been compromised and user passwords being stored in plain text. Bad fu. It’s pretty simple to encrypt passwords in MySQL, here’s how you do it.

I’m using MySQL’s AES which is better than older standards. Basically 128 bit key encryption …

To begin, let’s create a demonstration table. e.g.
CREATE TABLE users ( user_name VARCHAR(20) NOT NULL, user_password BLOB );

So now we have a simple table with a user_name field and user_password field. e.g.

mysql> DESCRIBE users;
+---------------+-------------+------+-----+---------+-------+
| FIELD         | Type        | NULL | KEY | DEFAULT | Extra |
+---------------+-------------+------+-----+---------+-------+
| user_name     | varchar(20) | NO   |     | NULL    |       |
| user_password | blob        | YES  |     | NULL    |       |
+---------------+-------------+------+-----+---------+-------+

If we were creating a new user we would insert using the AES_ENCRYPT function. e.g.

INSERT INTO users VALUES ('tim.koopmans', AES_ENCRYPT('mypassword', 'Q4QjOyKANXj3qO'));

In this case I’ve used AES_ENCRYPT which encrypts a string and returns a binary string. The AES (Advanced Encryption Standard) algorithm is encoded with a 128-bit key. The second parameter is the key string, important to keep that secure!

On its own, this offers a good level of security. If the database was compromised the hacker will see information like this. e.g.

mysql> SELECT * FROM users;
+--------------+------------------+
| user_name    | user_password    |
+--------------+------------------+
| tim.koopmans | ??iu?hQ:???Ti?} |
+--------------+------------------+

The problem with this, is that the user_password field can still be ‘guessed’ via brute force methods. Every password that is the same i.e. ‘mypassword’ will also have the same binary string.

You can add a salt to this by prefixing random digits to the plain text password before you encrypt. For example, we can use the first 12 digits of a UUID. e.g.

INSERT INTO users VALUES ('joe.blow', AES_ENCRYPT(CONCAT(LEFT(UUID(),12), 'mypassword'), 'Q4QjOyKANXj3qO'));

The result set now looks like this if compromised.

mysql> SELECT * FROM users;
+--------------+----------------------------------+
| user_name    | user_password                    |
+--------------+----------------------------------+
| tim.koopmans | ??iu?hQ:???Ti?}                  |
| joe.blow     | ?M}69?H?0?H}?/{.?we|?A?P7ZD?R?   |
+--------------+----------------------------------+

Every password that is encrypted should look different which will further complicate dictionary attacks.

To determine a password from a stolen hash, an attacker cannot simply try common passwords (such as English language words or names). Rather, they must calculate the hashes of random characters (at least for the portion of the input they know is the salt), which is much slower.

For best results, the value/length of your salt should be kept secret (and separate from your database). You should also keep the key secret of course!

Now if we ever need to extract the password we use the AES_DECRYPT function. e.g.

mysql> SELECT SUBSTR(AES_DECRYPT(user_password, 'Q4QjOyKANXj3qO'),13) AS password FROM users WHERE user_name = 'joe.blow';
+------------+
| password   |
+------------+
| mypassword |
+------------+

In this case we know the first 12 digits were the salt so I used SUBSTR after decrypting the hash to get the original password.

A couple of caveats, encrypting passwords in this fashion is reversible i.e. If your key gets compromised, then the hacker has access to all your passwords. Another option is to reverse the plain text and key parameters of AES_ENCRYPT, where the user’s password now becomes the key, and the key becomes the plain text. Even if the database and key is compromised, it is hard to crack user passwords. You would not be able to recover lost passwords of course … Security through obscurity!

In this [...]]]> A common requirement when load testing is to populate subsequent requests with dynamic data from previous requests. If you’re thinking about using a database to store static test data, such as usernames, paswords, credit card numbers etc then read on to find out how to extract this data at runtime and populate JMeter variables.

In this example we’ll need to add a JDBC Connection Configuration to our test plan. This will define and control the way in which JMeter will access your database.

Notice I’ve left the defaults for connection pool properties, although you may need to tweak this depending on how you’ll access the data. I’ve used a JDBC connection URL that accesses a local mysql server listening on port 3306. The database in use is called watirgrid

I’ve also defined the JDBC Driver class and specified the username / password to connect with.

Now we can make a JDBC request using the JDBC Request sampler. It will look something like this:

The Variable Name matches the JDBC Connection Config Variable Name i.e. mysql

I’m using a select statement that will print out results in a JSON-like format. That query is as follows:

SELECT
CONCAT("[",
GROUP_CONCAT(
CONCAT("{email:'",email,"'"),
CONCAT(",login_count:'",login_count),"'}")
,"]")
AS json FROM users

In doing this, it will return a result set that looks like this:

+-------------------------------------------------+
| json                                            |
+-------------------------------------------------+
| [{email:'tim.koops@gmail.com',login_count:'1'}] |
+-------------------------------------------------+
1 row IN SET (0.00 sec)

Now we need to add a Post Processor Regex Extractor to the JDBC request as follows:

The reason we do this is so we can extract the results from the JSON string returned by the database query. For example, to extract and populate a variable called ${email} our PPRE would look like this:

Now we’re cooking! Add a Debug Sampler to confirm everything works as you’d expect. You should see results like this in the response data

email=tim.koops@gmail.com
email_g=1
email_g0=email:'tim.koops@gmail.com'
email_g1=tim.koops@gmail.com
mysql=org.apache.jmeter.protocol.jdbc.config.DataSourceElement$DataSourceComponentImpl@10d69502

Attached is a demo test plan: jdbc_example.jmx

If that script references a data file (for parameters) then that data file is shared by *all* vuser groups. e.g. my parameter called {values} has

When running, vuser group_b will error with these settings e.g.

i.e.

So 2 separate vuser groups still equals one shared parameter file (and settings). Watch out [...]]]> When running multiple vuser groups with the same script i.e.

If that script references a data file (for parameters) then that data file is shared by *all* vuser groups.
e.g. my parameter called {values} has

When running, vuser group_b will error with these settings e.g.

i.e.

So 2 separate vuser groups still equals one shared parameter file (and settings).
Watch out for this!

Bug #8523 details MySQL’s deficiency in its ability to store and retrieve datetime values with millisecond or microsecond precision. When [...]]]> NOTE: This doesn’t generate a timestamp accurate to milliseconds. It just stores it in two fields. AFAIK there is no way to generate such a thing. It is accurately described here. Sorry for the confusion =)

Bug #8523 details MySQL’s deficiency in its ability to store and retrieve datetime values with millisecond or microsecond precision. When performance testing and perhaps storing trace logs or audit information in a MySQL table for later analysis, this shortfall is quite frustrating.

My hacky way to deal with this is to store the millisecond component as its own integer. Say for example you are interested in the difference between a Put datetime and Get datetime for MQ, your table might look like this:

CREATE TABLE `msecs` (
  `put_datetime` datetime DEFAULT NULL,
  `put_msecs` int(11) DEFAULT NULL,
  `get_datetime` datetime DEFAULT NULL,
  `get_msecs` int(11) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1;

So in this table I am storing the datetime component in its native MySQL format and the millisecond component as an integer. Here are some examples:

Now to retrieve and compare the values, I get the difference using TIME_TO_SEC for the datetime component, multiply it by 1000 to convert to milliseconds then add to the difference of the milliseconds component using basic maths:

SELECT ((TIME_TO_SEC(get_datetime) - TIME_TO_SEC(put_datetime))*1000)
+ (get_msecs-put_msecs)
AS diff_msecs
FROM msecs

That gives you difference in milliseconds for the demo table along these lines:
2300
1700
300
600700
700

Hopefully that makes sense. Pipe up if you have an easier solution!

I’m currently using a wamp box to serve up content for performance metrics. What I needed was a Ruby script [...]]]> If you are a rails fan you will probably recommend using active record instead, but if you just want to hack away at your mysql db from within your Ruby code, you may find this handy…

I’m currently using a wamp box to serve up content for performance metrics. What I needed was a Ruby script which could interrogate data on this installation.

You can download the binary required for this from here. After you install the gem (I do this off-line as my command prompt doesn’t allow me internet access here at work) you may find that you get an error similar to the following:
"NameError: uninitialized constant Mysql"

See what’s happening?

To find out what’s happening, you can jump into an irb terminal and type the following:
irb(main):002:0> require 'mysql'

When you do you will probably get a popup dialog like this:

Basically if you read the README that comes with the gem you will notice:

Dependencies …
For this to work, you must have libmysql.dll in your PATH environmental variable. If you have MySQL installed locally, just make sure that \bin is in your path. If you don’t have MySQL installed locally, you can install one or more of the MySQL tools, find the libmysql.dll included in their bin directory, and copy it to the %SystemRoot%\System32 folder.

So all you need to do is find out where your libmysql.dll lives:
D:\wamp>dir /b /s LIBMYSQL.dll
D:\wamp\bin\apache\apache2.2.8\bin\libmysql.dll
D:\wamp\bin\mysql\mysql5.0.51a\bin\libmySQL.dll
D:\wamp\bin\php\php5.2.5\libmysql.dll

And then add that bin directory to your environment path:
D:\wamp\bin\mysql\mysql5.0.51a\bin

Hope that saves some of you some time in getting connectivity on your windows boxes!
Attached below is some sample code to test your config.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

require 'rubygems'
require 'mysql'
 
def with_db
    dbh = Mysql.real_connect('localhost', 'myuser', 'mypass', 'mydb', 3306)
    begin
        yield dbh
    ensure
        dbh.close
    end
end
 
with_db do |db|
    res = db.query('select * from tablename')
    res.each do |row|
        puts row[1]
    end
end

Virtual Table Server (VTS) first edition, introduced the capability for LoadRunner virtual users, WinRunner and XRunner, to communicate in real time. Data extraction and communication was simplified but limited to column/queue level operations. The new Virtual [...]]]> Mercury’s Virtual Table Server (VTS) provides the following functionality when sharing data between vusers in your typical LoadRunner scenario …

Virtual Table Server (VTS) first edition, introduced the capability for LoadRunner virtual users, WinRunner and XRunner, to communicate in real time. Data extraction and communication was simplified but limited to column/queue level operations. The new Virtual Table Server II provides a higher degree of data manipulation and 5-10 times better performance. From row level queries, retrievals, updates, insertions, unqueue entries, to database access, VTS II provides the functionality and ease for enhanced inter-process, inter-virtual user communication

A useful tool in itself, but with some annoying limitations. MyLoadTest elaborates on this:

The API is simple, but unfortunately does not allow you to write SQL queries; instead you must use the functions provided, like lrvtc_retrieve_row() and lrvtc_query_row()…

Read on for an alternative when using web vusers in LoadRunner.

Using WAMP (Windows, Apache, MySql & PHP) I came up with a simple alternative called VTSFaker. The concept relies on the use of web_reg_save parameters obtained from a simple web_url call to a ‘central’ WAMP server. I say central in the sense you might have n load generators in your environment all sharing the single WAMP server.

In the LoadRunner script I have abstracted the call to the WAMP server using a custom dbApi() action which you can load in your vuser_init or include as a custom header.

In its simplest form, if you wanted to parametize a value called ‘advertiserId’, you simply need to call the dbApi function as follows:
dbApi("selectUniqueOnce", "advertiserId", "advertisers","","");

The purpose of the dbApi action is to call a web page hosted by your WAMP installation using query based parameters as follows:
http://d112295/VTSFaker/api.php?method=selectUniqueNext&column=advertiserId&table=advertisers&vuser=None_-1_0&value=&where=

This in turn returns a formatted page with the result in a custom tag. LoadRunner then parametizes that result using a web_reg_save_param function call as follows:
web_reg_save_param(column,"LB=","RB=","Search=Body","Notfound=warning",LAST);

All of this logic is abstracted in the dbApi action itself which hopefully simplifies the whole process.

In the background, I’ve created a simple API using PHP accessed via a webpage that makes calls to your MySql database. I think you get the picture … BTW, if you’re worried about performance, I did some preliminary tests on a 2GB RAM machine with the load generator and WAMP installation collocated with up to 600 concurrent vusers with no problems. Provided you index your MySql tables properly, result sets are returned in milliseconds, with not much fuss at all. In any case I’ve added an lr_start/end transaction statement to each API call, so that you could if necessary, subtract those average response times from your measured transactions if WAMP performance was degraded during your test run…

To get this all working follow the steps below:

1. Setup a WAMP installation. I’m using WAMPSERVER v2.0 which can be obtained for free at http://www.wampserver.com

2. Setup your database tables; I’m assuming you know how to drive MySql. Here’s an example table I’m using at present:

CREATE TABLE `advertisers` (
`id` int(4) NOT NULL auto_increment,
`advertiserId` varchar(64) default NULL,
`productId` varchar(64) default NULL,
`hasArtwork` tinyint(4) default NULL,
`vuserId` varchar(128) default NULL,
`update` timestamp NULL default NULL on update CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
KEY `advertiserId_idx` (`advertiserId`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

Note: the mandatory use of an id column (to help sort result sets) and a vuserId column (to provide unique result sets). The others are optional depending on what data you want to retrieve/store.

3. Setup your VTSFaker aliased website on WAMP (Apache -> Alias Directories -> Add an alias -> VTSFaker) using the example PHP files here.

4. Add to your vuser_init or custom header the dbApi action using the example code below.

/* ---------------------------------------------------------------------
Custom DB API for MySql

// enum errors
httpRC = web_get_int_property(HTTP_INFO_RETURN_CODE);
if (httpRC != 200){
if (httpRC < 200){
lr_error_message("Error: HTTP Return Code %", httpRC);
}
else {
lr_error_message("Error: HTTP Return Code %i", httpRC);
}
return 1;
}
return 0;
}


Enjoy