HttpWatch, a Free Alternative Using Ruby

When performance testing I often need to spy on TCP (http) traffic between the client and server on a frequent basis. This is pretty much a staple of any performance test script development effort.

In the past I’ve relied heavily on trace options within LoadRunner and the like, but often find this a tad cumbersome. LiveHTTPHeaders is a great plugin for Firefox but it only captures header information. WireShark is great if you need to ‘go deep’, but often equates to opening a walnut with a sledgehammer. FireBug and IE Developer Toolbar are great to see what’s happening on a page-by-page basis. TCPdump is great if I’m on a Mac or Linux box, but it can be tricky parsing binary data. In any case most client sites are windows based for me.

So that’s a great list of tools to help you correlate and sniff out dynamic data, but the closest I’ve found for IE to doing what I want, that is, capturing all HTTP data going back and forth with the ability to filter for specific information from a console/command line is HttpWatch. A $295USD price tag / crippled basic (free) edition really forced me down this path…to roll my own HttpWatch using Ruby and the WEBBrick proxy.

Read on for a description of the code.

The script itself when run, will listen on a port (e.g. 9090) which you configure your browser to send all requests through. (Note: FoxyProxy plugin is great for fast switching between different configurations) An optional parameter argument allows you to search using regex for the information you are trying to find. For example, I frequently need to know where all occurrences of a hidden field might turn up in the response body, so that I know exactly where to correlate my data in the corresponding previous requests. This is important for both LoadRunner and JMeter scripts (or any web-based performance test harness for that matter).

When executed, the script will then parse all requests and responses (with options to display header information as well if required) and just output the results to your console/command line. the following screenshot demonstrates:

Using an ANSI colour scheme (win32console gem on windows or highline gem on mac) any lines containing the search filter will be highlighted in green with the actual line number in the response body where it occurs. You can also see which GET/POST request triggered this. The code itself is pretty simple as the following demonstrates:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51

# A HttpWatch proxy clone (without the license!)
require 'rubygems'
require 'webrick/httpproxy'
 
begin
 require 'Win32/Console/ANSI' if PLATFORM =~ /win32/
 require 'highline/import'    if PLATFORM =~ /darwin/
rescue LoadError
 raise 'You must gem install win32console or highline to use color on Windows/OSX'
end
 
@proxy_port    = ARGV[0] || 9090
@search_body   = ARGV[1]
 
# Optional flags
@print_headers  = false
@print_body     = true
@pretty_colours = true
 
server = WEBrick::HTTPProxyServer.new(
    :Port => @proxy_port,
    :AccessLog => [], # suppress standard messages
 
    :ProxyContentHandler => Proc.new do |req,res|
        puts "-"*75
        puts ">>> #{req.request_line.chomp}\n"
        req.header.keys.each do |k|
            puts "#{k.capitalize}: #{req.header[k]}" if @print_headers
        end
 
        puts "<<<" if @print_headers
        puts res.status_line if @print_headers
        res.header.keys.each do |k|
            puts "#{k.capitalize}: #{res.header[k]}" if @print_headers
        end
        unless res.body.nil? or !@print_body then
            body = res.body.split("\n")
            line_no = 1
            body.each do |line|
              if line.to_s =~ /#{@search_body}/ then
                puts "\n<<< #{line_no} #{line.gsub(/#{@search_body}/,
                  "\e[32m#{@search_body}\e[0m")}" if   @pretty_colours
                puts "\n<<< #{line_no} #{line}" unless @pretty_colours
              end
              line_no += 1
            end
        end
    end
)
trap("INT") { server.shutdown }
server.start

If you can’t get the win32 console binaries from your dos prompt (i.e. behind a firewall or proxy) just install the attached zip file to \lib\ruby\site_ruby\1.8\
download http_watch_proxy

Note: this proxy won’t support HTTPS communication, so you will need to disable the proxy on whatever page uses HTTPS (typically logon screens etc) then switch back to it for remaining transactions…

Enjoy