Ruby Web Bench

This is the home of RWB, the Ruby Web Bench, a replacement for ab. RWB is a pretty small application, so it's got a pretty small website.

Table of Contents

Introduction

RWB was written to scratch an itch that ab couldn't quite reach. I wanted to be able to build a weighted list of urls with which to test a website. RWB will become a little language in which you can write such lists, run tests, and build reports. For now, I'm trying to get the engine and the reporting right. Please use RWB and let me know what else it ought to do. Criticism, suggestions, and especially patches are welcome.

Current Version & Downloads

The current release is 0.2.1 and can be downloaded from RubyForge.

Older versions can be found here:

News

version 0.2.1
  +  added quartile graphs by url or overall
  +  added more information to response objects
  +  released as a gem w00t!

Version 0.2.0
  +  added proxy handling throughout

Version 0.1.1
Changes in this version:
  +  added three kinds of warmup methods



Version 0.1.0
Changes in this version:
  +  starting to add documentation
  +  added more granular control over reporting levels
  +  fixed standard deviation


Version 0.0.1
Changes in this version:
  +  made reporting more modular
  +  added reporting based on slices of the results
  +  added standard deviation for mean time per request
  +  fixed standard deviation

Version 0.0.0
Changes in this version:
  +  Initial Release

To Do

In no particular order, these are some of the things I want to do with RWB. If you're interested in implementing one or more of them (or something not listed here), please contact me and we can coordinate our efforts.

Samples

I've created a screencast showing the creation of simple rwb script. It's kind of a toy screencast, but it's a start.

Here are two quick examples of scripts using RWB and their output. (Note that I'm showing both stdout and stderr in the output.)

A very simple test
require 'rwb'

urls = RWB::Builder.new()

urls.add_url(10, "http://www.example.com")
urls.add_url(10, "http://www.example.com/nonesuch")
urls.add_url(70, "http://www.example.com/entries")

queries = ['foo+bar', 'bar+baz', 'quux']
urls.add_url_group(10, "http://www.example.com/search?", queries)

tests = RWB::Runner.new(urls, 100, 5)

tests.run
tests.report_header
tests.report_overall([0.5, 0.9, 0.99, 0.999])

This script does a couple of things. It sets up an RWB::Builder object called urls, which contains three Url objects and a UrlGroup object. The three Urls consist of a weight and a URL -- the total of the weights don't matter, they're not percentages. The UrlGroup has a weight, a base URL and an array of extensions to be added to that URL.

Once the urls object is set, an RWB::Runner object called tests is built. This object is then used to run the tests and build a report header and an overall performance report. The output of running this script is shown below:

Output from a very simple test
$ ruby -Ilib vsim.rb
completed 10 runs
completed 20 runs
completed 30 runs
completed 40 runs
completed 50 runs
completed 60 runs
completed 70 runs
completed 80 runs
completed 90 runs
completed 100 runs
Concurrency Level:       5
Total Requests:          100
Total time for testing:  1.29918 secs
Requests per second:     76.9716282578242
Mean time per request:   64  msecs
Standard deviation:      9
Overall results:
        Shortest time:  10 msecs
        50.0%ile time:  62 msecs
        90.0%ile time:  78 msecs
        99.0%ile time:  88 msecs
        99.9%ile time:  88 msecs
        Longest time:   88 msecs

You're able to do a lot more with RWB, as shown below.

A slightly more involved test
require 'rwb'

urls = RWB::Builder.new()

urls.add_url(2, "http://www.example.com")
urls.add_url(3, "http://www.example.com/nonesuch")
urls.add_url(3, "http://www.example.com/entries")

queries = ['foo+bar', 'bar+baz', 'quux']
urls.add_url_group(10, "http://www.example.com/search?", queries)

tests = RWB::Runner.new(urls, 10_000, 50)

tests.sla_levels = [0.5, 0.9]

tests.warm_up(10)

tests.run
tests.report_header
tests.report_by_time
tests.report_urls([0.5, 0.9, 0.99, 0.999])

Which creates much more output.

Output from a more involved test
$  ruby -Ilib minv.rb
warming up with 10 runs
1 2 3 4 5 6 7 8 9 10
completed 1000 runs
completed 2000 runs
completed 3000 runs
completed 4000 runs
completed 5000 runs
completed 6000 runs
completed 7000 runs
completed 8000 runs
completed 9000 runs
completed 10000 runs
Concurrency Level:       50
Total Requests:          10000
Total time for testing:  38.848752 secs
Requests per second:     257.408526276468
Mean time per request:   41  msecs
Standard deviation:      12
Results by time:
results for requests 0 - 2000
        Shortest time:  23 msecs
        50.0%ile time:  39 msecs
        90.0%ile time:  50 msecs
        Longest time:   93 msecs
results for requests 2000 - 4000
        Shortest time:  24 msecs
        50.0%ile time:  42 msecs
        90.0%ile time:  54 msecs
        Longest time:   68 msecs
results for requests 4000 - 6000
        Shortest time:  25 msecs
        50.0%ile time:  41 msecs
        90.0%ile time:  57 msecs
        Longest time:   82 msecs
results for requests 6000 - 8000
        Shortest time:  26 msecs
        50.0%ile time:  38 msecs
        90.0%ile time:  58 msecs
        Longest time:   81 msecs
results for requests 8000 - 10000
        Shortest time:  25 msecs
        50.0%ile time:  33 msecs
        90.0%ile time:  62 msecs
        Longest time:   102 msecs
Results for http://www.example.com :
        Shortest time:  35 msecs
        50.0%ile time:  52 msecs
        90.0%ile time:  67 msecs
        99.0%ile time:  81 msecs
        99.9%ile time:  98 msecs
        Longest time:   102 msecs
Results for http://www.example.com/nonesuch :
        Shortest time:  24 msecs
        50.0%ile time:  34 msecs
        90.0%ile time:  55 msecs
        99.0%ile time:  62 msecs
        99.9%ile time:  80 msecs
        Longest time:   83 msecs
Results for http://www.example.com/entries :
        Shortest time:  25 msecs
        50.0%ile time:  33 msecs
        90.0%ile time:  55 msecs
        99.0%ile time:  61 msecs
        99.9%ile time:  72 msecs
        Longest time:   79 msecs
Results for http://www.example.com/search?:
        Shortest time:  23 msecs
        50.0%ile time:  34 msecs
        90.0%ile time:  54 msecs
        99.0%ile time:  62 msecs
        99.9%ile time:  88 msecs
        Longest time:   93 msecs


Pat Eyler
Last modified: Tue Nov 8 09:24:48 CST 2005