I’m a big believer in writing automated tests for applications. Whether it’s before coding or after, tests keep the application working and easy to maintain. But what can you do when the application and test suite grow and start taking a longer and longer to run? Redmine is starting to be affected by this problem, it’s current test suite is taking about 4 minutes to run. That’s 3 minutes too long for me.

There are several options for speeding up tests but how do you know which ones to speed up? How do you know if your changes actually helped?

Enter test_benchmark

test_benchmark is:

A ruby gem and rails plugin to show you how slow your Test::Unit tests run. Useful for troubleshooting/isolating slow tests.

It runs tests like normal but adds benchmarking times for each test method and class. With Ruby on Rails 2.3, installation and setup is an easy 3 step process.

1. Install the gem

test_benchmark is available as a Rubygem, named… test_benchmark! You can install it directly into your Rails application as a plugin or a gem but I prefer to install tools like this systemwide so I can use them with any application.

sudo gem install test_benchmark

2. Configure the gem in Rails

Once the gem is installed, you can load it into Rails by adding it to the testing environment. Just edit config/environments/test.rb and add the following line:

config.gem "test_benchmark"

To make sure it’s loading, you can run rake gems for the test environment and make sure test_benchmark is shown.

$ rake gems RAILS_ENV=test
(in /home/edavis/dev/redmine/redmine-core)
 - [I] thoughtbot-shoulda 
 - [I] nofxx-object_daddy 
 - [I] mocha 
    - [R] rake 
 - [I] test_benchmark 

I = Installed
F = Frozen
R = Framework (loaded before rails starts)

3. Run your tests

To use test_benchmark, you don’t need to do any more configuration. Just run your tests and it will print out the benchmarking results right to the console.

rake test:units
(in /home/edavis/dev/redmine/redmine-core)
/usr/bin/ruby1.8 -I"lib:test" "/usr/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake/rake_test_loader.rb"
Loaded suite /usr/lib/ruby/gems/1.8/gems/rake-0.8.7/lib/rake/rake_test_loader
Started
...........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
Finished in 112.694542 seconds.

475 tests, 2029 assertions, 0 failures, 0 errors

TEST BENCHMARK TIMES: OVERALL
54.475 test_load(DefaultDataTest)
2.904 test_issue_edit(MailerTest)
1.514 test_lost_password(MailerTest)
1.424 test_account_information(MailerTest)
1.311 test_register(MailerTest)
1.242 test_call_hook_should_not_change_the_default_url_for_email_notifications(Redmine::Hook::ManagerTest)
1.210 test_clear_listeners(Redmine::Hook::ManagerTest)
1.204 test_issue_add(MailerTest)
1.190 test_call_hook_with_project_added_to_context(Redmine::Hook::ManagerTest)
1.158 test_call_hook_from_view_with_request_added_to_context(Redmine::Hook::ManagerTest)
1.150 test_call_hook_with_context(Redmine::Hook::ManagerTest)
1.136 test_call_hook_from_view_with_project_added_to_context(Redmine::Hook::ManagerTest)
1.134 test_add_listener(Redmine::Hook::ManagerTest)
1.112 test_call_hook_from_view_with_controller_added_to_context(Redmine::Hook::ManagerTest)
1.107 test_call_hook_from_view_should_join_responses_with_a_space(Redmine::Hook::ManagerTest)

Test Benchmark Times: Suite Totals:
54.485 DefaultDataTest
15.467 Redmine::Hook::ManagerTest
13.373 MailerTest
8.125 ProjectTest
4.165 IssueTest

You’ll see two different reports, one for the overall times and one for the suite times (test classes). Looking at Redmine’s report above, you can see the test_load from the DefaultDataTest suite is taking almost 55 seconds to run. That’s almost 25% of the entire test suite. So if we could optimize that test it would have a significant impact on the speed of the rest of the tests. Next we’ll take a look at the test and see what we can do to speed it up.

Eric Davis