At some point in our respective programming careers, most of us have heard that Test-Driven Development (TDD) is what we should be doing.
The TDD thesis is that we write tests (known as unit tests) that consume the functions, classes, and interfaces of the code that we intend to write before we write it, and that doing this improves the quality of our code due to increased foresight and better ability to catch a specific type of bug known as a regression, which is the technical term your boss uses for something that worked fine before you added that last bit of code.
The gist of TDD is:
- Each test should test one unit of code.
- Tests should not cross process boundaries in a program.
- Tests should not establish network connections.
If you’re writing tests that don’t follow these rules, tsk tsk, you’re doing unit testing wrong! The mantra is that instead of doing this, you need to tease apart the code so that you can introduce mocks and fake access to those other components.
While I’m on board with the idea that we should be writing code with testing in mind, it is all too easy to get swept away with the purist TDD approach, and — before you know it — you’re not actually writing code anymore; you’re thinking about how to change your real code so that you can write fake code to test your real code. (As a humorous semi-diversion, getting lost in JUnit-derived test frameworks just makes me think of building a spice rack.)
The problem I see with this approach, especially if you’re retrofitting test facilities to a project, is that the cost of implementing all the mocks and fakes can be prohibitive, and it may not even be an effective use of time! Are your users going to run your fake code? What if there’s a bug in your fake code that doesn’t match up to the real world? Will your tests save you from that problem?
Integration Testing
There’s another type of test known as an integration test that describes testing the ability of your program to talk to other modules, databases, and services. The idea is that your test suite starts up instances of its own services (web server, database, &c.) for the test programs to execute against.
Running its own services guarantees that everyone who runs the tests are running with a consistent environment; this is how you eliminate the “works for me” class of problem.
I’ve recently started to build up a test suite for Mtrack (yes, it is long overdue) that uses this approach. Mtrack has a couple of key interfaces:
- Web UI (PHP, with a bit of Backbone.js for the frontend)
- REST API
- SSH access to repositories
It also relies on a couple of services, depending on how it has been configured:
- Zend Framework Lucene Full Text search index
- Solr Full Text search index
- PostgreSQL database
- SQLite database
A driver script called runtests sets up an Apache web server,
initializes a SQLite database, starts Solr, starts an SSH server, and
launches the Selenium server. It then runs the Perl prove tool to
execute the test programs.
I tend to use a Test::More-derived test framework in my integration tests (because we work with Perl and other systems a lot); you can certainly use PHPUnit if you’re more comfortable with that. I like the Test::More style, because it feels very lightweight.
With the services up, we can then write tests that exercise our code via the REST API; these tests can use PHP’s streams or cURL functions to talk to the web server. The test driver for Mtrack exports the web server port number to an environment variable, so the test program knows which server it should talk to.
The Mtrack REST API tests look something like this (simplified for the sake of this article):
<?php
require 'Test/init.php' // Include test functions.
plan(10); // Plan to run 10 tests.
// This makes a POST to the /ticket API, returning the
// HTTP status code, the headers, and the content. It
// automatically converts to/from JSON and gets the
// host and port from the environment variables set by
// the runtests script.
list($st, $h, $body) = rest_api_func('POST', '/ticket',
null, null, array(
'summary' => 'my test ticket',
'description' => 'the body of the ticket'
));
if (!is($st, 200, 'created ticket')) {
// Failed; this is not something we expect, so dump
// out the body.
diag($body);
}If the script doesn’t run 10 tests, the prove utility will fail the test
run, because something unexpected happened. If the is() function
returns false (indicating that something didn’t work), then that will
cause the test to fail and indicate where and why. We augment that by
outputting some diagnostics via the diag() function.
We use Selenium (albeit very lightly right now) to drive through some workflows like creating a new ticket via the web UI (and this has already caught a couple of corner cases for fresh installs).
The SSH server isn’t currently used by tests, but it is there to validate
that the repository creation and permissions work correctly when folks
use their SCM tools (hg, git, svn) over SSH.
I use a couple of command line options to toggle Selenium on and off (it takes more than a few seconds to launch, making the tests take longer to run), and another to switch between using the ZF Lucene code and the Solr search server.
Each time the runtests script is run, it wipes out the Solr, SQLite, and
other state that was initialized from the last run so that we have a
consistent environment on each run.
Summing Up
Automated testing is definitely something that I would advocate to anyone writing any kind of serious code. Designing your code for testing from the outset is also strongly desirable. Unit testing is important, but don’t get swept up in the purist approach; mocks and fakes have their uses, but if you’re investing a significant amount of time to build them up, you may want to consider spending that time on automating your integration tests instead.
Developer Gift
It can be tricky to think of good gifts; so much of what software developers do is intangible. I’ve come to appreciate things that help me relax and unwind, and these days, this tends to be assisted by a nice pour of a single malt whiskey. Depending on your budget and your insight into the tastes of your prospective gift recipient you might consider:
- Whiskey stones. “on the rocks” is not something you hear too often from a single malt drinker, because the ice melts and dilutes the complex flavors. Whiskey stones are a non-ice alternative.
- A nice set of tumblers. (Here’s another option.) I have a couple of nice, weighty glasses that I particularly enjoy handling. If your giftee is really into tasting and nosing, there are special whiskey snifters to be had.
- A bottle of whiskey. There’s an element of taste to factor in here; some have very strong flavors that might be fine for an occasional glass, but otherwise unappreciated. If you have no idea, try to find out if there are favorites, and go for something similar. If you really have no idea, find someone else that does, and try to avoid the cheaper blends; it’s better to go for a smaller bottle of something nice than a larger bottle of something not.
