Unit tests

There was a time, not that long ago, when tests were something you wrote once, ran by hand a few times to convince yourself the code worked, and then threw away. Test Driven Development changed that: tests became permanent, automated, and as much a part of the codebase as the code they check. This chapter is about what happens once that's true - namely, that test code needs the same care as production code, because a test suite that rots is worse than no test suite at all.

The Three Laws of TDD

TDD can be reduced to three rules, and the interesting thing about them is how small a loop they force you into:

1. You may not write production code until you've written a failing unit test for it.
2. You may not write more of a unit test than is sufficient to fail (and a compile failure counts as a failure).
3. You may not write more production code than is sufficient to pass the currently failing test.

Followed literally, this means you're rewriting test and production code in a cycle that repeats every thirty seconds or so. The immediate consequence is that you end up with close to 100% test coverage, simply because it's structurally impossible to write untested production code under these rules - there's no code without a preceding failing test to justify it.

Test code needs to stay clean too

It's tempting to hold test code to a lower bar than production code - after all, it doesn't ship, and it doesn't run in front of users. That reasoning quietly destroys the value of having tests at all. If tests are sloppy, duplicated, and hard to follow, they get harder to maintain than the production code they're supposed to be checking. Once a test suite gets expensive enough to touch, developers start skipping updates to it when they change the production code, and once a few tests start failing "for reasons unrelated to my change," people begin ignoring red test runs altogether. From there it degrades fast: a red suite that nobody trusts is functionally the same as no suite, except it also costs CI minutes.

The standard for test code isn't "the same amount of polish as production code" - it can be less elaborate, since it doesn't need to handle every edge case a production API does. But it does need to be readable at a glance, because the whole point of a test is that a future reader (possibly you, in six months) can look at it and understand what behavior it's pinning down, without archaeology.

Tests are what let you change things

The core argument for keeping tests around at all: they're what makes it safe to change code later. Without an automated suite, every change to a nontrivial codebase risks silently breaking something far away from where you're working, so people become afraid to change anything, and the design calcifies. With a trustworthy suite, you can restructure, rename, and refactor as aggressively as the design demands, because the tests will tell you within seconds if you broke something. Clean production code is what makes a system pleasant to read; clean, trustworthy tests are what make it safe to keep improving. Lose the tests and you lose the courage to change the code, no matter how good the code looks today.

One concept per test

A common early rule of thumb is "one assert per test method." Taken as an absolute law it's overly strict - sometimes a single logical concept genuinely requires checking two or three related values that all describe one behavior. The more useful version of the rule is: test one concept per test. If a test's name can't describe what broke in a single clear sentence, it's probably checking multiple unrelated things, and a failure there won't tell you much beyond "something is wrong somewhere in this list of asserts."

def test_page_content():

page = wiki_page.make_page_with_content("test", "some content")

request = wiki_page.make_request(resource="test")

response = wiki_page.make_response(request)

 

assert "test" in response.title

assert response.status == 200

assert "some content" in response.body

assert response.body.count("<div") == response.body.count("</div")

 

// Four different concepts are being checked in one test.

// A failure here tells you *something* broke, not *what*.

def test_page_title_is_rendered():

page = wiki_page.make_page_with_content("test", "some content")

response = wiki_page.render(page)

assert "test" in response.title

 

def test_page_returns_ok_status():

page = wiki_page.make_page_with_content("test", "some content")

response = wiki_page.render(page)

assert response.status == 200

 

def test_page_content_is_included():

page = wiki_page.make_page_with_content("test", "some content")

response = wiki_page.render(page)

assert "some content" in response.body

 

// Three tests, three names, three specific failure messages.

// The shared setup can move into a fixture without changing this.

Splitting by concept also means shared setup logic naturally wants to move into a fixture or helper, which is a good sign - it's evidence the tests were about behavior, not incidentally sharing a block of code.

F.I.R.S.T.

A useful acronym for what makes a test suite worth trusting:

• Fast. Tests that take minutes to run get skipped. A team that stops running the suite because it's slow has, in effect, stopped having a suite.
• Independent. One test's outcome should never depend on another test having run first, or on the order tests happen to execute in. Order-dependent tests fail in confusing ways and make it hard to run a single test in isolation while debugging.
• Repeatable. A test should pass or fail the same way in any environment - your laptop, a teammate's laptop, CI, offline on a train. A test that only works when it happens to be connected to a particular network is a test you can't run when you actually need it.
• Self-validating. The test itself produces a boolean pass/fail. If verifying the result means a human has to read a log and decide whether it looks right, it isn't really automated yet.
• Timely. Write the test just before the production code it covers, not weeks after. Code written without a preceding test has a way of turning out hard to test after the fact, which is itself a sign the design could be better.