Specdown, README Driven Development, and the evolution of testing

Testing used to be simple. Derive a class from Test::Unit, write a method, make an assertion, red-green-refactor, and you’re done.

Then Dan North fucked all that up. Suddenly we had to actually talk to our stakeholders. We began drafting acceptance criteria in a domain specific language understandable by the programming-impaired. Our testing API exploded. Gone were the days of assert. Suddenly we had to memorize a plethora of methods in order to write our tests: feature, scenario, describe, context, it, should, include, be, example, shared_examples, include_examples, cover, etc. etc. etc.

In 2010, Tom Preston-Werner wrote a blog post about README Driven Development. He tried to strike a balance between over-specified waterfall techniques and under-specified agile cowboy coding.

I embraced README Driven Development. I started all of my projects with a README. I reaped all kinds of benefits from it:

  • I started out focused on my API, not implementation details
  • I thought about how my potential users might use my project (and how I might convince them to try it)
  • I wrote about my project without any of the constraints of Gherkin or Connextra

I write clear, concise, and comprehensive READMEs. Anyone should be able to read my README and learn everything they need to know to use my project.

But I had a problem. Every step I took after my README took me further and further from that clarity. My cucumber features replaced my concise prose with repetitive “Given/When/Then” scenarios. They hid my API behind step definitions, mitigating against the likelihood that they would ever serve as the primary documentation for my project. My RSpec suffered a similar fate; my narrative was lost behind a sea of syntax that was ironically intended to give tests a more human readable cadence.

Even worse, I was now maintaining documentation about my project in three different places: my README, my cucumber, and my rspec. And my README could all too easily get out of date. Like code comments, it had the propensity to lie.

Then I had an idea: what if my README was executable? I wouldn’t need any Gherkin or step definitions or rspec. And even better, my README wouldn’t lie - at least no more than any other test.

So I created specdown. It makes your READMEs executable, letting you expose your API while giving you a way to hide your assertions behind plain english (or any other language). Check out the README and hit me up on twitter or github if you want to discuss or have any issues.

Saturday, February 4, 2012   ()