1. A case against Mocking and Stubbing

    10 December 2008

    I used to be a big advocate of mocking and stubbing back when I was doing a lot of Rspec. To the point that when I interviewed at Thoughtbot I brought that up as my reason for using Rspec over Shoulda. (note: if you’re interviewing for a company, it might be in your best interest to be using their stuff and be familiar with it) However, since I’ve been TATFT with TDD and some BDD I’ve come to believe that mocking/stubbing is a horrible idea and it can hurt the development process.

    A Case for Mocking and Stubbing

    So, when I hear people defend mocking and stubbing (I’ll say m&s to save typing) it is for one of two reasons:

    1. The controller tests should only be testing the controller. The model tests should only be testing the model. Your controller code should not touch the model so you must m&s out your models to keep things separate.
    2. Hitting the database is slow. M&S speeds up the testing process allowing me to focus on writing Red/Green/Refactor rather than wait for the test suite to finish.

    Let’s look at the first reason…

    This is how I develop.
    1. I write user stories, write the steps watch them fail.
    2. I write the functional tests, watch them fail
    3. I write the unit tests, watch them fail
    4. I write the models, unit tests pass
    5. I write the controllers, functional tests pass
    6. If I have done everything right then my stories should now pass
    7. I can go back and refactor or continue by writing a new set of stories

    So, if I am m&sing my way through my functional tests it defeats Red/Green/Refactor. My functional tests should rely upon the models being written properly. There are dependencies here that cannot be ignored or wished away for the sake of an idea testing environment. (separate but equal) What if I change something on my model down the line, my unit test will fail, I fix that. But my functional tests still pass because I m&sed that functionality. In the real world, the app will break.

    As a developer I need my tests to fail if I change something. I need to be told exactly where the tests are failing so I can correct this. You do not get this when you m&s.

    I’m sure many people will disagree and I am very interested in hearing the counter-argument or even an argument for why (if) I’m completely wrong.

    So, for the second point… speed.

    This is somewhat of a valid argument. Having fast tests is necessary for effective TDD. You need immediate feedback on what works, what doesn’t. However, the price you pay for that speed is not worth it IMO. So, what would be the alternative?

    What the TDD community needs is an in-memory ANSI compliant database that can be used solely in the testing environment. This way, nothing gets written/read to disk. The current practice of using MySQL or Postgres or SQLite3 for your testing environment is nuts. These databases are meant to handle millions of records, organize and serve them up. How many records within a given test are you using? For most of my tests I usually have no more than 3 or 4.

    Another benefit is a much faster transactional rollback after every test. Just delete the memory, RAM is fast. Disk IO is not.

    There is NullDB, but it is quite limited in that it cannot do ActiveRecord::Base.find. (I’m guessing this will cause a few problems for people)

    SQLite3 can be an in-memory database. Problem solved, right? Not quite. SQLite3 is pretty limited. Most people are probably using MySQL and rely upon many of the SQL functions that are included.

    What would be nice (and well beyond my ability) is to have a Gem that simulated the database you use, only it is in-memory. Optimized for small data sets. No need to go through a heavy hashing algorithm. Keep it light. Keep it fast.