Tuesday, January 6, 2009

Mystery Of The Demon RSpec

In another age, near the dawn of time, monsters walked the earth. The Elder Gods live on, and they hunger for blood. When they can't devour your vital organs, haunt your dreams with eldritch visions, or enslave you with hallucinations, they settle for annoying the hell out of you with haunted testing frameworks.

My supernatural visitation began on a spooky day. I live in Los Angeles, but the cold was mysteriously intense. It was below 50º F. I knew something was wrong, but I pretended everything was normal, and I ran my specs.

Since I like to run specs as fast as possible, I first ran the file:

spec spec/models/foobar_spec.rb

Everything passed; next step, run all model specs.

rake spec:models

This failed. The failing spec: foobar_spec.rb. The same specs passed when just its file ran, but failed when run via rake.

It was some serious WTF.

Not only that, one additional spec would throw a random ghost error. The error came and went; you'd see it maybe one time out of ten. The rate of WTFs per minute was escalating. The code was acting crazier than a bored octopus.

The cause turned out to be a sneaky little cattr_accessor. I've always suspected cattr_accessor was a lapdog of Satan, and now I have proof. Some other spec reset the cattr_accessor for its own naughty purposes, and this caused my spec to break. The disappearing/re-appearing magical ghost error came from a similar source, and its unpredictable presence and absence came from the random order RSpec examples run in.

Having figured that out, the ghost was easy to bust. Just reset the cattr_accessor to have the correct value, or clear out the spec which modifies it. You could even make an argument for stubbing cattr_accessor, but I think it would be a weak argument.

In terms of the general WTF?, there are two answers here. The local answer: if you've got cattr_accessors in your code, make sure your specs don't reset them. RSpec will blow out the DB to make sure your data's the same every time, but cattr_accessor allows you to store data in the class, and RSpec doesn't reload the classes every time to prevent this madness from getting out of hand. The global answer, of course, is to bug the RSpec team about this. I'd like to see RSpec auto-reload the classes, but I don't have a clue where in its massive codebase to start, and I don't know if they'll be down.

There's a third answer as well. Don't fear the reaper.