Sunday, April 20, 2008

Archaeopteryx: Logging (And Soon Unit Tests)

In developing my music system Archaeopteryx, I haven't written many tests, and the few I did write, I threw away when the implementation changed, because changes in the implementation made the tests completely out of date. Unreliable tests are worse than no tests at all, so I stopped writing tests altogether, and I used the excuse that I was exploring this very mysterious problem space where I hadn't written any code before. (I'm not the only person to argue in favor of the explorer exemption.)

The first problem with that is I have written code for this problem space before. In fact, if you count writing MIDI by hand, I've done it many, many times. I did it when I was 15, with a Korg M1, and later in my 20s with an MPC-2000. I've done it so many times that writing music with a GUI feels weird to me. Writing raw MIDI notes and events within a hardware sequencer differs in many ways from writing a software meta-sequencer, but it still means familiarity with the fundamentals of the problem space, which makes the explorer exemption harder to justify. And this isn't even the first time I've written Ruby to generate MIDI, so this explorer exemption thing is really kind of a lie.

The second problem with that is that the explorer exemption has to come with an expiration date anyway; once you've explored a little while, by definition, the territory becomes more familiar. But the third and most troublesome problem with that is writing code without tests means writing legacy code. This means you have to test, or you're just screwing yourself.

However, I knew that already. The reason I had stopped testing wasn't because I wasn't enjoying that tasty testing Kool-Aid. I'm digging on the Kool-Aid. In fact I've been using various music generators as pseudo-tests, running them repeatedly after every change to be sure my code still works, just because I'm so hooked on that damn Kool-Aid.

The reason I stopped testing is because I couldn't figure out how the hell you unit test a system whose primary purpose is to control a third-party, closed source, proprietary application. I did some asking around and some googling and the recommendations I got mostly involved mocking the ever-lovin' beejezus out of my system. But that didn't appeal to me at all. So I held off entirely, until I figured out a solution.

If you've seen my presentation on code generation, or read the blog post where I illustrated one example of the technique, you may already know what my solution was. Last night I did a tiny bit of refactoring and threw some dead-dumb simple logging via puts into Archaeopteryx.

The end result is that Archaeopteryx now generates playable Archaeopteryx code as its log files. This code isn't in svn yet, and I still haven't had time to put the project on GitHub at all, but all these things will happen soonish. I've actually got to add a little bit to it before it really does what I need. Hopefully it's pretty obvious that the remaining changes are absolutely trivial. First, the only way you can capture this as a log file is with a Unix pipe. Second, logging to the command line by default is lame.

Obviously, I can fix this with a Logger right out of Ruby's standard library.

I hope it's also obvious why this is useful. Archaeopteryx doesn't play back sequences; it generates them. The user doesn't define the music to play, but rather a process for deciding which music to play. Arkx then generates the music accordingly. In practice this is all a combination of probability matrices and a random number generator. San Francisco's awesome underground techno DJ Andy W, who's done a ton of this, told me the most important thing is to always be recording the music your system generates, since every once in a while it'll generate something awesome, not even necessarily due to your code, but just to the sheer fact that if you throw a random number generator at a beatbox, once in a while you'll find some random number generator funk.

But automatically recording everything is kind of a pain - it can eat up your hard drive fast, and you sacrifice a shit-ton of hard drive space for those few moments of serendipity. At the same time, there is lurking off in the land of vaporware another, related problem: what if you use Archaeopteryx for live performance? Trekking out to your gigs with a ton of wires, cables, power adapters, and hardware is a pain in the ass, and I know from personal experience that I'll go to extraordinary lengths to minimize complexity, especially in loud places with bright lights, dark corners, cool music, and hot women. Adding another hard drive to record every performance? Just in case? That would never happen in a million years - even if there weren't such an obvious risk of the extra hard drive being stolen. I hate to bring this up in the context of the dancefloor paradise, but thieves are everywhere.

However, the extra hard drive isn't actually necessary. If you're working within the confines of programmatic MIDI, logically then recording every performance can mean nothing more than capturing a log file. Currently Arkx only generates brief snippets, and recording audio to a hard drive would work perfectly well in that context. Scale Arkx to a full-length DJ set, however, and recording audio doesn't scale, while capturing a log file scales effortlessly. It's not even about adding a layer of abstraction; it's about leveraging the layers of abstraction your system already contains. The whole point of Archaeopteryx is to play music; playing the same music twice should be a nonissue.

I'm kind of belaboring my point here, but just to wrap up, let me say that this is actually not just a pleasant side effect - it's actually one of the reasons why algorithmic composition matters. In an MP3 world, recordings are cheap. Therefore, the whole fundamental business model of selling recordings is completely doomed. A business model based on expensive recordings sells recordings; a business model based on cheap and abundant recordings gives recordings away. This makes the body of recordings of your live musical performances equivalent to a blog - you should be updating it all the time. Recording your live performances should be as cheap as capturing a log file, since distributing those recordings is as cheap as publishing an RSS feed.

Obviously, also, the same strings which my Logger returns can be repurposed in a different way for testing. Ultimately all I need to test is that this system generates the correct MIDI. If I can represent the MIDI as text, then mocking the system in all its complexity is a total waste of energy. My specs are going to consist entirely of string comparisons. Running them will be fast, writing them will be fast, and life will be good.