Wednesday, April 16, 2008

Fight Corporate Obfuscation With Programmatic Analysis

Like many corporate projects, my current project includes some enterprise integration - the bane of the Rails world, not because of any inherent difficulty, but because of a cultural mismatch. There's a large, expensive product which some people are implementing, and they publish relevant details of their implementation in an Excel spreadsheet.

I just had an interesting conversation with one of these implementers, and although he's usually a nice guy, I didn't enjoy the conversation. It included some extremely bad advice, and a round of questioning about my power in the corporate structure, and a number of other questions on matters so irrelevant that I not only didn't know the answer, I wouldn't be able to remember the answers if anyone told me.

The extremely bad advice concerned what to do with the data he'd given me. He essentially recommended creating some kind of rules system based on the Excel file. The Excel file contained a very large number of columns and rows governing a very large number of variables, all named in classic enterprise application style - that is to say, all of them impossible to remember and utterly devoid of any semantic significance.

Fortunately, although the conversation did irritate me, it didn't do any harm, because I had already written a 39-line Ruby script to analyze the data this Excel file described the schema for. Although the Excel file dictated setting a large number of different variables, in practice, this script showed me that only five of these variables ever actually changed in value, which means that the whole thing can be handled with an ERB template and some variable interpolation.

Pretty much every beauraucrat out there seeks power over other people by making it difficult for those other people to accomplish their goals without the beauraucrat. It's very rare that these intentions are conscious; it's got to be an unconscious thing, I'm sure, because anybody who approached life that way on purpose would be pretty messed up. But unconsciously or consciously, it's what they do. In governments they can threaten you with jail time or confiscating property if you don't follow their rules; in corporations, unless they're actually managers, all they can do is give you baffling Excel spreadsheets and bad advice, confident in the fact that nobody will ever be able to figure out the spreadsheet, so everybody will have to follow the advice, no matter how bad it might be.

These time-wasting, inappropriate power trips are the reason why it's really useful to write code which can analyze code - and the truth of the matter is, it's absurdly easy. All you have to do is throw regular expressions at strings and store results in hashes. I learned this with Perl but it's just as easy with Ruby.

More generally, this illustrates a fundamental rule. Never trust business users to analyze their problem space correctly, whether they irritate you or not. If you build a system to their inaccurate understanding of the data, they'll blame you. (Conversely, if your code exposes flaws in their business process, they'll tell you "we can live with the way you handled that.") Their job is to give you the data and tell you the results they want. Your job is to give them the results - and since interpreting the data happens after they give you the data and before you give them the results, that means figuring out the data is your concern.

(And if you ever find a beauraucrat wasting your time, telling you how to do your job, and asking you rude questions about your role in the corporate power structure, relax, and just remember, you can probably route around their obstructions with only 39 lines of code.)