Thursday, July 30, 2009

Tuesday, July 28, 2009

Our Old Reputation Was Better

Monday, July 27, 2009

Newspapers Are Dying, Just Like Rock N Roll

After Public Enemy's 1987 Nation Of Millions album proved samplers could rock harder than guitars, a lot of kids in England sold their guitars and bought samplers. In 1988, house music arrived in England, and a whole ton of kids had these samplers just sitting there ready to rock. Dance music and sample-heavy mashup bands (like today's Girl Talk) took over the British music press, and you couldn't open a Melody Maker or an NME without reading that rock music was dead forever.



Sample music spent a lot of time stealing riffs and soundbites from rock, and rock changed as a result, but it didn't change that much.

Blogs are going to kill mainstream news the same way that samplers killed guitars. Turn on CNN, you see bloggers where you used to see newspaper reporters. What else will change? Nothing. Blogs repost mainstream media news the same way dance music remixes every new hit track. If you're a band, your five minutes of fame will be retweeted on the dance floor live and in real-time by the best DJs on the blogosphere.

Ecosystems change, but kings remain kings.

Available For Consulting

If you need Ruby and/or Rails work, give me a holler. JavaScript welcome.

I also boast remarkable skill in boasting, as well as blogging, arguing with strangers on Reddit, drawing attention, and giving presentations. Last but not least I have some underdeveloped skill in art and design which I would love to expand on.

Sunday, July 26, 2009

Simplified 2D Particle Systems In Ruby & SVG

Last year, I read Chapter 1 of Topher Cyll's Practical Ruby Projects and used it to build an improvising AI drum machine. The other day, I read Chapter 2.



Chapter 2 explains how to create animated graphics in Ruby, using ERB to generate SVG. A lot of the code in this chapter deals with animating text in isographic pseudo-3D cubes. That didn't interest me, so I cut to the chase with a much smaller code base just animating a circle moving across the screen.



Once this worked, I knew I could get started doing some more sophisticated stuff. Rather than re-use Topher's isometric text animations, I wanted to see if I could re-create a Flash animation I made around 2005-ish.



This animation came almost entirely from a tutorial by Keith Peters, aka bit101. It didn't take long to write some new code which recreated the Flash animation in its basic form. To be honest, however, I skipped a lot of the subtleties that make the Flash version interesting to watch.



SVG defines Photoshop-style filters, so I changed the code a little and re-rendered. Practical Ruby Projects starts you off with convert, the command-line tool for ImageMagick which includes the ability to render SVGs. However, ImageMagick's implementation of the SVG spec is incomplete, and to render filters, I had to switch to Apache Batik.



Practical Ruby Projects also points out that you can use convert to turn the rendered JPEGs into movie files. However, it recommends iMovie instead, because convert takes a long time. I rendered all these animations in Adobe After Effects, the professional standard for motion graphics and compositing, but this highlights a flaw in the whole project: struggling with SVG's quirky spec and a Java renderer just to put Gaussian blur on an image is pretty pointless when you can do so much more in After Effects with so much less effort.

For contrast, here's a version with no SVG effects, only effects added in After Effects.



I'm not sure how useful this code will be, but if you're interested, it's on GitHub. The bit101 tutorial is available on his site, but only as a Word doc, so I've reposted it in PDF format. If it's not obvious where the 2D particle animation comes in, I made the particles invisible and only drew the lines between them, just because I like the look of it better.

Related: in the past I've blogged about similar projects built with JRuby/Processing, Ruby plus OpenGL, and a custom OS X Scheme.

Friday, July 24, 2009

More About "Magic" In Rails

Joe Fiorini e-mailed me:

Overall, I really enjoyed the article. You hit some very important points that I needed to hear, such as discomfort with "magic" exposing programming deficiency. I have no problem using features like eval, mixins, open classes, etc. where appropriate.

However, when you say "magic", I'm not sure you mean the same thing as I and a number of other Rubyists I know mean. From your article, I gathered that by magic you mean the advanced metaprogramming techniques with which most Ruby newbs are unfamiliar. I tend to think of magic in Ruby as important things that are happening "behind the curtain" so to speak.

For example, when working on a Ruby on Rails app, we have a number of ActiveRecord models that include modules. Sometimes, those modules will use the "included" hook to pass has_many, belongs_to, etc. messages back to the class. Given that relationship definitions like that define a large part of the model's interface, I consider this magic and don't like seeing it done (plugins and extensions excluded assuming they are well documented). What is your preference in this instance or similar instances?


I got a similar kind of thing from a Python guy on James Bennet's blog. The answer is: there's no such thing as magic. A superstitious mindset is bad for you as a programmer. It makes no difference what you happen to be superstitious about.

This is a much broader complaint. I talk about the specifics of eval etc. because those are interesting specifics to me. However, the general remains true irrespective of the specifics. I'll give you an example. Consider the phrase "right now." Go into a programming meeting, or talk to a programmer about any system they're working on, and count the number of times that you hear the phrase "right now." Then spend a week or a month where you ban yourself from using that phrase.

There is only right now. Only the moment is real. Ask any Buddhist; ask any quantum physicist. "Right now" is a mental crutch. It separates you from the reality of the system you're building. "Magic" is also a mental crutch. It imposes unnecessary limitations on what you can do.

How I feel about this technique of modifying a model when including a module is how I feel about all the other so-called "magic" in the original blog post. It's not magic. It's a technique. It's a technique with risks and benefits. People fuck it up from time to time because it's a technique they don't have a lot of experience with. As I said before, it's no surprise that most people come to Ruby without the experience of code that modifies code, and that a lot of the confusion that Rubyists experience revolves around places where this type of code does things they don't expect. Any technique, you learn over time where it is and isn't appropriate.

In short I think if you ask this question at all you need to go back and read the original blog post. There are a lot of specific sentences that apply to other things, but the general thrust, I don't even understand how could fail to see that these are the same thing. You've got code modifying code, it surprises you, sometimes unpleasantly, so you put it in a box and you call it "magic." Now we don't have to think about it. Now it's just this scary thing that we can tell our children to be afraid about. That's horrible. Stop doing it.

Or at least, if you're doing it, acknoweldge it for what it is. You're being unscientific. You've chosen superstition over understanding as a guiding principle in how you write code. That's your perogative, but don't expect rational people to respect it. Any programmer who uses the word "magic" should be ashamed. It cuts the discussion short. It's a way of saying "here's something we won't discuss." But a conversation about when to use that particular technique, and when not to, is a conversation worth having.

My own thought is, use it wherever avoiding repetition would justify it. I see people do that in Rails apps all the time, and it's very useful if a lot of your models are similar to each other, especially if it's in ways that have a clear, obvious, necessary relationship to a particular aspect of your site's functionality. However, if that relationship is not clear or obvious to you, you should either take the repetition out, or acknowledge that is at least necessary; and in those circumstances, collecting the common functionality into one file which modifies multiple models the same way could make it clear and obvious to you why those identical or near-identical modifications are necessary.

Clear your mind of fear and say no to superstition. Fear is the mind-killer.

Thursday, July 23, 2009

Blog Comment For James Bennet

(James Bennet's blog software won't let me post a comment this long, so I'm putting it here instead. It's a reply to this post.)

Andrew Ingram: as Philip Brocoum said, the magic incantation you must painstakingly memorize, if you dare to meddle with supernatural forces and summon up the request object from the distant, mystic realms within the innards of Rails, is you type the word "request."

Seriously, how is that magical? How is that hard to remember? I can't imagine anybody sits down and goes "oh my God, in order to get at a request, I have to remember to type 'request'. How am I going to remember that? I'd better make some flash cards." Is that you? I don't think it's you. I don't think it's anybody.

Anyway, I hate to say this, but I know nothing about Django and have no interest in Django, aside from a mild curiousity and having heard a few good things. The only interesting thing in that "I hate magic: why Django rocks" article *to me* was the term "magic." I said this on Reddit, debating Django's finer points based on my rant is like seeing a bull charge and asking if the matador's cape should have been a different shade of red. I wish I had something useful to say about Django but I don't, and I don't pretend to.

Reg - you can lead a horse to water but you can't make him drink. Thanks all the same tho.

All the people who want to debate about me, i.e., all you crazy ad hominem types and the guy who thinks I brought a shitstorm on my head or whatever - well, sorry, I realize this is ironic, but I just don't think the topic is that interesting.

Likewise the topic of mysterious and/or surprising behavior in Rails, I don't think it's a very interesting topic to write about. It's a very interesting topic to write *code* about, and I think it's very interesting what's going on in Rails with the heavily Merb-influenced refactoring and differentiating of public vs. private APIs, but if you have opinions about that, it's a lot more useful to either express them in code or complain about them in more detail, so that people who can solve the problems know what the problems are. I admire that effort but it's just not my personality type, there are a bunch of other things I'll be working on before I spend much time on that myself.

All the comments etc. out of the way, James, I think your argument is reasonable but timid. You acknowledge that calling something magic is lame, but you don't call it something else. You downgrade it from magic to "magic." Those quotation marks are not strong enough to undo the damage that this awful, inappropriate variable naming does.

I mean basically if I can paraphrase you're saying that you have to find a balance between clever tricks and drudge code. To some extent, you're agreeing with something I said. At the end of my post, my conclusion is that since a lot of people are new to this whole phenomenon of code which writes code, or code which rewrites itself or modifies itself, there are a lot of people doing it wrong and making various mistakes as part of the ongoing effort to discover that balance. Implicit (but apparently not explicit) is the assumption that you need to go through those mistakes in order to get to that balance. You're going to get through those mistakes quicker if you look at your techniques as techniques then you are if you wave your hands and complain about nonspecific "magic."

For instance, you tell this story about Django moving from code generation to in-memory code generation. If I recall correctly, Rails went through a transition where a lot of model attribute/association methods were handled by method_missing and are now generated. In other words, Rails probably encountered the opposite problem because it went in the opposite direction. Now we're not talking about magic. Now we're talking about how one framework discovered they didn't like how they were using code generation, and preferred an in-memory solution, while the other framework made almost exactly the opposite decision. That's an interesting contrast. If we were to look at the histories of the two projects and the specifics of these design changes, and their contexts, we would probably learn something of value to both communities about the appropriateness of these strategies at different times and in different contexts. The big difference between this and a dialogue about "magic" is that *this* is USEFUL. People babbling about "magic" are not doing anyone any favors. They're just wasting everybody's time, and I think that's very greedy.

However, there is also in my opinion a massive flaw with your argument, which is that so-called "magical" features are not necessarily clever tricks at all. In some cases, they're very egregious, blatant self-indulgence. In other cases, they achieve great terseness. A very terse code base is a wonderful thing. However there is a downside. The Lisp community is a little autistic, and I think part of the reason is that their code gets too terse, because you can modify the language so much that you almost have a different dialect of Lisp for every single program you write. That's a judgement call, just like the judgement call of clever tricks vs. drudge work, but it's a different type of judgement call.

The "clever tricks vs. drudge work" spectrum is a valid spectrum, but it's not the only spectrum. It might not even be the most important spectrum. Zed Shaw just tweeted me that he thinks the practical problem with "magical" code is that it's difficult to obtain a clear, simple stack trace when you're invoking methods that don't map directly to any actual method definition anywhere in your code base. And the thing is, as far as I understand it, people were complaining about this with Lisp before I was even born. But the question becomes, is it even about clever tricks vs. drudge work? Maybe it's just that you can't debug complex, self-modifying code with a debugging tool designed for simple, literal code. Maybe it's just that when you graduate to more sophisticated forms of programming, that little pocket calculator we call the stack trace becomes totally fucking useless and we have to throw it away.

Maybe at that stage we need a debugger. I'm no fan of debuggers, but a lot of Smalltalk people tell me that the Smalltalk debugger is a wonderful thing. They're used to working with power, and they refer to this as their power tool. Maybe they know something. Since they've been working with these kinds of language features for a very long time, and they swear by this tool, maybe tools of this nature are more valuable than I personally ever realized. And certainly asking this question, about when you have chosen to use unusual programming features, is it not reasonable to expect other things in your approach to programming to change as well, this question is just so much more interesting than all these ridiculous people waving their arms and peeing their undies over "magic."

Wednesday, July 22, 2009

Do You Believe In Magic?

A couple months ago, a pro-Django, anti-Rails screed annoyed me, but not for the normal reasons. I have no problem with people writing these screeds. As long as there are people, the majority will always quibble over minutiae and choosing to aim higher will always be unusual.

Most people who use either language don't make full use of its advanced features, and programming simple monkey code in Python is so similar to programming simple monkey code in Ruby that they might as well be the same language, at that level. My Hacker Newspaper mini-site runs on both Python and Ruby, and it doesn't make a shit's worth of difference. But X vs. Y is a reasonable question, for the many people whose ambition fails them and strands them in the swamps of mediocrity. I pity them, but I don't think there's anything wrong with comparing notes about the sizes of mosquitos from swamp to swamp. No problems with any of the usual "Python vs. Ruby" blah blah blah, except for the fact that the whole question is boring and small-minded. Say what you want; choose whichever language you prefer.

What really bothered me was the title:

I Hate Magic: Why Django Rocks!

Specifically, the horrible term "magic."

So I got my rantin hat on, stomped down to the comments section, and I ranted me up a rant:

The concept that particular language features are more "magical" than others is insane bullshit. None of it involves fairy dust or anything particularly complicated; it's all just Ruby. There is no such thing as metaprogramming. It's all just programming.

This is a thing that's been bugging me lately. I have no interest in this thing of Rails vs. Django. Use what makes you happy. But there is absolutely no logic in calling language features "magical." It implies that certain programming techniques are inherently fantastical and strange. That's only the case if you don't understand the language.

If you take on this attitude that some language features are "real" and some language features are "magic" you're not going to make rational choices when it comes to deciding what language features to use. You're going to make superstitious choices, because this division between the "real" and the "magical" is a superstitious world view.

Rationality is a better mental attitude for programming than superstition.


If you want to see my prediction about superstitious programming decisions come true, find a place where there is no logical reason not to use eval() in your code, put eval() in there, and watch motherfuckers freak out like you were throwing elephant dung at the Pope. There are plenty of places where it is intensely dangerous and foolish to use eval(). I'm not disputing that. But it is so easy to find ways to use eval() which are completely immune to any danger of any kind, and yet which still trigger paranoia in superstitious programmers.

The problem goes much deeper than this, however. Ask yourself this question: if you were a Lisp programmer coming to Ruby, would you describe any of Ruby's features as magical? From a Lisp point of view, Python has more magic in it than Ruby. From a Lisp point of view, both languages feature a certain number of invisible walls, blocking you (for no comprehensible reason) from taking simple, direct paths from A to B. Python is more magical than Ruby, from a Lisp point of view, because it features a larger number of invisible walls, and the paths around them are less consistent.

A superstitious programmer might answer that the Lisp point of view is a strange point of view. The implication is that a point of view which considers "normal" various arbitrary restrictions in a programming language's power is the rational point of view. I disagree. It might be the common point of view, or the popular point of view, but it's not the rational point of view. If it were the rational point of view, it wouldn't include a belief in magic.

Like many of the current, new generation of Rubyists, I came to Ruby because of Rails. However, unlike many of my generation, I stayed because Ruby is an acceptable Lisp. Prior to discovering Ruby, my programming fascinations were artificial intelligence and Lisp. If using Ruby as a Lisp were as cumbersome as using Python as a Lisp (or as unruly as using Perl as a Lisp), it's very likely I would have moved on.

One of the great things about Ruby is that you can come to Ruby from Lisp, and be happy, and you can come to Ruby from PHP, and be happy. Ruby is a fantastic melting pot, in this regard. But let's take that metaphor a little further. Here's what's going on if you complain about "magic" in Ruby. Some of us came to Ruby from more sophisticated places and we know that this so-called "magic" is a little on the quaint and limited side, but we don't make a big deal about it. But if you're coming to Ruby from PHP, that's like coming to New York City from Palookaville, Omaha. You're staring at the electric lights and you're wondering if there's some kind of witchcraft involved.

I'm from Chicago. I used to live in San Francisco. San Francisco is a beautiful city, but I always used to think it was silly when people from smaller towns called San Francisco "the big city." Chicago's population is about 40 times San Francisco's population, and it felt tiny to me. Beautiful, no doubt, but tiny. That's how I feel when I hear small-minded programmers calling a few mostly-elegant shortcuts "magic." Because if you think that's magic, you haven't seen shit. That is some Palookaville, Omaha bullshit, and you need to see the fucking world.

Now, again, when people complain about "magic" the reason they do it is because Ruby's mostly-elegant shortcuts are not quite as elegant as they could be, and also because since everything at the beginner level in Ruby is easy to get right the first time, a lot of Ruby beginners assume its "magic" will be also, and codebases suffer as a result - because Ruby's "magic" is only easy to jump right into for Lispers and Smalltalkers who have seen it all before and already learned the necessary skills. For everyone else, there's pain, there's codebases falling apart, there's ridiculous incomprehensible errors. I'm not disputing that. I understand that. I sympathize. I have felt that pain.

However, a rational viewpoint will serve you so much better, as a programmer, than a superstitious one. It's not magic. There is no magic involved. It's just, if you never learnt Lisp, then you never learned to program, and Ruby's power is exposing a deficiency in your skills. That's all. And with so many former PHP people and Java people who never really learnt to program jumping on the Ruby bandwagon, it's no surprise that a lot of code is breaking at that exact point in Ruby where the monkey-code features end and the real-language features begin.

The correct response is not calling arbitrary features "magic". The correct response is to finish learning how to write code. And please believe me, you're not going to be able to master it until you stop calling it magic. Recognize it for what it is. It's programming. That's all.



Will he finish what he begins?

Saturday, July 18, 2009

Miniapp: Online Portfolio

As with many of my monthly miniapps, I'm kind of cheating here. My online portfolio isn't really an app at all; it's just HTML, CSS, graphics, and a tiny bit of JavaScript. However, if there's any lesson I'm learning from this miniapps thing at all, it's that you get a better workout from building something small than you do from imagining something big. (I'm all for imagining big things, but the goal here is the workout.)

Tuesday, July 7, 2009