Thursday, January 10, 2008

An Utter Disregard For Freeloaders

Emboldened by Zed's fine example, Dreamhost is letting off some steam:

the solution from the Rails community itself was quite honestly, stupid. They said to stop using Apache and FastCGI (a combination that successfully powers bazillions of websites, just not Ruby on Rails ones) and instead to entirely retool our servers with Lighttpd and SCGI (a FastCGI like protocol that may be technically superior, but next to nobody uses). That suggestion shows either a complete lack of understanding of how web hosting works, or an utter disregard for the real world. It’s all good and fine to recommend that users use higher end dedicated server hosting for their commercial applications but you simply cannot ignore the fact that nearly everyone will want to use lower cost shared hosting for getting started. It’s just simple economics. Additionally, people who use systems like Ruby on Rails want to spend time programming and not time setting up servers. Recommending technologies that are not widely used or supported by any major web hosting company is putting too much of a burden on your users, the people you want to keep happy!

An interesting, somewhat weird thing about the Rails community is that everybody knows Rails is a free open-source project created because somebody wanted to have a really cool Web framework, yet people complain that Rails is hard to use for some reason or another. Rails is not a sofa. It is not designed for your comfort. It is designed for usefulness to the people who develop it. The basic idea is that if you want to make it better, go for it. Complaining is simply disregarded as noise. There are code contributions and there is noise. That's basically how it works.

Since anyone can join the ranks of the people who develop Rails the framework, the fact that Rails is designed for the people who develop it is not a bug, but a feature.

It's not a Bug, it's a feature.

Everything I just quoted Dreamhost saying came from a paragraph which started off like this:

These issues were later mostly mitigated by a single user of Ruby on Rails who wanted it to work better on our servers

That's how it's supposed to be. Open source works in the same way markets do: supply and demand. If the demand exists for a patch, it will come into being, guaranteed. That's why complaints are just noise. If they were worth paying attention to, they would be expressed as patches.

I'm not saying the Rails core team is perfect. I'm saying the whole idea that Rails ought to be easier or different or whatever is only true if you make it true by (to use Ghandi's phrase) being the change you want to see in the world. If you think the world needs somebody to fix Rails, then be that somebody.

I've come across this idea before:

And responded before as well.

DHH responded too, and said almost exactly the same thing:

Most Rails contributors are not big users of shared hosting and they tend to work on problems or enhancements that'll benefit their own usage of the framework. You don't have to have a degree in formal logic to deduce that work to improve life on shared hosting is not exactly a top priority for these people, myself included.


The Dreamhost guys in particular sounds like they're experiencing a lot of hurt running Rails in their shared hosting environments. That should be a great motivator to jump in and help improve things.


they have a substantial economic interest in making the shared hosting scenario for Rails easier.


That's both a personal motive for having a less stressful day and a profit motive for making your business more money. Sounds like a match made in heaven for someone like Dreamhost to get involved and help do the work to make Rails a great shared host experience.

Although I think he's a brilliant programmer, I have to say DHH might not be the greatest diplomat, because he followed this excellent common-sense reasoning with a request for Dreamhost to shut up and quit whining:

Don't treat the current Rails community as your unpaid vendor. Wipe the wah-wah tears off your chin and retract the threats of imminent calamity if we don't drop everything we're doing to pursue your needs.

But then again, given the title of this post, for me to call DHH a lousy diplomat is at best the pot calling the kettle black, and at worst the black object calling the grey object black. That's not the point, anyway. I bring in his post because I initially wrote this a few days ago, before his response, and the response I predicted is pretty much the response the world got. It's probably not worth yammering on about, except for one important piece:

Open source projects accurately reflect supply and demand. This is why they clobber commercial software so frequently.

This whole Rails attitude of "I'm only going to work on something if it makes a difference to me personally, so don't expect me to do work for you unpaid to make your life easier, and don't expect me to care if you complain but don't contribute code" is actually a perfectly reasonable attitude to drive an open-source software project with. The self-interest which motivates code contributions is, in a way, a nice, dependable metric for determining demand for a feature. If enough demand for the feature exists, the feature comes into existence. There's a tautological nature to it that makes it very clean and neat. Supply and demand are so closely linked as to become almost indistinguishable.

The huge irony of commercial software development is that commercial software development by its nature cannot capture the market efficiencies that make capitalism so valuable. In software development, it's only free, open source projects that get free-market effects. That might seem like a contradiction in terms, but only if you don't really understand capitalism. Capitalism is effective not because greed is inherently good; capitalism is effective because market efficiencies can only exist where freely-operating agents act for their own benefit. Greed is just one mode of leverage which can create that kind of situation. Capitalist economies are systems where freely-operating agents act for their own benefit, so they get free-market efficiencies. But open-source software projects are also systems where freely-operating agents act for their own benefit, so they get free-market efficiencies too.

It's not really about the market; it's about the efficiency. The capitalist free market uses greed to achieve that efficiency; open-source software projects leverage convenience. In either case, you get a system with the required freely-operating independent self-interested agents, so you also get the inevitable concomitant efficiency. (Technically even the process of evolution is another example of this same fundamental dynamic.)

The nature of commercial software development is such that nobody really writes commercial software for their own benefit. They write it for some hypothetical customer. The customer's hypothetical nature means any and all features match their needs. Since the customer is hypothetical, their needs are hypothetical too. This is a big reason so many pieces of commercial software are chock-full of garbage features that could clearly only ever have originated with some soulless human gerbil who wanted to prove that when they came to work they were in fact awake throughout the entire day. Or at least most of it.

(Yes, Adobe Photoshop developers, I saw what you did there. You can fool some of the people some of the time, and apparently you can fool all of Adobe's project managers all of the time, but if I ever find you, I'm gonna kick your ass.)

I too have been emboldened by Zed's fine example. Everybody's getting punk rock on the Web in 2008.

(Probably because nobody ever has to put their money where their mouth is. Yay the Web. Now everybody can be a tough guy.)

Anyway, if you really want to understand what I'm talking about, you need to get this book:

Linux is clearly a decentralized system, since it has no formal organization and its contributors come from all over the world. What decentralization offers Linux is diversity. In the traditional corporate model, top management hires the best employees it can, pays them to work full-time, generally gives them some direction about what problems to work on, and hopes for the best. That is not a bad model. It has the great virtue of making it easy to mobilize people to work on a particular problem, and it also allows companies to get very good at doing the things they know how to do. But it also necessarily limits the number of possible solutions that a corporation can come up with, both because of mathematical reality (a company has only so many workers, and they have only so much time) and because of the reality of organizational and beauraucratic politics. Linux, practically speaking, doesn't worry much about either. Surprisingly, there seems to be a huge supply of programmers willing to contribute their efforts to make the system better. That guarantees the field of possible solutions will be immense. There's enough variety among programmers, and there are enough programmers, that no matter what the bug is, someone is going to come up with a fix for it. And there's enough diversity that someone will recognize bugs when they appear. In the words of open-source guru Eric Raymond, "Given enough eyeballs, all bugs are shallow."

In the way it operates, in fact, Linux is not all that different from a market, as we saw in Chapter 2 on diversity. Like a bee colony, it sends out lots of foragers and assumes that one of them will find the best route to the flower fields. This is, without a doubt, less efficient than simply trying to define the best route to the field or even picking the smartest forager and letting him go. After all, if hundreds or thousands of programmers are spending their time trying to come up with a solution that only a few of them are going to find, that's many hours wasted that could be spent on something else. And yet, just as the free market's ability to generate lots of alternatives and then winnow them down is central to its continued growth, Linux's seeming wastefulness is a kind of strength (a strength that for-profit companies cannot, fortunately or unfortunately, rely on). You can let a thousand flowers bloom and then pick the one that smells the sweetest.

The long and the short of it is that free-market efficiencies aren't really about capitalism. Capitalist economies are just one way to harness the fundamental dynamic. The fundamental dynamic is essentially that good decision-making emerges naturally from any context where large groups of people operate freely, acting for their own individual benefit. Capitalist economies are one such context, free open-source software projects are another. Science, as Surowiecki explains in the book, is another. Every scientist who puts forward a successful new idea or who finds a flaw in an existing idea or who validates an existing idea sees their career improve as a result of the accomplishment. So science is an arena in which large groups of people act independently for their own individual benefit, and blam, we get electricity, TiVo, spaceships, and sliced bread. Or whatever it is science has given the world. I don't know off the top of my head but I'm sure there's something.

LSD. Science has given the world LSD. Good call, Science. Now give us sex robots, world peace, and cholesterol-free hot dogs. Daylight's burning, Science. Look alive.

ANYWAYZ...I forgot my point. But dammit, I had one, and it was worth making, probably.

Oh yeah. People who complain about Rails are silly. You must be the change you wish to see. You must invent the cholesterol-free hot dogs you wish to eat.

The general superiority of open-source software to commercial software is intrinsic and inevitable. Capitalism is not the defining example of free-market efficiencies, and harnessing free-market efficiencies by giving things away for free makes a lot more sense than you might think, primarily because "free-market efficiencies" is a misnomer. It's really "efficiencies due to the free operation of independent agents acting for their own benefit." (You can see why the less accurate phrase is nonetheless the more popular one.) Capitalism isn't the definitive example for situations where good decision-making operates as an emergent phenomenon from the interaction of independent agents; it's just one example of a broader dynamic which can be harnessed economically but which is not actually intrinsically economic in nature. Thus commercial software development serves a capitalist purpose but operates in a manner very unlike capitalism, while open-source software development operates in a manner very similar to capitalist free markets but has very different outcomes and goals. ("Market efficiencies" are not inherently market-oriented.) Where commercial software development is not inherently powered by people who are being the change they wish to see, open-source software development is. Finally, Science has given the world LSD, but not sex robots, yet.

I'm sure they're working on it.