Friday, June 15, 2007

Worry About Scaling Later? No.

37 Signals' Getting Real is a great book, but one piece of its advice seems dead wrong: worry about scaling only if and when you have to. Consider servers that get slashdotted or boingdotted. Consider startups with explosive viral growth. Traffic spikes on the Web can be gigantic, and completely unpredictable. Companies should plan for that.


  1. True, I think Twitter makes a pretty good case for worrying about scaling upfront. Granted they had pretty amazingly explosive traffic growth, but isn't that what many startups are hoping for?

  2. I disagree, and Tritter is the exception, not the rule. The fact is, most apps you build will never attain that kind of success. Every one might like to think they're building the next Tritter/Google/MySpace/FaceBook, but it just isn't so.

    Spend you time and effort building a good app with the plan to build a user base slowly and handle scalability when it actually becomes an issue, in MOST cases it never will, and if it does, well, that's a nice problem to have.

    Premature optimization is the root of all evil!

  3. I used to agree with the Getting Real thing, Ramon, but not any longer. Here's why: social networking apps necessarily depend very heavily for their success on network effects. Being able to seize that moment when your app suddenly blows up is crucial. I think the Getting Real approach is still very valid in that your primary goal should be to build something solid and good, but if something like that does happen, you want to be able to take advantage of it. Having that fluke success as your primary business goal is still a flimsy plan, but disregarding its significance would be a mistake. Katerina Fake said at one of Adaptive Path's UX conferences that timing was the biggest part of Flickr's success. You shouldn't be building to cash out on a lottery ticket, but if somebody drops one in your lap, you should be able to redeem it.

  4. I beleive that absolutes don't work. I mean, yeah, scaling *could* be a problem, but, as ramon pointed out, 9 out of 10 times it won't.

    Now, I beleive that if you do things right, you really don't need to worry (much) about massive scaling, as that usually just is throwing more servers.

    If you take a little time to work with that in mind, then when you have a problem you just throw more servers in, and things will work fine for a while, until you need to, say, add more database servers (thinking on twitter, or all rails apps for that matter), but once you are (near) there, you know it might happen, so that's a good time to take a bit more time.

    But just develop thinking "this will be huge, so I just should replicate this data here and there and complicate things because that will make me churn out 12903812903 requests per second without a problem" is plain out wrong. If you need to change direction on the project a bit, then that will only make things worse...

  5. "Being able to seize that moment when your app suddenly blows up is crucial."

    Based on what evidence? People still use Twitter even though it had scaling issues. Myspace has awful scaling issues, but is bigger than ever.

  6. Based on my theory that Twitter and Myspace probably both lost sleep scaling in a hurry. My theory could be false but I think the idea of worrying about scaling later is a bit facile. Scaling shouldn't be the focus, it shouldn't bog down your process, and it shouldn't be your expectation - all true. However, you don't expect to run anybody over on your way to work, but you do insure your car.

  7. Here's the metaphor: natural disasters. I usually live in either Silicon Valley or New Mexico. (My new love is Los Angeles, and I'll be back there some point soon.) California is prone to earthquakes and wildfires, and New Mexico is prone to wildfires and tourists. Do you cower in fear because an earthquake could happen? No; Los Angeles has its fair share of skyscrapers. Likewise people in NM build near forests even though they know wildfires can happen. You don't plan your whole strategy around the outside chance of a freak occurence.

    However, when a freak occurence is a known risk, you plan for it. Nobody builds a home in a New Mexico forest without accomodating the risk of fire; likewise skyscrapers in Los Angeles have different structural engineering requirements than skyscrapers in Chicago. Worrying about the earthquake when it hits is what animals do. Human beings plan ahead.


Note: Only a member of this blog may post a comment.