Thursday, December 30, 2010

Universal Translators Are Totally Doable

SingularityHub points to research efforts, but it's much simpler. Here's all you need to do:
  • iOS app streams audio to outsourcers with relevant language skills
  • outsourcers transcribe text into BabelFish and paste resulting translation back into web app
  • web app pipes text into cheap text-to-speech software and pipes audio back to iOS app
  • boom, Star Trek, 400 years ahead of schedule

Google Has Its Head Up Its Ass

If you make a fortune by creating a spectacular innovation, which goes on to become the de facto worldwide standard, should you focus on creating new spectacular innovations? Or should you attempt to force yourself into every market as the new worldwide standard in everything?

For Google, nearly all of whose profits depend on advertising revenue, dominance expressed as clickstream traffic is the currency. To maintain that dominance the “Don’t Be Evil” company has been willing to go into business in China despite all evidence of rampant human rights violations, get into bed with the worst phone carrier to rape net neutrality, let its “walled backlot” search become a cesspool of SEO swindlers, collect unauthorized data via illegal WiFi mapping all over the globe, risk exposing private email account data in hopes of capturing social graph info by default, favor its own properties in search results in surreptitious ways and so on.

Saturday, December 25, 2010

Automated Refactoring: This Spec Passes

The code needs a lot of improvement and expansion, but I'm stoked.

Update: just to be clear, this spec passes too:

Kangaroo & Kangaroo

My parents uncovered one of my books from when I was a kid, which revolves around a poem concerning hoarder kangaroos. It's out of print and exceedingly difficult to find, so I'm posting the poem.

Kangaroo and Kangaroo
Just like me and just like you
Except that they were kangaroo
And hadn't very much to do
But spend their lives collecting stuff
Afraid they would not have enough.
For everything they found a space
Because they figured --"just in case...".

"Just in case we need a bit, we'd better save a lot of it.
Just in case the martians come
We'll have the stuff and give them some.
Just in case, well just in case..."
And so they filled up every space.

The house was filled from floor to floor
With brics and bracs and then some more.
Every room was overstuffed,
Every pillow overpuffed.
The closet doors could hardly hide
The piles of stuff piled up inside.
Indeed so full were drawers and shelves
They almost couldn't fit themselves!

The first floor had a rocking chair
A darning egg shaped like a pear,
An anchor from a fishing boat
A 1920's old fur coat.
A portrait of their old aunt Lil
And old aunt Lil's old coffee mill,
A fishing rod, a fishing reel,
A green contraption with a wheel,

A section that was full of crates
And single shoes that had no mates.
Of fountain pens that had no ink
And yes, there was a kitchen sink
In which a bike took up the space,
And left no room to wash one's face.

The second floor was just the same,
And had more things than you could name.
Among them, though, to name just some,
A basketful of bubble gum
Potted plants and pots and pans
Corners crammed with coffee cans
Eight keys, a goat and seven limes,
And one whole room of Sunday times.

The attic too was very full
Of skeins of golden knitting wool,
With gramophones and green guitars
And pictures showing baseball stars.
And on a trunk of tools was set
A plaster bust of Lafayette.
Each day they filled another space,
To satisfy the "just in case".

There wasn't room for one thing more
Except a spot behind the door,
And there, with hardly room to sneeze
The kangaroos themselves did squeeze.
Said one "you know it's time to eat
Lets go and get ourselves a treat,
I'd love a piece of chocolate fudge
But neither kangaroo could budge!

For days they stood behind the door
And as their stomachs rumbled more
They dwelt upon their sad distress
And tried to figure out the mess.
"I've got it!" said one finally
We'll give it all away for free!"


"For free?" Oh dear oh dear
I can't be hearing what I hear
Give it all away? You're mad!
Things could never be that bad!"
"Oh yes they could, what's more,
They're worse! This stuff of ours is like a curse.
Bur while our things do us no good
There must be some for whom they would --
Now let's get going one two three
And start to give it out for free".

By now of course the two were thin,
So, pulling both their bellies in,
They squeezed their way along the floor
And somehow schwuzled out the door.
Then quickly made a sign with zeal
That said:

It's Free, It's Just For You
Signed, Kangaroo and Kangaroo

And soon a crowd had gathered there
But all the people did was stare.
The kangaroos were at a lack
To know why folks were holding back.
And finally from out the crowd
A little boy cried out aloud.
"Free? For free? It couldn't be
I can't be seeing what I see".

"Of course you're seeing what you see!
Anything you want is free,
And incidentally, little boy,
You might like a special toy
That's in the box that's on the chair
That's near the desk beneath the stair"
The boy replied "but what's the catch?
You're sure there are no strings attached?"
"Why sure there's string, third room in back,
It's in an orange leather sack."

And finally, convinced at last,
The people started moving fast
They cleared out every bit of stuff
Except the house (enough's enough).
"Thank you for the soldier's hat
I've always wanted one like that"
"Thank you for the model train
I left my old one in the rain"
"Thank you for the potted plant
It's just the thing to give my aunt".

And last of all, the little boy
Said, holding tightly to his toy
"A boatmobile! How could you tell!
Boy, you fellas sure are swell!"

The kangaroos felt wonderful
With house quite bare and hearts quite full.
"You see" said one "who needs full shelves
When all good things come from ourselves?"
"You're right! We needn't save up stuff
And be afraid of not enough .....

But, since we've not kept anything
It wouldn't hurt to save this string....
We might need paper too, some day,
No need to throw it all away.
I can't help feeling 'just in case'
And after all

Monday, December 20, 2010

No New "No New ..." Post (Yet)

2009: a new miniapp every month. Taken literally, the project failed, but with interesting results. Seen as a consistent attempt to establish a habit of creating new things on a regular basis, instead of just daydreaming and thinking about it, it succeeded wildly.

2010: a new minibusiness every month. Taken literally, the project failed, but with interesting results. Seen as a consistent attempt to free up time and mental energy for my own projects and establish launching entrepreneurial ventures as a habit, it succeeded wildly.

2011: ???

(2008: more than one conference presentation per month, on average.)

Sunday, December 19, 2010

Please Tell Me I Got This JavaScript Wrong

I wanted to code a naive implementation of Mustache templates in JavaScript.

If I read the JavaScript regex docs right, this is what I have to do:

mustached = "{{foo}} bar";
object = {foo: "baz"}
function(string, parens) { return string.replace(/\{\{([^\}]+)\}\}/,
// to get the result "baz bar"

In Perl, it'd be:

mustached =~ s/\{\{([^\}]+)\}\}/$hash{$1}/;

In JavaScript, you can't access a variable $1, only a string argument "$1" to replace, or parens, a string argument to a callback function.

I'm enjoying JavaScript for the most part - it's a Lisp with a beautiful object model and horrific syntax - but this is just silly.

(If you don't get the regex, read the Friedl book; it's great.)

Update: I got it wrong! Thank the Flying Spaghetti Monster. Lucas Luitjes clued me in to the better way:

[lucas@nicotine:~]$ node
Type '.help' for options.
node> var mustached = "{{foo}} bar";
node> var object = {foo: "baz"};
node> mustached.replace(/\{\{([^\}]+)\}\}/, function(m,p1) { return object[p1] });
'baz bar'

Also, I turned this into a tiny templater:

And, Adam van de Hoven made with the tweakage.

Internet Delivery Services Are Green

I'm carless in Los Angeles at the moment, so I'm thinking about how to handle that, and it made me realize something: Internet delivery services from to Zappos (with in between) are all better for the environment, in a widely spread out city like Los Angeles, than the pre-internet physical stores model. The reason is hidden transportation costs.

Assume for the sake of argument that environmental damage is a "cost" we're looking to minimize. It's much more expensive, in that sense, to ship every package individually to the consumer than it is to ship all the packages en masse to a store - but only if you're not paying attention to the hidden costs. Consider, for example, the cost of driving to the store, not finding what you wanted, and driving to the other store where you finally find it; then multiply that cost by every single unit sold. For all units not sold, add the cost of shipping them back to the manufacturer. Stores leverage economies of scale, but you can use distribution centers in a more sophisticated hub-and-spoke model to do the same thing, but better; and if every single person in a city gets in their car and drives to the store to buy Item X, that's obviously going to cost you much, much more environmental damage than it would to have a small fleet of delivery vehicles deliver Item X to every home in the city.

It'll also cost you more in lost productivity from the time wasted in traffic, physical and psychological health problems from the stress of the traffic, civic engineering costs to build roads, traffic lights, and other systems to handle the traffic, cops to police it, ambulances to handle the traffic accidents, and last but most certainly not least, injuries and deaths from some percentage of those accidents. Delivery services reduce the number of people driving per Item X, and they make a qualitative shift as well, in that they increase the probability that somebody driving is going to be a trained professional rather than a distracted amateur.

Although these shifts are small, they are likely to be noticeable in aggregate, especially if Internet delivery services continue to replace traditional brick-and-mortar retailers.

Monday, December 13, 2010

Hollywood vs. the Internet, and Inbred Geek-Culture Fantasies vs. Reality

One of the reasons I'm going to build a Twitter client is that if you block somebody, and someone you follow retweets them, Twitter doesn't block the RT. Consequently, I had the misfortune to read this today:

While some variant of this wish is inevitable, the wish itself is unlikely to see fulfillment, and as an actor I'm very glad. Although I appreciate the tech world's libertarianism in many ways, there are very good reasons that Hollywood is a strong union town. There's a time and a place for everything.

Consider one tech giant's foray into Hollywood.

On Tuesday, launched Amazon Studios, what they are calling “a new online business that invites filmmakers and screenwriters around the world to submit full-length movies and scripts to make money, get discovered and get their movie made”. might sound like an interesting idea and a project that aims to support aspiring filmmakers. But it’s not...

you give Amazon an exclusive 18-month option for your project without any pay. Meaning you can’t pitch or sell your idea to anyone else during that 18-month period. If Amazon decides to option it, you could get up to $200,000...

or as the rules also state, “or, if we determine appropriate, no award money at all”.

It gets worse. Because Amazon Studios doesn’t think their users will want to read full screenplays, they want filmmakers to submit “test movies”. What’s a test movie you ask? This is how Amazon describes it:

“An Amazon Studios test movie should be an inexpensive, full-length movie that tells the whole story of the script in a compelling way, has very good acting and sound, but that doesn’t necessarily have polished production values.” So Amazon wants you to produce a feature length film with no budget, but it must have excellent acting, music and sound...

Your other choice is to create a feature length (has to be at least 70 minutes) animatic or storyboard that shows people what your movie would look like. But remember it still needs to have great acting and sound. So basically you have to create a full-length animated film or a no budget live action test movie as your pitch. What a joke.

Oh and if they like your test movie and want to re-make it into a fully funded film, they can take your project to Hollywood and kick you out as the director. They say this on their site.

Going back to the Netflix idea, it's useful to remember that the hated robber barons of the 19th century were initially, to use Dick Cheney's term, "greeted as liberators." When they built the railroads, they transformed the country, opening up new opportunities for travel, adventure, and prosperity; it was only later, when they fully controlled the backbone of interstate commerce and began charging whatever rents they felt like for access to it, that they garnered so much animosity.

A robber baron could dream of nothing so wonderful as owning both the creation and the distribution of movies. That's almost what we have today, with the studios being closely tied to cable channels, and it's the reason so many movies suck. Hollywood doesn't produce endless remakes of 1970s TV shows because nobody in the entire town has any ideas. Los Angeles is overflowing with creativity.

I've seen this car driving around my neighborhood:

Its creator: a prolific artist who does work for the movies from time to time. Another such artist, a set designer, produces truly amazing Halloween scenes on his lawn:

These guys don't want to work on endless sequels to movies that weren't any good in the first place, but he who pays the piper calls the tune, and many of those paying the piper in Hollywood today are media executives who own both the creation and the distribution of entertainment. Endless remakes of boring horseshit is what they want, so endless remakes of boring horseshit is what the entire world gets. This combination does not result in more creative films; it results in fewer creative films.

If there's a way that technology can change Hollywood for the better, it is absolutely not by aggregating control over creation and distribution under one corporate roof. Not only is that a worsening form of change, it isn't really a form of change at all; it's the status quo, repositioned on a new pair of shoulders. If you want to see how the Internet is making Hollywood a better place, look at YouTube, and more specifically, look at teenage actor Lucas Cruikshank, who's parlayed a silly YouTube comedy channel he created into a three-picture deal and a six-figure income (which he had before the three-picture deal, and without Hollywood involvement).

(Yeah, it's ridiculous, but that's kind of the point.)

You'd think Cruikshank was an exception, but he's not: he's a new, emerging norm, insofar as any form of success in entertainment can be called normal. YouTube is making entertainment careers, at multiple levels. Justin Bieber was "discovered" on YouTube, Soulja Boy built his career with YouTube, Felicia Day both established a show for herself in The Guild and augmented her Hollywood career in the process, and the number of YouTube "celebrities" who also work in Hollywood as character actors is extraordinary.

The face at the very top left of the picture is mine

Consider Brandon Hardesty, whose acting career came about as a result of his YouTube re-enactments of scenes from classic movies.

Long story short, in his Netflix idea, Zed's barking up the wrong tree. Consolidating corporate power in the entertainment industry didn't work well before the Internet, isn't working that well in tandem with the Internet, and isn't something to perserve going forward. The win is in platforms which enable entirely independent publishing and broadcasting.

Monday, December 6, 2010

The Great Clay Shirky, Greating It Up All Over The Place

Citizens of a functioning democracy must be able to know what the state is saying and doing in our name, to engage in what Pierre Rosanvallon calls “counter-democracy”, the democracy of citizens distrusting rather than legitimizing the actions of the state. Wikileaks plainly improves those abilities.

On the other hand, human systems can’t stand pure transparency. For negotiation to work, people’s stated positions have to change, but change is seen, almost universally, as weakness. People trying to come to consensus must be able to privately voice opinions they would publicly abjure, and may later abandon. Wikileaks plainly damages those abilities. (If Aaron Bady’s analysis is correct, it is the damage and not the oversight that Wikileaks is designed to create.)

And so we have a tension between two requirements for democratic statecraft, one that can’t be resolved, but can be brought to an acceptable equilibrium. Indeed, like the virtues of equality vs. liberty, or popular will vs. fundamental rights, it has to be brought into such an equilibrium for democratic statecraft not to be wrecked either by too much secrecy or too much transparency.

As Tom Slee puts it, “Your answer to ‘what data should the government make public?’ depends not so much on what you think about data, but what you think about the government.” My personal view is that there is too much secrecy in the current system, and that a corrective towards transparency is a good idea. I don’t, however, believe in pure transparency, and even more importantly, I don’t think that independent actors who are subject to no checks or balances is a good idea in the long haul.

If the long haul were all there was, Wikileaks would be an obviously bad thing. The practical history of politics, however, suggests that the periodic appearance of such unconstrained actors in the short haul is essential to increased democratization, not just of politics but of thought.

We celebrate the printers of 16th century Amsterdam for making it impossible for the Catholic Church to constrain the output of the printing press to Church-approved books, a challenge that helped usher in, among other things, the decentralization of scientific inquiry and the spread of politically seditious writings advocating democracy.

This intellectual and political victory didn’t, however, mean that the printing press was then free of all constraints. Over time, a set of legal limitations around printing rose up, including restrictions on libel, the publication of trade secrets, and sedition. I don’t agree with all of these laws, but they were at least produced by some legal process.

Unlike the United States’ current pursuit of Wikileaks.

Sunday, December 5, 2010

The Pro Bursting-Bubble Argument

Facebook brings in $1.5B but the market values it at $35B. Why? Because historically, the greatest profit you can find in an era of tumult lies in seizing control of territory where you can charge rent. The argument for Facebook's colossal valuation is that the territory they seized will become one where they can charge rent.

This argument assumes that Facebook is like the Yellow Pages, a virtual monopoly (to use Warren Buffett's term) in the making, which will one day set its own prices, and continue to do so until the passage of time takes us all to the very, very tail end of Geoff Moore's brontosaurus.

In 2007 I outlined my reasons for believing differently. tl;dr: Facebook is not the Yellow Pages; Facebook is a nightclub. Like all nightclubs, it will one day close its doors. Like a very few nightclubs, it will never be forgotten.

This points to the likelihood of a bubble bursting.

But I'm not 100% sure I was right when I said that. Like many twentieth-century phenomena, The Yellow Pages appear to be following Moore's technology adoption curve in slow motion, so the real question is not whether Facebook is mortal, but whether Facebook dies before making $35B or after. I don't doubt Facebook will make money off its virtual monopoly. I just don't know how much or how fast - or how much time they have.

I am skeptical, of course. There's no doubt the financial classes caused the housing crisis by gambling with the lives of millions on an everyday basis. Whether they win or lose on any particular round of roulette, however, is difficult to predict, in the same way that the motion of a river is. In a sense, predicting bubbles in an age of rapid change and volatile economics is very much like predicting when bubbles will rise in a river. Math can describe it perfectly but not predict it at all.

But then, if you've been building your business on solid, sensible principles, why would you even care? Always remember, you can't con an honest man.

Saturday, December 4, 2010

Fly, Pupa, Fly! Advance Or Abscond!

I love the hell out of this comic. The writing is amazing, and I say that as a dude who knew a girl in high school who went on to get nominated for the Pulitzer twice. The writing is AMAZING. It is ridiculous and insane and BRILLIANT.

Wednesday, December 1, 2010

Current Projects: Twitter Newspaper & Automated Refactoring

tl;dr: I'm building Twitter Newspaper, a sequel to Hacker Newspaper. It's extremely private beta status at the moment, but you can sign up to hear more when more is ready.

Sign up for beta invite
* indicates required

Powered by MailChimp

Since it appears the Freakonomics guys have conclusively discredited Derek Sivers's claims that publicly stating your goals diminishes their probability of success, I'm no longer of two minds about announcing my projects. So: I'm working on putting together some iOS apps, and I spent most of the holiday building a successor to Towelie, the code repetition and similarity detector I wrote in 2008. The new version is closed source and intended to serve as the foundation for an automated refactoring business - maybe a service, maybe a web app, I'm not sure yet. What I already have is much faster and more effective than Towelie was, and covers one additional language beyond Ruby; the new angles of attack I'm taking are even more interesting. Unfortunately, I can't go into more detail about that yet.

I'm also working on a Twitter client, both to prevent people from reaching me with exasperating tweets, and as a way to replace Hacker Newspaper, since Hacker News got less interesting. Twitter is a much more effective link aggregator than Hacker News, but it's also much noisier. Its "follow" model means the range of available sources is much more finely-tuned to your personal interests than any site like Hacker News could ever be. For instance, in addition to the usual tech industry and open source suspects, I get tweets daily about dance music and the film industry. The follow model also makes Twitter completely immune to sudden population shifts, like the one which appears to have caused HN's most recent dip in interestingness. Much more importantly, link aggregation via Twitter operates within the conditions necessary to leverage the wisdom of crowds - namely, independent agents working without coordination, in competition, to achieve disparate goals.

No link aggregator site in the (by now very old) Slashdot/Digg/Reddit/Hacker News model can harness those conditions. After I explained what I believe to be the economic naivete of Hacker News's design assumptions, Peter Cooper asked me what a better system would look like; I've been thinking about the question off and on ever since, and I became convinced that passive/implicit link aggregation was the answer long before came along with things like The #graffiti Daily. I think Peter's own project follows a similar line of reasoning.

I prefer Twitter to Hacker News because there's something very artificial about attempting to build a community around gamified link aggregation. I also find Hacker News annoying because I'm banned on Hacker News; not only can I not comment on anything, but if you attempt to post something from my blog there, it'll get automatically blocked. I never received any warnings or communications of any kind from HN about this, and I never post my own stuff on any site of that nature; I only know because people tried to post my stuff, failed, and told me about it. I don't know why it happened or how to fix it, and finding out is more trouble than it's worth. Like a gated community, Hacker News is fundamentally suburban, in a very artificial way. Twitter, to paraphrase William Gibson, is like a city; anyone can come and go, so you have a living, vital community with no intentional theme or all-powerful "benevolent" overlord.

However, cities are noisier than gated communities, and that can be a problem. Going back to the wisdom of crowds, if anybody has anything to say about the wisdom of crowds that does not actually stem from having read The Wisdom of Crowds by James Surowiecki, not only do I not want to hear it, but that is exactly the kind of lazy, not-doing-your-homework horseshit which I am building this Twitter client to keep people from wasting my time with. The sad truth is that opinions are like assholes; everybody has them, and most of them are not worthy of detailed investigation. (This is especially true when it comes to the wisdom of crowds, or any concept which comes from a book which many people have heard of and only a few took the time to actually read.) The Twitter filter I'm working on is partly a cure for that problem. You can think of it as a pair of dark sunglasses to rock on your dome while you go about your business in the big city.

In the past I've gotten angry with people for talking to me on Twitter about things like the wisdom of crowds without doing any research first, but to be fair, it's really almost impossible to convey in 140 characters whether I happen to be in the mood for random, pointless banter or serious analysis driven by research. I read a lot; half the time I'm reading doctoral-level research on postmodern story structure and half the time I'm reading a comic book written by a teenager. There's really no way to predict via Twitter which of these two extremes I happen to be in the mood for at any given time, if you happen to be outside of my brain. On the inside-my-brain side of the equation, however, it's trivial to just choose to use an awesome new Twitter client with built-in filtering and aggregation when you're in the mood for a research-driven, grown-up approach, and fall back to the usual inane gabber at other times, as appropriate. Since there does not yet exist an awesome new Twitter client with built-in filtering and aggregation, and since its existence will make the choice of using it an easier choice to make, I'm building it.

I'm hoping to make it a business as well, but I haven't done enough work on it yet to say anything further on that.

I'm also working on some new musical projects. More news about that soon. And if you're wondering what happened to the SEO-loving, direct-marketing Giles Bowkett of 2010, don't worry. He has plans for 2011 too.

Oh yeah, I'm also working on a book.

Incompetent Geek Rap Annoys Me

Soon after discovering a pair of rap videos by geeks, one which annoyed me with its terrible rapping and another one which I kind of liked despite being annoyed by its terrible rapping, a Node.js community drama thread on Reddit alerted me to yet another rap video by a geek which annoys me with its terrible rapping.

I just fucking hate this shit, so I recorded my own rap mp3. Here it is.

You can download it for free, and/or buy it for 99 cents.

And by the way, I claim in the rap to have a million-dollar hustle. It might just be true. The new thing I'm working on has brains like Archaeopteryx but a very simple, money-oriented focus.

New iPad App Includes Highly Original Guarantee

Andrew Burke, creator of Remembary, an iPad diary, is supremely confident his app will never lose your entries - so much so that he guarantees it. In fact, if Remembary eats any of your diary/journal entries, Andrew will personally write fictional ones for you to use instead, and as a bonus, he'll make them even more interesting than your real life.

I couldn't pass up an opportunity like that, so I immediately requested a sample replacement set of diary entries, and Andrew was happy to oblige:

Friday November 12, 2010

Today, I'm going to stay awake only with the sunlight. When the sun drops, I drop. It'll be more natural that way. Spent the day writing Lisp in JavaScript. As usual whenever I write too much Lisp, it becomes self-aware and starts getting in my face, making its own damned refactoring suggestions. This one's just a server app, so it's not as funky as the musical ones that came out of Archeopteryx - it just obsesses over efficiency and speed all the time. Finally needed to shut it up because I was having trouble keeping focussed, so I ran it on Internet Explorer 6, which scared it straight. Fell asleep at 9:30, since it was already dark. Should have started this daylight thing in the summer. Maybe I can move to Alaska and stay up all through June and sleep through December.

Saturday November 13, 2010

Woke up at 2:30 in the morning. Dozed until 7 because what’s the point of getting up at 2? Did more Lisp in JavaScript work today. Whenever it looked like it might be getting sentient again, I pointed it to Wikipedia and Jimmy Wales' cold, soul-eating stare shut it up good. Having a lot of trouble focusing - keep hitting Twitter and finding assholes who need telling. Went to acting class and Megan Fox was there - since she got kicked out of the next Transformers movie, she's back to finding a good monologue and head-shot and trying out for commercials and community theatre. Told her I might have a gig for her. After class speed-walked up and down a hillside for an hour or so, building up my legs of steel.

Sunday November 14, 2010

Spent the day making music. Seeing how much I can do just with just the iPad. Noticed something on the porch so went out to see what it was and it was Yehuda Katz. I couldn't tell if he had a home or was trying to move in or what. I let him come in - although I warned him I wasn't doing much Rails 3 these days. He seemed happy enough on my couch, although he later let himself out and chased the squirrels around the back yard - not very successfully though. He disappeared soon after that - must have gone back home or something.

Still needed to work on my focus, so I went down to Intelligentsia Coffee. Since my sleep schedule is now synced to the daylight, coffee doesn't wake me up as much as it just provides super brain power. Didn't even sit down - instead I wrote up a shell script for distraction-free GMail, while flooring the place with my popping and locking to the choice hip-hop on the PA.

Had to stop early and hurry home before the sun went down. Feel like a reverse vampire. Had enough time to do a few runs up and down the steep hill steps. While dodging a bearded hipster in one of those stupid gear-less bikes I came up with a killer idea for an iPad app - maybe I'll try to find a few cheap devs to work on it for me. Looking for something with a text box - maybe a diary program, perhaps one that automatically includes Twitter and RSS feeds. Nah, that's a stupid idea.

Monday November 15, 2010

Woke up and found my cell phone in the fridge. I guess I shouldn't have tried running that Lisp/JavaScript on it - it must have become sentient in the middle of the night and tried to steal my kale stash, but got caught inside.

Megan Fox came by today, to help me concentrate on my node.js work. Every 8 minutes or so, I'm having her jump into my office wearing a BSD booth babe red devil outfit, with an In-n-Out burger in one hand and a .357 magnum in the other. Really helped me stay off Twitter and Hacker News and focus on work.

Asked her if she's available this coming week - if I can stay off Twitter long enough to build my new custom annoyance-filtering and mass-blocking client, it'll make my life and the world a better place. She said she had some auditions next week - one for a littering Public Service Announcement, and another one for a Pentecostal dinner-theatre show - but if those didn't work out, she'd totally be up for it. It sure beats waiting tables. I've rigged up an Arduino system that notices if any of my devices try to hit Reddit, Hacker News, or Twitter, and turns on an alarm light that calls in Megan - and I've told her that if it goes on more than three times in a day to start using that revolver.

And oh yeah, para-fucking-diddles.


This annoys me because of this, this, this, and this, but most of all this.

Seriously. The Mandelbrot set is an organism, you idiots, and one which only grows in abstract mathematical spaces. IF LIFE CAN ARISE SPONTANEOUSLY IN ABSTRACT MATHEMATICAL SPACES, THEN BY DEFINITION LIFE CAN ARISE SPONTANEOUSLY IN ANY OTHER CONTEXT.

They're fucking RETARDED.

Update: a reader named Matthew and/or Meta sent me this, with the following: "If you have repeated reproduction and mutation, it seems you get genetic recombination and sex automatically as a mathematical emergent phenomenon."

Friday, November 26, 2010

A Brief Word Of Gratitude

Very seasonal: big thanks to James Golick and John Baku for really making me think; to Paul Dix for making me take my startup ideas more seriously; to Courtenay Gasking for being a very accomodating boss (a year or so ago) and giving me the opportunity to work with (and thereby learn from) amazing programmers like Rick Olson, Kyle Neath, Jeremy McAnally, Trevor Squires, Courtenay himself, and several others; to Frank Wimberly for helping me understand statistical AI; to my customers and coaching clients for all the business over the past year; to everyone who's bought anything through an affiliate link on my site; to all my readers and Twitter followers; to my awesome landlords for hooking me up with such a fantastic apartment; to my acting teachers for teaching me so crazy much, especially Jen Krater and Larry Drake; to Evan Phoenix and Shane Becker for; to Coby Randquist, Alf Mikula, and many many others for LA Ruby; to everyone who's had me speak at their awesome conferences; to Pete Forde, for special secret favors (ooh la la); to you for reading this; and to so many other people that I can't remember at the moment, my bad for that.

Crucial Vim Tip: Prevents Copy/Paste From Driving You Nuts

Pasting text into a terminal running Vim with automatic indentation enabled can destroy the indentation of the pasted text. This tip shows how to avoid the problem.

Wednesday, November 24, 2010

Tuesday, November 23, 2010

Canopy: Parsing Expression Grammar in JavaScript

Canopy: inspired by TreeTop, runs on JS.Class.

Sunday, November 21, 2010

Where They Put apache2ctl On OS X


Saturday, November 20, 2010

How To Get In The Zone

Recently, James Golick tweeted the question this post answers. I'm going into detail here because the answer is too long for a tweet, and I'm currently on a self-imposed Twitter ban (I'm not allowing myself back onto Twitter until I've written my own client which enforces my own, very idiosyncratic idea of an ideal Twitter user experience).

The most important thing to understand is that to my knowledge only one researcher has investigated this question in detail and with exclusive focus; that researcher is Mihaly Csikszentmihalyi, and he summarizes his research in the essential book Flow: The Psychology of Optimal Experience.

Csikszentmihalyi's term "flow" expresses the same essential concept as "getting in the zone." A flow state is a state where a person experiences no consciousness of anything except the task at hand. The second and third most important things to understand are 2) that regular experiences of a flow state are the consistent unifying feature of the lives of happy people, and 3) that achieving a flow state requires only two things: that the task at hand be both achievable enough to supply a reward experience of some kind within a relatively short time frame, and that the task at hand be difficult enough that success requires engrossing concentration.

When faced with tasks that are neither achievable nor challenging, flow is impossible; otherwise, all you have to do to achieve flow is segment your tasks into chunks which are both achievable and difficult.

A very important, related data point: procrastination occurs most frequently in the context of vaguely-defined tasks. Thus the same segmenting which sets you up for flow also minimizes procrastination.

It also cures depression, or at least, it did in my case. When I first learned point 2, that flow experiences are the consistent unifying feature of the lives of happy people, I immediately began to train myself on drum essentials like paradiddles. I did this because I was depressed at the time, probably suffering from clinical depression in fact. It cured me of that problem, and I had known from the research that it would. The drum techniques were both achievable and difficult, which made learning them a guarantor of flow, which made me a happy person.

Thursday, November 18, 2010

Higher-Order JavaScript

To my shame, I had to flake on a conference in Sweden called Oredev; to Piers Cawley's credit, he covered for me with what must have been a terrific talk on higher-order JavaScript. Instead of providing his slides online, he instead did a nice, detailed writeup.

Sunday, November 14, 2010

Productivity Boosting Shell Script: Search Gmail Without Viewing Inbox

I'll often go into Gmail to accomplish a specific task and get distracted by shit that doesn't really matter. To solve the problem of entering Gmail to send somebody e-mail, only to be distracted endlessly by new mail, I created a micro-business experiment called Email Without The Inbox. To solve the problem of entering Gmail to search for something, only to be distracted endlessly by new mail, I created a simple bash script for OS X.

Here it is, in its entirety:

# search Gmail THIS WAY, not by going to the Inbox
search_gmail() {
open "$*"


search_gmail whatever

This takes you directly to your search results from the command line, and allows you to skip entirely the whole distracting experience that is your inbox. Note that it only works for one-word search terms, and would need some minor tweaks (namely URL encoding) to support fuller searches. (I'd put the gist on GitHub and encourage you to fork it, but GitHub appears to be down at the moment.)

Saturday, November 13, 2010

Easily Modify Your Sleep Cycle: Ban Artificial Light After Sunset

JD Moyer blogged about a simple sleep cycle experiment: no artificial light between sunset and sunrise. I read about this a couple weeks ago and immediately put it into action, albeit in an incomplete way (I would do a few hours of limited artificial light after sunset). It made me more energized and rested, and had me waking up much earlier, but I've scaled it back for two reasons. First, it's simply very different from the lifestyle I'm accustomed to; second, I have an acting class every Thursday night, which usually runs to 11pm or later, and a hacker meetup Tuesday nights which goes to 10pm. Both events had me experiencing artificial light until well into the night.

I think this produced a destabilizing effect. Certainly, my diurnal rhythms have varied in some fairly unprecedented ways over the past couple weeks since I first began the experiment. This morning, for example, I woke up at 3:15AM fully rested and energized; last night, I was completely exhausted at 9:30PM and fast asleep soon after. I went back to sleep again this morning, and stayed in bed til around 6:30 or 7, not because I was tired, but simply because without any artificial light, there just wasn't that much else to do. This happened to me at least a few times.

This experiment is a little too much for me as a permanent lifestyle choice, but it's proven incredibly effective, so I'll probably do another, similar short run with it at some later date. I suspect the highly artificial nature of "normal" Western sleep patterns degrades health, so my thinking is I'll use this as a kind of restorative technique to limit that damage.

Friday, November 12, 2010

Simple Web App In Sibilant (Lisp + Node.js)

Sibilant is an awesome project which allows you to write Node.js web apps in Lisp. I used it to create, a simple Twitter scanner which catches tweets about RubyConf. The code is on GitHub.

Hello World In Sibilant (Lisp / Node.js)

my fork of sibilant (contains one tiny patch for Node.js API changes)
Original by Jacob Rothstein
Interactive docs / Web REPL

Saturday, November 6, 2010

Blog Comment Similarity Detector (Free Code For Disqus)

A little while back, a Disqus plugin annoyed me in some way. I don't remember how, but I do remember that I tweeted Disqus about it, and they fixed it, mostly. I've decided to return the favor.

Yesterday, the MetaOptimize blog post NLP Challenge: Find semantically related terms over a large vocabulary (>1M)? attracted a ton of retweets on Twitter. A Disqus plugin on the blog post adds those RTs to the post as "comments," using a system called BackType.

For instance, in this screenshot, the top "comment" comes from a guy called turian, and the bottom comment retweets him.

It would be easy to eliminate the pure, classic, literal RTs with a regular expression:

next if alleged_comment =~ /$RT @/

But if you look in the middle, there's a nearly identical tweet with no actual "RT" string. That's because it comes from @hntweets, which apparently tweets links found on Hacker News, and is apparently not the only such account. Here's another account which Disqus also posted as a "comment" on the MetaOptimize blog post.

There was a third one, too, which I spotted, and probably others that I didn't. I don't know why so many people want to build Twitter bots that retweet links on Hacker News, but I don't see them stopping any time soon, either. Likewise, systems like BackType are systematically vulnerable to spam and noise; for instance, Disqus picked up my tweet complaining about their signal/noise problem as a "trackback."

A spammer could easily tweet a link to the post along with "ch3ap vi4gra here," and, more to the point, a spammer could easily write a script to identify every site that uses Disqus and another script to tweet "ch3ap vi4gra here" along with a link to every blog post on every such site.

There's also two tweets from the same guy, both referring to the same URL (once via and once directly):

This means that a spammer could easily fill an entire screen with "che4p vi4gra here," just by submitting the same URL to a variety of URL shorteners.

Any code which combs the firehose needs a noise filter, and the question is how to build it. The regular expression solution won't work here. You can't test for string equality, either. If you take a look at the text of these tweets, you'll notice a ton of very minor variations. Here's one in all lower case:

Here's another screenshot from the same blog post, further down the page, where Disqus posted a ton more tweets as alleged "trackbacks." Again, many feature very minor variations, and none are worth reading.

I've had frustrations with Disqus before, so I should probably just solve this problem for myself with a Zepto bookmarklet which auto-hides the HTML ids typically used by Disqus, but that's not much help to Disqus at all, which means it's no way to return a favor. Plus, it's so easy that it's not worth blogging about. For the record, I've looked at the HTML, and it would probably be as simple as:


And the only bit you'd have to think about would be the process of including the Zepto library in a bookmarklet. I've never done that, but it's probably easy.

Because I like a challenge, and I want to create something Disqus can get some use out of, I've written a simple blog comment similarity detector. Since the Disqus code sets up the comments data as JSON, I wrote the similarity detector in Node.js, but I didn't bother to build a server; this is just command-line JavaScript. (Node.js has excellent command-line features.) I used the excellent underscore.js to supplement JavaScript's weaknesses as a language. I used Node v0.2.3, so if you have any trouble running this, just install the old version of Node. (I'm sorry but Node's API changes way too fast for me to give a shit.)

The code knows how to skip highly similar tweets like "foo http://bar" and "RT @baz: foo http://bar", but it doesn't know how to split text strings on hyphens, which would eliminate even more repetition, so if you want to understand this code, get in there and add that trivial feature. The code is simple, brief, and well-commented. It's 146 lines with sample data and comments, 37 lines without. Check it out on GitHub.

Wednesday, November 3, 2010

Monday, November 1, 2010

Having Something In Common With Rush Limbaugh Makes Me Feel Soiled

I'm reading a neat book called The Millionaire Next Door, about how the majority of American millionaires and multi-millionaires live modest lives almost utterly devoid of bling, and I have to say, I'm really glad I bought it on Amazon, because it's a very interesting book, but if I had seen it in a store, I never would have bought it. This is because the cover features an admiring quote from Rush Limbaugh. What kind of idiot wants to read a book about self-discipline and personal responsibility which is endorsed by an obese drug addict? What's next, How To Get Into Harvard, written by Sarah Palin?

Anyway, despite the obvious insanity of whoever is marketing this thing, or perhaps the obvious insanity of the world it's being marketed in, the book is worth a read. And if an obese drug addict can endorse a book about self-discipline and personal responsibility, why can't a painfully recently-failed entrepreneur with no cash endorse a book about multi-millionaires? Seriously, it's well-researched, it documents something counter-intuitive, and it gives insights into how to get rich. All interesting places to start.

Saturday, October 30, 2010

Node.js Miniapp:

tl;dr: The app. The code.

I've developed a habit of creating aggressively editorial miniapps which modify sites I like to make them better. It started with BoingBoing Minus Disneyland, a miniapp on Heroku that for a while presented BoingBoing's RSS feed, filtered to exclude any mention of Disneyland (I just don't get why they think Disneyland is interesting).

This was one of my first monthly miniapps in 2009. It had some bugs, and Heroku gem changes killed the app, and I didn't maintain it. Next came Hacker Newspaper, which is Hacker News plus better typography, and minus any story from a few domains I don't want to read, including TechCrunch, Coding Horror, and some other idiot.

Bans come direct from my arbitrary whims. On the day the iPad was announced, I banned any story mentioning it, because there were just too many, and most of them were ridiculously boring. I'll probably also add Zed Shaw to the banlist when I get around to it. It's not actually a priority, though. The goal is superior legibility; the editorial aspect is just icing on the cake.

After Hacker Newspaper, much more recently, came my Minimal GitHub Dashboard. This is a list of the original projects for a given username. I use GitHub every day and I love it, but there's a lot of shit on the home page that I just don't care about.

My Minimal GitHub Dashboard uses Node.js and the excellent GitHub API. My new miniapp,, uses Node.js and the competent but imperfect API. It's almost exactly the same code. The only significant addition was the ability to serve static assets using node-static. gives you a form which returns a shortened URL. This is's core functionality. also has some awesome analytical features I very rarely use, plus the usual slew of graphics urging you to tweet this and Like that on Facebook, way more Ajax than I can handle, and a drawing of a puffer fish which looks like an unfortunate, strangled Muppet.

I don't want to look at this fish. His deformity makes me uncomfortable. The guy's got two fins on one side, and less than one fin on the other. You could even call that other side unfinished.

I don't want to see the graphics urging me to tweet, either. I like simplicity.

Fortunately, I can have simplicity, because supplies an API, even to aggro critical bloggers like myself. Apologies to for my terrible attitude, and big thanks for the API. I should point out, to be fair, that my own Minimal GitHub Dashboard CSS is completely useless on iOS devices. (Update: fixed that.) is 125 lines of code, including the Node.js miniapp, views written in Jade, some (iOS-friendly) CSS, and some simple jQuery. As with the Minimal GitHub Dashboard, I built this in Node.js in one evening, and it didn't take long, but it did take longer than I thought it would.

Wednesday, October 27, 2010

Geek Rap Videos

Very mixed feelings about this phenomenon. Reporting it without comment, criticism, or praise.

Tuesday, October 26, 2010

Neo Geo: Web Page To Test HTML5 Geolocation API

I'm looking into creating a location-aware Web-based game for an upcoming conference, and I need to know what level of resolution iPhone (etc.) geolocation supports. Walk around with an iPhone reloading Neo Geo for a while, and you'll have a ballpark answer; walk around with Neo Geo and a spreadsheet, filling in the details, and you can have an exact answer.

To be clear, I hosted this with WiFi and OS X "Web Sharing," which is actually an Apache server which looks for content in /Library/WebServer/Documents. So drop Neo Geo in there, turn on Web Sharing, and Bob's your uncle. Or, put it on your Slicehost (etc.) and you can ping it from all over town. (I was hoping to see long/lat vary just when I walked around my living room, but this did not transpire.)

Update: now hosted at Check it with your iPhone to find out your latitude and longitude.

Pictures Of Altitude And Solitude

Firesheep: Must-Read And Must-Implement

As soon as anyone on the network visits an insecure website known to Firesheep, their name and photo will be displayed...Double-click on someone, and you're instantly logged in as them...

Websites have a responsibility to protect the people who depend on their services. They've been ignoring this responsibility for too long, and it's time for everyone to demand a more secure web. My hope is that Firesheep will help the users win.

Sunday, October 24, 2010

Tech Bloggers Be Lynchin A [Person]

Malcolm Gladwell said that Twitter was no Civil Rights Movement. While this seems like the most reasonable thing anybody has ever written about the internet, Cory Doctorow called it "silly."

The World According To Tech Bloggers



Doctorow was not alone in criticizing Gladwell's post, nor was he alone in disdaining Gladwell's argument as unworthy of serious criticism. Offended, apparently, by the implication that Michael Arrington was no Malcolm X, tech bloggers like Anil Dash and Chris Dixon have been raining shallow condescension on Gladwell for a month.

Dixon, who appears to be a a white millionaire with an Ivy League background, said:

I don’t know if Malcolm Gladwell is right when he claims “the revolution will not be tweeted,” but I can say with certainty that the Twitter he describes is not the Twitter I know...I’d love to engage in a debate with smart people like Gladwell about the impact of the social web on culture, politics, activism and so on...But it’s hard for me to take them seriously when they don’t seem to take their subject matter seriously.

Dash said:

I don't come to refute Gladwell's strawman argument...The traditional method sit-in and picket-in-the-streets form of protest is clearly a failure online.

Dash goes on to argue that people who build stuff are the true revolutionaries, and describes the maker "movement" -- a movement which is, so far as I can tell, nothing but a brand which the brilliant Mark Frauenfelder made up to sell a wonderful magazine. Dash calls it a proud Web-era answer to the Civil Rights Movement.

If we put the making movement in the context of other social and political movements, it's had amazing success. In city after city, year after year, tens of thousands of people pay money to show up and learn about taking control of their media, learning, consumption and communications. In contrast to groups like the Tea Party, the crowd at Maker Faire is diverse, includes children and adults of all ages, and never finds itself in conflict with other groups based on identity or politics.

It's unusual to measure the success of a social movement by the number of customers it converts, but Dash does not clarify this, nor his unusual concept of a revolutionary movement which never finds itself in political conflict with any other group. While Dash's post reads like the kind of unprovoked, incoherent outburst you might expect from an elderly man with Alzheimer's, Doctorow said:

Anil Dash hits one so far out of the park it attains orbit...It's all must-read stuff, but here's the bit that made me want to stand up and salute.

Not everyone who uses the language of revolution is doing anything revolutionary at all. For example, Glenn Beck recently held a Tea Party rally on the anniversary of Martin Luther King's "We Have A Dream" speech, and described it as "reclaiming" the Civil Rights Movement.

The logic in Dash's "must-read" blog post almost sank to the plane of Glenn Beck's logic, and although its conclusions were less destructive than Beck's, it's hard to see them as entirely non-destructive. To put the Civil Rights Movement on a level footing with a bunch of nerds connecting their coffee machines to the Twitter API in their spare time seems to erode the Civil Rights Movement's dignity, at the very least.

The World According To Tech Bloggers

These two things are equivalent:

Hacks are cool, for sure, but if the future of our society depends on them, we might be completely fucked. Social software comes in incredibly handy for pimps, hookers, thieves, creative thieves, abusive ex-husbands, and stalkers. I think seeing technology as a what is inaccurate and irresponsible; technology is a how.

It's not that surprising to find irreponsible politics in an industry with a history of irresponsible design decisions. In my opinion, there's nothing wrong with tech bloggers being too interested in business to have anything worthwhile to say about politics, but if that's where you're coming from, you might as well own up to the fact; and anyone who criticizes Gladwell's argument really should counter its terrific depth and historical background with similarly well-reasoned thought.

Friday, October 22, 2010

iPad Film Experiment

More Live Events Based On TV

A prediction I made, which this book casts light on, came true for the third time.

Friday, October 15, 2010

Indestructible Engineering

An experiment conducted by British TV show Top Gear in 2006 offers one explanation [for the incredible popularity of the Toyota Hilux with warriors worldwide, including American forces]. The show’s producers bought an 18-year-old Hilux diesel with 190,000 miles on the odometer for $1,500. They then crashed it into a tree, submerged it in the ocean for five hours, dropped it from about 10 feet, tried to crush it under an RV, drove it through a portable building, hit it with a wrecking ball, and set it on fire. Finally they placed it on top of a 240-foot tower block that was then destroyed in a controlled demolition. When they dug it out of the rubble, all it took to get it running again was hammers, wrenches, and WD-40. They didn’t even need spare parts.

Facebook's Real Privacy Threat: Your Friends

Here's how you can make your profile on Facebook perfectly secure. First, unfriend all the idiots you knew in high school. Second, unfriend all your friends from high school too. Next everybody you used to go to raves with. Next unfriend that really hot girl who dumped you because she thought you were an insane "genious" and who still hasn't figured out how to spell it. (Do I sound bitter?) Then it's time to unfriend all your relatives.

Continue unfriending until you've unfriended everybody who doesn't work with computers at a very high level of literacy on a daily basis. Then, of those remaining friends, unfriend everybody who knows they should floss, but doesn't do it, or who knows they should go to the gym, but doesn't do it, because chances are that they know they should be using strong passwords too, and they aren't doing it.

This image totally breaks the flow of my blog post, but I had to include it, because what the fucking hell is going on here? Why is this monkey flossing?


Imagine you're a black-hat hacker. You might want to target a specific individual to determine embarassing things about them for the sake of malicious social engineering, such as extortion and/or blackmail. You might want to sell marketing and demographic data to unscrupulous corporations and/or organized crime. (Tangent: is the difference between unscrupulous corporations and organized crime qualitative or quantitative? Read McMafia.)

Anyway, you're a black hat, and you're up to no good. You hit Facebook. Like a street thug in a city with a lot of street traffic, or a cheetah facing a gigantic herd of wildebeest, you have a lot of options to choose from, and you focus on easy targets for the sake of convenience and time management.

What do you do? The smart place to start is with a simple dictionary attack of common words. Try it, and if it doesn't work, simply move on to the next target. Eventual access is more or less guaranteed. Most people don't use strong passwords.

Bruce Schneier wrote:

passwords have outlived their usefulness as a serious security device. Over the years, password crackers have been getting faster and faster. Current commercial products can test tens -- even hundreds -- of millions of passwords per second. At the same time, there's a maximum complexity to the passwords average people are willing to memorize (.pdf [link to research]). Those lines crossed years ago, and typical real-world passwords are now software-guessable. AccessData's Password Recovery Toolkit would have been able to crack 23 percent of the [34,000] MySpace passwords in 30 minutes, 55 percent in 8 hours.

Now consider that the future holds a lot of parallel processing, peer-to-peer massively parallel computation, and virtual machines.

Even if Facebook fixes its issues with technical incompetence and suddenly (or not-so-suddenly) decides to take privacy seriously, its security model is still so stupidly flawed that all Facebook data should be considered effectively public.

Good news for anybody who engages in identity theft. Bad news for everybody else. All the Silicon Valley optimism could turn out to be correct, but if it turns out to be another extended sprint of irrational exuberance, Facebook could face (and be booked with) the biggest class-action lawsuit in American history.

Thursday, October 14, 2010

Ray Kurzweil Is Smoking Banana Peels

Update: Annalee Newitz blogged about this today too (which is weird because I wrote this days ago and put it in scheduled publish mode).

Tuesday, October 12, 2010

(Node) Miniapp: Minimal GitHub Dashboard

(Update: this is dead now, due to API changes. Fun experiment though.)

After watching a great little screencast, I wanted to experiment with Node.js, so I used it to build a minimal GitHub dashboard.

I get distracted easily, so I've wanted (for probably a year at least) to be able to get a page that only shows me my personal GitHub repos, not counting other people's repos I'm also a member of, or forks of other people's projects. That's what this does, and it works for any valid GitHub username. The url looks like this:

If you want to see a different user's personal original repos, all you have to do is substitute their username. I suppose it's a bit antisocial, but it's also very streamlined and convenient. More social features are on the roadmap, theoretically, although in practice I usually just build stuff and move on. Patches welcome; please do investigate. The code is on GitHub, of course, and in addition to Node.js, also uses a nice REST client library called Restler, for the awesome GitHub API, and a lovely haml-esque templating language called Jade.

ln -s: which one's which

Posted because I can never find this via Google with less than three clicks.

ln -s foo bar

creates a link named bar which points to foo.

Update: Several people have pointed out that it works like cp and mv.

Monday, October 11, 2010

Upcoming Experiment: Money Management System

Money management fail has caused me problems in the past, most recently costing me the independence that my entrepreneurial experiments had won me. So I picked up a book on the topic from Amazon with my latest affiliate gift card (I let Amazon pay me with gift cards because I spend so much on books anyway). The book is very badly written, so much so that I couldn't stand to even read it, but I went through it with a highlighter, and in the process discovered a useful money management system that I'm going to experiment with soon. (I won't have time to fully implement it for a few days, as it requires creating several new categorized bank accounts.)

I can happily recommend buying the book, because the content's good, even though the writing is awful. There's a much more readable overview of the system here. With apologies to the author of the book, I recommend the overview more, unless the combination of good personal finance advice and bad writing entertains you for its own sake. In that case, you should check out this guy, who says of making good financial decisions that "it's only 20% head knowledge."

Saturday, October 9, 2010

Two Nations In One State

Nearly everything I thought I knew about American politics was wrong.

A few years ago, I read The Creative Class, by Richard Florida, which documents how artists, programmers, writers, and others who create wealth are clustering in specific cities and fleeing others. Florida's excellent book also documents how this population shift results in greater wealth for cities favored by the creative class, and economic fail for the cities they abandon; and how this creative class typically favors liberal politics.

So I should have seen this coming, but I didn't. If specific cities are typically better educated, with a higher average income, and more liberal, there must be other cities which are typically less well-educated, with a lower average income, and less liberal.

The Big Sort, by Bill Bishop, shows how Republicans and Democrats have been separating geographically since the 1970s, and further traces this phenomenon to the growing polarization of American politics. (In 2006, the Senate broke its previous record for polarization, which was set two years after the end of the Civil War.) Bishop provides ample evidence that these two trends are closely related, and very probably causally related, both through historical analysis of the voting record, and by delving into psychological and sociological research about the polarizing effects of like-minded groups.

One reason Democrats and Republicans so rarely agree on anything any more is that the social dynamics of like-minded groups dramatically encourage polarization; but another is that the two parties increasingly represent entirely distinct nations. When Sarah Palin referred to rural America as "real America," she echoed and confirmed a growing Republican identification with, and habitation of, exurban areas. In Red America, most people live outside of the city, education is less important, incomes are lower on average, nobody doubts the justice of the invasion of Iraq, and Bush had a terrific approval rating. In Blue America, it's the reverse.

Yes We Did, by Rahaf Harfoush, tells the story of the Obama campaign, but leaves out some very important historical context, which The Big Sort provides. Although the technologies and systems of the Obama campaign came straight from social software, with a founder of Facebook leading the effort, the strategies and tactics came straight from the 2004 Bush campaign, which had in turn adopted them from tactics which Evangelical Christianity developed in India and Asia, and later re-used with phenomenal success in the United States.

The tactics rely on gathering groups of like-minded people to encourage one another to vote. Despite Obama's bipartisan rhetoric, he ran (and won) an entirely partisan campaign. It's unlikely he would have won any other way; The Big Sort contains countless stories of moderates losing seats to partisans in recent years, as, indeed, do the headlines. Moderates not only lose seats, they are disappearing entirely as a political constituency. The Big Sort gives you the statistics, so I'll just give you the short version: both the absolute number of moderates, and the percentage of moderates in any particular election, have been shrinking rapidly since the 1970s.

As Public Enemy would say, the future holds nothing else but confrontation.

At the Federal level, this doesn't say good things for our democracy. If like-minded group effects continue making politics more and more partisan, elections could get more and more vituperative, laws more and more controversial, until the common ground disintegrates entirely. However, at the local level, the transformation from heterogenous to homogenous communities means that political unity is very easy to achieve, and consequently you have liberal states decriminalizing marijuana while conservative ones pass laws like the Texas School Children's Religious Liberty Act.

The same forces that debilitate government at the Federal level make it dynamic and innovative at the local level; whether this is a good thing or a bad thing remains to be seen. Although the ability to create local spaces which reflect how people want to live sounds terrific, the larger-scale repercussions are unnerving, to say the least, for a nation with a potent, sizeable military and intense involvement in foreign affairs all over the planet. If you're a smaller country watching a big country with a lot of nuclear bombs alternate wildly between polar extremes, it could make you nervous. The Big Sort also cites research which shows a similarly divided situation in Germany immediately preceding the rise of the Nazis. (Probably fearing Godwin's Law, the book only mentions the research in a footnote.)

The Big Sort made me rethink everything about politics. I'm not done rethinking and it'll probably take me quite some time. I can't recommend this book strongly enough. I'd even go so far as to say that forming political opinions without reading it is reckless.