Tuesday, April 14, 2009

Miniapp: Hacker Newspaper

In 2008 I took on one of the worst jobs of my life, at a major metropolitan newspaper in Southern California. My managers were inept. One of them was a two-faced political schemer. The other one was spotted at 10am carrying a bottle of Jack Daniels the size of a small child. Not only that, I had actually been working with this particular manager for over three months before I realized they were a manager at all. I thought they were a leftover journalism person who was curious about the Web and had lots of free time.

The environment was if anything even more dysfunctional than you would expect of a dying industry. We played out all the rituals of Scrum, with none of the spirit. We had "standups" where everybody waited until the standup was over so the de facto status meeting (the "parking lot") could begin. During the "parking lot" (status meeting), managers sat and everyone else stood. This frequently happened in the actual "standup" as well. Things got worse. At one point a developer kicked my desk with no warning and then smiled innocently at me in the creepiest way. The job ended when I lost my temper with somebody and emptied a bottle of cream soda in their face.

So when I see the hate that newspapers get from certain corners of the Web, I understand. When I hear that newspapers are dying, I'm less upset than some people. But now matter how awful my personal experience was, I'm not dancing on the grave of the newspaper industry either. I think there are a lot of people on the Web right now throwing the baby out with the bathwater.

And the baby is kind of beautiful.

The typography and layout of a newspaper add a complex, structured hierarchy to the information in the text and photographs.

Let's geek out on this sucker and turn it into an outline.

  1. Tony Blair faces defeat
    1. Labour rebels shred his plans, challenge his future
    2. More detail
    3. A sidebar highlighting his terrorism plan's failure
  2. Child prodigy
  3. Cocaine
  4. Sketch: Hubbub then humiliation
  5. Shit that matters less
    1. Jordan
    2. Nurses
    3. Tesco
    4. Planet

Compare that to Hacker News.

  1. DiggBar
  2. Geolocation DB
  3. TC Tablet
  4. Neural networks
  5. Sorting algorithms
  6. Power
  7. Cleantech/VC
  8. TC Tablet
  9. DiggBar

I personally never read the newspaper, but I read Hacker News every day (and frequently complain about it). Actually not every day. I have a love-hate thing with Hacker News. When I'm not reading it every day, I have it mapped in my hosts file to to prevent myself from seeing it at all. But that's not the point. The point is: compare the outlines. If you are a geek, you are probably consuming information in a less sophisticated structure then your non-geek peers.

For all the bells and whistles in this UI, there's nothing to connote the information's structure, other than a plain, unadorned vertical list.

April's miniapp fixes that problem, or at least a subset which was for me particularly aggravating.

It uses a resizing-friendly CSS grid setup called Typogridphy. The grid leverages the hundreds of years of honed and refined typographical knowledge that most news sites on the Web ignore - even newspaper sites - and since the CSS is done well, you can zoom the page out to tiny font sizes.

Even at tiny font sizes, the top three or four headlines on Hacker News are actually still more readable on Hacker Newspaper.

It's faster, too. It's much faster. Hacker Newspaper runs on a cron job. Every hour, it uses a Python script to download the RSS feed from Hacker News, plus every page that the feed links to. It then uses a Ruby script to parse the feed, pass it through a template, and produce flat HTML.

It serves the HTML statically, which makes it much faster than Hacker News, which runs dynamically in real time on Arc. It is a faster, compiled alternative. Y Combinator has millions; I'm running Hacker Newspaper off a $20 Slicehost account. But switch to Hacker Newspaper for one day and you'll never go back, because Hacker News is a slow-ass beast.

Hacker Newspaper is a superior user interface for Hacker News. It's more performant, more readable, it doesn't turn visited links damn near invisible for some insane reason, and it makes it much easier to skim the headlines and avoid getting into useless, time-wasting blather. As is usually the case with my monthly miniapp side projects, I created it to scratch my own itch, but you are more than welcome to use it too. You'll enjoy it best if you have the classic Bodoni SvtyTwo ITC TT on your machine, but it degrades gracefully as well.

This might fall on deaf ears, but if there's anything Web geeks could use more of, it's typography. Imagine how much nicer it would be to read log files if your log file reader automatically reformatted your log files to emphasize the information you were searching for, not in a garish, clumsy way but in an elegant, readable way. Imagine how much nicer Google searches would be if they exploited the sophisticated structure of newspaper-style layouts. A lot of what people on the Web are doing with typography reinvents a smooth, polished, and very round wheel. Since it's existing knowledge which loads of people have failed to notice, you can get a technological edge on your competition using technology invented in the 1800s. We might be done with the printing press for day-to-day news, but disregarding the craft of typography is a ridiculous mistake.

Update: Hacker Newspaper's on GitHub.

Update: Any links to TechCrunch articles on Hacker News automatically get turned into null links on Hacker Newspaper.

Update: Hacker Newspaper now just skips any links to Coding Horror or TechCrunch. It doesn't set them as null links any more, it just skips the stories entirely and treats them as if they weren't even there. The same is true for skorks.com. In general, you should consider Hacker Newspaper a biased and curated version of Hacker News. For instance, when the iPad was released, I briefly banned any stories with "iPad" in the title. There were simply too many.

Update: Hacker Newspaper looks pretty great on an iPad, btw.


  1. Make me think of Times: http://www.acrylicapps.com/times/

  2. Yeah, Times was one of the inspirations for it. I didn't dig the app but I liked the idea. Also tag clouds. Tag clouds do the same thing with size and position that newspaper headlines do, but they don't go all the way.

  3. The other thing that newspapers do that's endangered besides the logical organizing of information is, you know, actually collecting new information that is non-obvious or that someone want to keep secret. There is pretty much no digital-only entity that's ever done this (with a few notable exceptions like Talking Points Memo). Before people on the web start dancing on the grave of newspapers, they should try from them some of how to actually do new reporting about events in the world.

  4. Well yeah. Blogs actually have the same relationship to newspapers that Hacker Newspaper has to Hacker News. HN(paper) wouldn't be useful at all if HN wasn't supplying the feed. It's just a reformatting feed consumer. Blogs feed off of the traditional news media in the same way, although there's also a lot of original material.

    That's why people are still going to journalism school. It's not about the thin sheets of tree or the classified ads in the back. However I think "no digital entity" borders on hyperbole. I'm sure a few counter-examples exist.

  5. Can you please explain what typogridphy does ? Couldn't make out much from the page.

  6. great execution! a much better experience than hacker news

  7. Cool app! Could you explain a bit how your Ruby script pulls out the "meat" of the linked pages for the content after the headline? It's something I've always wondered how to do appropriately.

  8. Twenty years ago, ACM printed a study of the use of typography to make C programs more readable:

    I'm the only C programmer I know who's even heard of this book.

    Even though I don't agree with all of it, and even though it's technically non-trivial to integrate typographic knowledge smoothly into normal programming, I think it's a crying shame that we're still using such incredibly impoverished presentations for our ideas.

  9. Somebody anonymously posted a link to a discussion of this post on Hacker News. Thanks (or something) but I deleted it. I just spent five minutes writing a bunch of swears to some ignorant dipshit who wasted my time with his arrogance while knowing nothing about typography or user interface. People, PLEASE, no links to discussions on Hacker News. The whole point of this system is to reach a higher level of information hygiene by keeping that ridiculous time-wasting garbage out of my perception.

  10. Giles,

    Thanks for showing us what is missing from the online world. Amazing that typography changes information consumption.

    I like that you are grabbing some of the content from the linked site. Would there be anything to be gained by providing more content from the linked article?

    The sizing and spacing seem random, what decisions cause the layout of the page?

  11. @64B - sizing and spacing come from the order links appear on Hacker News. I actually grab all the content from the linked article, and the code could also grab images etc., but I didn't take it that far. I just wanted enough to add dimension to skimming, so I could avoid clicking links that turned out to be boring.

  12. So Giles, does your ruby app swill cheap whiskey and chain smoke ... got to take the bad with the good!

  13. Looks nice and readable. Would it be worth it to archive older snapshots and allow stepping through them, the way newspaper sites have links to yesterday's paper? Also, while there's a link to the main news.yc page, I searched in vain for a link to each article's discussion page. Sometimes the comments are more valuable than the original article.

  14. @Anon - the inaccessibility of comments is a feature.

  15. Oh nicely done. Actually makes me prepared to go back and look at HN on a daily basis again.

  16. What we really need is an easier way of extracting the underlying data of the website and an easier way of making whatever interface we want for it.

    This is why browsers don't download "compiled" HTML/CSS/JS. They download the source code, the raw stuff but that isn't enough if we can't modify it easily.

    Your HackerNewspaper may be fine for you and others, but what if I want a tree structure of all the news items, or I want them all sorted by their rank, then how many comments there are, or what if I want them displayed in a 3d rolodex or whatever? Can't do it without all that hacking that you had to do.

    The real problem is access to data, not the interfaces. If we had better access to data we could design our own interfaces as you've shown.

  17. @OMouse yes, HN needs an API but no, access to data is not the real problem. access to data is an irrelevant threadjack. if you want a tree structure of all the news items, or you want them sorted by their rank, or you want to display them in a 3D Rolodex or whatever, you can get off your ass and do it, just like I did. I wish the code was already on GitHub so I could show you, this project was easy as a motherfucker. It's like ten lines of Ruby, plus a bunch of free shit other people had already written. "What we really need is an easier way of extracting the underlying data" is ridiculous. This project couldn't have been any easier if a robot had done it for me.

  18. @Dale my project even has its own built-in girl Friday


Note: Only a member of this blog may post a comment.