Sunday, June 10, 2018

A Problem With Side Projects

I see a lot of people on Twitter talking about how, when the tech industry requires side projects, it disadvantages people who don't have free time to devote to side projects, and that those people are typically poorer people, and/or people from under-represented minority groups. In other words, the emphasis on side projects reinforces pre-existing imbalances.

This is legit, but there are really only two ways to keep your skills current: training, or independent study. Which basically means supervised side projects, or unsupervised side projects.

You might be able to do some of this at work. I'm doing a semi-supervised side project at work right now, writing a transpiler we need, and one of my co-workers is doing semi-supervised side projects too, exploring animations in React Native. You could arguably define both of these projects as paid independent study, but we are both cishet white men who got to the point of being able to do paid independent study by first doing unpaid independent study. So it's not exactly a solution.

I think the real problem is that we apply 20th-century mass production thinking to a form of work where it just doesn't fit.

The 20th-century model worked like this: you go to college, emerge with a degree, take a job with one company, work there, retire, and die. But this model only works if your degree sets you up with all the knowledge you need for the rest of your career, or if the process of acquiring additional knowledge is relatively quick and low-effort. This model has notoriously been a failure from day one for lawyers, who, like developers, need to stay up to date as laws and legal precedents change, and who, like developers, need their understandings of those laws and legal precedents to be precise and error-free.

The 20th-century model handled this gap by saying "well, lawyers have to work harder than everybody else, but that's fine, they get paid more too." That "they get paid more" part isn't really even true any more, except for the absolute cream of the crop (or just the luckiest lawyers), and it was never a good solution. Lawyers were and are notoriously overworked for their entire careers because of this flaw in the 20th-century model.

As developers, we either need to take downtime between jobs to learn new things, or we need to study outside of our working hours (including building side projects). Otherwise, you end up writing Fortran while all the good jobs are hiring for Rust and Kotlin.

Fast forward a few decades, and we may find that society has adapted by making it normal to only work 30 hours a week, and have 10 "working" hours for ongoing study. Or we may find that the current situation, where gaps in a resume are considered a red flag, gets reversed. "I don't see any gaps in your resume. How did you stay current without taking breaks to learn new things? Are your skills up to date?"

Getting societies to adapt to new working paradigms is not an easy process, but it's worth the effort.

Saturday, August 5, 2017

Drive Refactors with a Git Pre-Push Hook

When I'm writing code, if I see something I want to fix, I write a quick FIXME comment. When I'm working on my own projects, this is no big deal. But I don't like adding FIXMEs to code at work, or in open source repos. And really, I prefer not to keep them hanging around in my own repos, either. So I set up a git pre-push hook. It looks like this:

 ᕕ(ᐛ)ᕗ cat .git/hooks/pre-push
#!/bin/bash

if ag fixme . -i --ignore tmp/ --ignore log/ --ignore node_modules/ --ignore vendor/; then
  echo
  echo 'not pushing! clean up your fixmes'
  echo
  exit 1
else
  exit 0
fi

This code uses ag to make sure that there are no FIXMEs in my code, skipping dependency directories since they're not my problem, and either returns a Unix failure code, interrupting the push, if FIXMEs are present, or a success code, allowing the push to proceed, if they aren't.

In other words, if I tell git to push code which contains a FIXME, git effectively says no.

This worked well for a while. But soon, at work, somebody else added a FIXME to a repo. So I commented out my hook for a few days or weeks, and then, when I had a spare second, I fixed their FIXME. But soon enough, somebody else added a FIXME, and it was harder to fix. So I commented out the hook, and forgot all about it.

Eventually, though, I had to do some pretty complicated work, and I generated a lot of FIXMEs in the process. As an emergency hack, I uncommented the hook, did my main work, tried to push my branch, and got automatically chastised for trying to push a FIXME. So I went in, fixed them, and pushed the branch.

This is my routine now. If I'm dealing with a big enough chunk of work that I can't take a break to fix a FIXME, I just uncomment the hook for the period of time that I'm working on that topic branch. It's a really good way to stay on task while also putting together a to-do list for refactoring. You basically can't finish the branch until you go through the FIXME to-do list, but you're 100% free to ignore that to-do list, and focus on the primary task, until it's time to push the branch. So you can easily build refactoring into every topic branch without getting distracted (which is the primary risk of refactoring as you go).

It's probably pretty easy to modify this hook so that it only looks at unstaged changes, so I might do that later on, but it works pretty well as a habit already.

Sunday, June 4, 2017

Reddit Users Are A Subset Of Reddit Users

An interesting blog post on view counting at Reddit reminded me of an old post I wrote almost ten years ago. Reddit engineer Krishnan Chandra wrote:

Reddit has many visitors that consume content without voting or commenting. We wanted to build a system that could capture this activity by counting the number of views a post received.

He goes on to describe the scaling and accuracy problems in this apparently simple goal. But he never addresses a few really weird assumptions the Reddit team appears to have made: that every Reddit user is logged in, or even has an account in the first place; that every Reddit user has only one account; and that every account is only ever used by one person.

As I wrote back in the ancient days of 2008:

Assuming a one-to-one mapping between users and people utterly disregards practical experience and common sense. One human could represent themselves to your system with several logins. Several humans could represent themselves as one login. Both these things could happen. Both these things probably will happen.

In other words, "users" aren't real. Accounts and logins are real. The people who use those accounts and logins are also real. But the concept of "users" is a social fiction, and that includes the one-to-one mapping "users" implies between accounts and people. In some corners of the vast Reddit universe, creating more than one "user" per account is as common as it is in World of Warcraft.

As I say, this is all from a blog post back in 2008. More recently, in 2016, I decided that the best way to use Reddit (and Hacker News as well) is without logging in. In other words, the best use case is not to be a "user":

reading Reddit without logging in is much, much more pleasant than being a "user" of the site in the official sense. The same is true of Hacker News; I don't get a lot out of logging in and "participating" in the way that is officially expected and encouraged. Like most people who use Hacker News, I prefer to glance at the list of stories without logging in, briefly view one or two links, and then snark about it on Twitter...

neither of these sites seems to acknowledge that using the sites without being a "user" is a use case which exists. In the terms of both sites, a "user" seems to be somebody who has a username and has logged in. But in my opinion, both of these sites are more useful if you're not a "user." So the use case where you're not a "user" is the optimal use case.


On the Reddit blog, discussing all the technical aspects of view counting, Chandra said the team's goal was to make it easy for people to understand how much activity their posts were driving, and to include more passive activity like viewing without commenting. But there's this obvious logical disconnect here, where Reddit excludes from this analysis any user who isn't logged in. You could even make the case that they'd be better served just pulling the info from Google Analytics, although that'd be naive.

First, given the scale involved, there's likely to be enough accuracy issues that a custom solution would be a good idea in either case. Second, Reddit's view-counting effort made no attempt to identify when two or more accounts belonged to the same human being, or when two or more human beings shared the same login. An off-the-shelf analytics solution could probably provide insight into that, but not definitive answers.

My favorite explanation for all this is that it's just a simple institutional blind spot: people who work at Reddit are probably so used to thinking that people who don't log in "don't count" that it seemed perfectly reasonable to literally not count them. The concept of "users" is such a useful social fiction that the company's employees probably just genuinely forgot it was a social fiction at all (or never realized it in the first place). However, if your goal is to provide view counting with accuracy, skimming over these questions makes your goal impossible to achieve.

Monday, November 28, 2016

ERROR: Permission denied for relation

Stack Overflow will have you mess around with GRANT all day long.

But if you're seeing this error, you're probably doing dev on your laptop and you probably don't need to care any further than this:

ALTER USER myuser WITH SUPERUSER;

Enjoy.

Saturday, October 1, 2016

How Vim Shaped My Writing

Many years ago, when I was a more active blogger, another blogger did a word-frequency analysis of my blog vs other popular blogs in the same niche. They found I had something like three times as many unique words per post as my peers. So they concluded I was a bot, and created a GitHub repo to prove it.



At the time I considered it a compliment to my vocabulary, but what I've realized recently is that it's mostly just a side effect of Vim navigation. By far the most efficient method of getting around in Vim is to search for terms that you know to be unique. I've been using Vim a long time, so I picked up a habit of using as many unique words as possible in my writing.

Sunday, September 25, 2016

JavaScript Is Gradually Supervised Chaos

Imagine if, after Google graduated from tiny startup to megacorp, it had been replaced with a well-run public utility. This is a controversial idea in America, but public utilities can be pretty great. The public subway system in London, for instance, is fantastic. Compare that to private utilities like Comcast.

In this alternate universe, there's some kind of system which kicks in after the success of a startup. "Congratulations, you've made billions, you're rich now, but what you've created is so essential to our society that we can't risk it turning into another Comcast, so we're going to run it as a public service which every American gets for free." This would be a very different America.

There's this weird thing in the US where conservatives refuse to believe that the government can do anything well, but also are outraged if you criticize the military or the police. It's as if working for the government is a guarantee of incompetence, but working for the government and carrying a gun is a guarantee not only of competence but also moral superiority. I don't understand it.

But this is just a mental exercise, set in an alternate universe, so humor me. Imagine this alternate-universe America is like Switzerland, where public utilities are run so efficiently that trains arriving three minutes late constitute a political scandal.

So we're in this alternate universe where the US government takes over companies that become so essential to basic everyday life that we can't risk them turning into Comcasts. In this alternate universe, Comcast itself was taken over, and nobody ever has to deal with their shit. It's a different place, but it's a nice place.

The purpose of this mental exercise is to explain JavaScript. Every time a developer from some other language dives into JavaScript, they freak out at the chaos. You can import specific language features from a spec which has not yet been actualized. There's multiple different package managers. You use them to install each other. There's more than one require() syntax, but there's also really no require() syntax at all.

This chaos occurs because JavaScript, which was born in chaos anyway, operates by allowing its open source communities to develop solutions for failings in the language, and then folds some of those solutions up into the language itself. ES5 turned jQuery into querySelector; ES6 adopted () => from CoffeeScript. This folding-up necessarily lags behind the development of new solutions. And it works well enough that you often get a situation in JavaScript where multiple solutions which are pretty good compete with each other; npm vs Bower, for example, or CommonJS vs RequireJS (which will fade in importance once ES6's import becomes viable, but not disappear immediately).

Most language communities don't operate this way, but most language communities don't have the enormous size or reach that JavaScript does. Everything runs JavaScript today, and every programmer uses it. Go on a job ads site and search for "JavaScript" — you'll see ads for jobs in every programming language that exists, with a little bullet point that says "oh btw you also need javascript," far more often than you'll see jobs that are about JavaScript. Getting a group as colossal as the JavaScript user base to agree on hypothetical descriptions of their needs, or hypothetical solutions, would be incredibly difficult. Letting this massive "community" splinter into subgroups and compete is a better solution.

I'm not saying it isn't messy. I'm just saying there's a reason.

Thursday, September 22, 2016

Elm Is The New Rails

About 10 or 11 years ago, a friend asked me if I'd heard of Rails. She had a developer building something for her, and he wanted to build it in Rails. She asked me to look into it; she figured anything that could get the job done was fine with her, but she didn't know if Rails could get the job done. So I looked into it, and I was like, "holy shit." Then I told her, "yeah, that can get the job done," and aside from some dabbling in Node and front-end work, and a whole bunch of bootstrapped entrepenurial whatnot around information products, Rails has pretty much been my career ever since.

That last part's been true for a lot of people. I don't know if Elm will see the same massive success, but when I compare what it felt like to discover Elm vs. what it felt like to discover Rails, I see a lot of similarities. First, both Elm and Rails represent very curated experiences with a very strong emphasis on being enjoyable to use. Second, that curation involved deep thinking, in each case, about the best way for the framework to do its job, which thinking came in each case to idiosyncratic and (in my opinion) superior conclusions.

Third, they both showed up in very similar environments. Rails had two main sources of competition: an over-ambitious, platform-y technology which was painfully boring and tedious to work with, and an overly simple alternative which encouraged you to write very hacky code once your project grew beyond a certain size. Angular 2 would be the equivalent to J2EE in this analogy, and obviously React, as a view-only technology which intermeshes presentation markup with application logic, would be the PHP.

OK, I apologize. I know Angular intermeshes markup and logic too, and I know both have insane ambitions of being platform-agnostic mega-/meta-frameworks, and indeed, Elm itself has you write your HTML as Elm functions, so it too blurs the line between logic and markup somewhat. But I needed a J2EE and a PHP, and Angular is definitely the over-engineered one.

Only time will tell if these two analogies hold, and indeed I'm a lot more confident in the Angular snark than the React snark. I could be wrong about both. I built a toy recently with React and enjoyed it a lot (but then again, PHP makes a ton of sense for really simple barely-dynamic pages). And certainly, Evan Czaplicki doesn't seem like a DHH to me at all, although I could totally see him as a Matz.

Still, one of the weirdest aspects of Rails's success has been a particular contrast: everybody steals ideas and vocabularies from Rails, but virtually no other project has said, "yes, Rails is right, programmer happiness should be a top priority." You can find migrations, REPLs, code generation, and "REST means CRUD" in every language now, but you won't find "programmer happiness is the top priority" in many other places.

I think this is partly because a lot of the old blog posts where DHH praised and imitated Kathy Sierra have vanished, so nobody remembers that making users happy is about making them awesome, and a framework which makes its users awesome is doing what a framework should do. It could be that the "look at me, I'm awesome!" vibe of early Rails devs was hard to take, so nobody did any serious thinking about it. It might even just be because it's hard for anybody to take DHH seriously in the first place when he has such an egregious straw man problem.

However, one way or another, DHH told everybody that programmer happiness is the secret to Rails's success, and everybody gathered around wondering "what's the secret to Rails's success?" and virtually nobody ever considered the possibility that when DHH said it was programmer happiness, he was telling the truth. And this confused me, and it continued to confuse me for a full decade, and then I found Elm.

I have absolutely no idea if Elm's focus on being fun to use comes from any awareness of Rails at all. But Elm's designed to be fun to use, and, as a consequence, it's fun to use. What a shock! And as it turns out, everything that makes Elm fun to use also makes it a good framework. It has a clean mental model. It doesn't throw mysterious, aggravating runtime errors. Its compiler errors are easy to understand, easy to correct, friendly, and so polite that its British users assume the language itself is British. It's fast as hell in development and production — pull up a Chrome timeline sometime and compare an Elm "hello world" with the festival of document.write that is an Om "hello world" — and it's basically awesome.

I think my new rule of thumb should be that all of my code should be fun to use. It worked for Rails, and it's working for Elm too.

Thursday, September 15, 2016

God What An Awful Directory Name

One aggravation I have with the asset pipeline, and it's not the worst by far, is that app/assets/javascripts is just a terrible directory name.

First, if you're building anything remotely modern, your JavaScript code isn't really an asset, it's an app.

Second, "JavaScripts" is not appropriate here. These are not scripts written in Java. The only meaning the term "JavaScripts" could ever actually have in the English language is if you are talking about varieties or dialects of JavaScript. For instance, ES6 is a much more usable JavaScript than the JavaScript of 1997, but both these JavaScripts have some really weird edge cases, especially around numbers and numeric comparison.

If the directory were called app/assets/javascript-scripts, that would still be stupid, but it would at least have a meaning.

Unless you are, for some insane reason, storing ES7, ES6, ES5, and/or several other entire dialects of JavaScript in app/assets/javascripts, that plural is just wrong.

Update: and I forgot the most obvious reason!

Wednesday, July 13, 2016

Online Trolling Is A Mainstream Thing

This is a music video about misogynistic online trolls. The album this video comes from hit number 1 on the "alternative" Billboard chart. (Both the song and the album appear to have made it into the top 40, for those of you old enough to remember what that was.)



This is a late-night talk show where the last Republican nominee for President reads "mean tweets" (aka online trolling) including one written by the next Republican nominee.



What a time to be alive.

Tuesday, July 12, 2016

How Robotics Will Transform Hollywood

Before CGI, aka 3D animation and modeling, special effects meant practical effects: building little spaceships out of plywood, putting the camera really close to them to make them seem huge, and then setting them on fire. Or a giant, malfunctioning, mechanical shark. Filmmakers like Christopher Nolan still favor practical effects over CGI, because they give the actors something real to react to.



But 3D and CGI are in every movie, because they give filmmakers the ability to depict just about anything, and they've advanced to a degree where they look incredible. So they're not going anywhere.

But I think practical effects are going to come back in a huge way. Today, you don't have to build the little spaceship out of plywood. You can fill it full of tiny servos and sensors. You can make little walking tanks. You can do almost anything.

Animatronic effects and simple machines have been a part of filmmaking for a long time, but that whole process is probably about to become a lot more effective. So there's probably going to be a little practical effects renaissance in about 5 to 10 years, which will make movies feel more real again. It's probably starting already.