Saturday, August 5, 2017

Drive Refactors with a Git Pre-Push Hook

When I'm writing code, if I see something I want to fix, I write a quick FIXME comment. When I'm working on my own projects, this is no big deal. But I don't like adding FIXMEs to code at work, or in open source repos. And really, I prefer not to keep them hanging around in my own repos, either. So I set up a git pre-push hook. It looks like this:

 ᕕ(ᐛ)ᕗ cat .git/hooks/pre-push

if ag fixme . -i --ignore tmp/ --ignore log/ --ignore node_modules/ --ignore vendor/; then
  echo 'not pushing! clean up your fixmes'
  exit 1
  exit 0

This code uses ag to make sure that there are no FIXMEs in my code, skipping dependency directories since they're not my problem, and either returns a Unix failure code, interrupting the push, if FIXMEs are present, or a success code, allowing the push to proceed, if they aren't.

In other words, if I tell git to push code which contains a FIXME, git effectively says no.

This worked well for a while. But soon, at work, somebody else added a FIXME to a repo. So I commented out my hook for a few days or weeks, and then, when I had a spare second, I fixed their FIXME. But soon enough, somebody else added a FIXME, and it was harder to fix. So I commented out the hook, and forgot all about it.

Eventually, though, I had to do some pretty complicated work, and I generated a lot of FIXMEs in the process. As an emergency hack, I uncommented the hook, did my main work, tried to push my branch, and got automatically chastised for trying to push a FIXME. So I went in, fixed them, and pushed the branch.

This is my routine now. If I'm dealing with a big enough chunk of work that I can't take a break to fix a FIXME, I just uncomment the hook for the period of time that I'm working on that topic branch. It's a really good way to stay on task while also putting together a to-do list for refactoring. You basically can't finish the branch until you go through the FIXME to-do list, but you're 100% free to ignore that to-do list, and focus on the primary task, until it's time to push the branch. So you can easily build refactoring into every topic branch without getting distracted (which is the primary risk of refactoring as you go).

It's probably pretty easy to modify this hook so that it only looks at unstaged changes, so I might do that later on, but it works pretty well as a habit already.

Sunday, June 4, 2017

Reddit Users Are A Subset Of Reddit Users

An interesting blog post on view counting at Reddit reminded me of an old post I wrote almost ten years ago. Reddit engineer Krishnan Chandra wrote:

Reddit has many visitors that consume content without voting or commenting. We wanted to build a system that could capture this activity by counting the number of views a post received.

He goes on to describe the scaling and accuracy problems in this apparently simple goal. But he never addresses a few really weird assumptions the Reddit team appears to have made: that every Reddit user is logged in, or even has an account in the first place; that every Reddit user has only one account; and that every account is only ever used by one person.

As I wrote back in the ancient days of 2008:

Assuming a one-to-one mapping between users and people utterly disregards practical experience and common sense. One human could represent themselves to your system with several logins. Several humans could represent themselves as one login. Both these things could happen. Both these things probably will happen.

In other words, "users" aren't real. Accounts and logins are real. The people who use those accounts and logins are also real. But the concept of "users" is a social fiction, and that includes the one-to-one mapping "users" implies between accounts and people. In some corners of the vast Reddit universe, creating more than one "user" per account is as common as it is in World of Warcraft.

As I say, this is all from a blog post back in 2008. More recently, in 2016, I decided that the best way to use Reddit (and Hacker News as well) is without logging in. In other words, the best use case is not to be a "user":

reading Reddit without logging in is much, much more pleasant than being a "user" of the site in the official sense. The same is true of Hacker News; I don't get a lot out of logging in and "participating" in the way that is officially expected and encouraged. Like most people who use Hacker News, I prefer to glance at the list of stories without logging in, briefly view one or two links, and then snark about it on Twitter...

neither of these sites seems to acknowledge that using the sites without being a "user" is a use case which exists. In the terms of both sites, a "user" seems to be somebody who has a username and has logged in. But in my opinion, both of these sites are more useful if you're not a "user." So the use case where you're not a "user" is the optimal use case.

On the Reddit blog, discussing all the technical aspects of view counting, Chandra said the team's goal was to make it easy for people to understand how much activity their posts were driving, and to include more passive activity like viewing without commenting. But there's this obvious logical disconnect here, where Reddit excludes from this analysis any user who isn't logged in. You could even make the case that they'd be better served just pulling the info from Google Analytics, although that'd be naive.

First, given the scale involved, there's likely to be enough accuracy issues that a custom solution would be a good idea in either case. Second, Reddit's view-counting effort made no attempt to identify when two or more accounts belonged to the same human being, or when two or more human beings shared the same login. An off-the-shelf analytics solution could probably provide insight into that, but not definitive answers.

My favorite explanation for all this is that it's just a simple institutional blind spot: people who work at Reddit are probably so used to thinking that people who don't log in "don't count" that it seemed perfectly reasonable to literally not count them. The concept of "users" is such a useful social fiction that the company's employees probably just genuinely forgot it was a social fiction at all (or never realized it in the first place). However, if your goal is to provide view counting with accuracy, skimming over these questions makes your goal impossible to achieve.

Monday, January 16, 2017

JS Testing In Rails Apps: The Problem Space

I've got to do several things before my new book, Modern Front-End Development with Ruby on Rails and Puppies, is finished. It's currently only in version 0.2! One of the biggest things is to delve into JS testing in serious depth. The "with Puppies" part of my book's title doesn't add any extra challenges to that task, but the "with Ruby on Rails" part totally does.

The Rails culture has very high standards and expectations when it comes to the sophistication and specificity of its testing options. Say you've got a big Rails app which has presenters, decorators, and serializers. That's three different categories of view model objects — and standard methods of testing these objects exist both for RSpec and for classic TDD syntax (i.e., the assert style of Test::Unit and minitest). So you can go out and find, for example, specific, opinionated guidance on how to write tests for presenters using minitest. And I've obviously defined a 3x2 grid here, with 3 categories of view model object, and 2 categories of testing strategy. All six slots in the grid represent a strain of thinking within the Rails culture that lots of people are working on in detail. If you want to research any one of those six slots, there's a ton of prior art.

And that's just for view model objects. What about objects which move business logic out of the framework? Those can go in lib, or app/services, or app/interactors, or several other places. Want to test those quickly and well? There are projects which give you several different ways to do it. If you see open source development as a form of research, then you have this huge body of ongoing research. All these different strategies for separating business logic from the framework, and all the different ways of testing the objects which implement these different strategies — they all represent ongoing research projects into the best way to structure an application, and the best way to design an application, and/or secure that application, and/or prove its correctness (depending on how you see the purpose of testing in the first place, which itself is an area of ongoing "research," in this sense).

The common structure of Rails apps creates a shared vocabulary, not just for application logic, but for testing as well. You can have subtle, specific discussions about how to compartmentalize, organize, and structure responsibilities and processes across Rails apps, even when those apps do very different things.

Compare that to JavaScript. How many JavaScript applications share a common structure? Most don't.

The size of the Ruby community also helps here, in comparison to JavaScript. "The JavaScript community" basically means the same thing as "every programmer on the planet who ever works on or near the web." In fact, it's larger than that; if you want to script Adobe applications, for example, you use JavaScript. So for Ruby, you have a small community, where the overwhelming majority of programmers work within the same application structure, and where virtually everybody agrees that testing is important. For JavaScript, you have a group of people which is far too large to ever function as an actual community. Most applications have relatively idiosyncratic structures, and many of them change radically from month to month. What kind of consensus can emerge there?

The good news is that there are plenty of JS programmers who value testing, and plenty of JS test frameworks, to enable that. But the bad news is that many of these testing systems never get much further than unit testing. There's no Capybara of JavaScript, for example — not really — which is pretty crazy, because front-end coding is the type of work where you would expect to find a Capybara being developed.

Say you've got a Rails app with a React front-end. Maybe you're using Rails controllers for the regular web stuff and Grape for the API stuff that your React code consumes. So you can put your Grape specs in spec/requests and write simpler specs, and keep your controller specs in the more usual spec/controllers, and deal with the unfortunate downsides, i.e., the slowness, the intricate connection(s) to the framework, et cetera. But when you want to test your JS, you don't get these kinds of fine-grained distinctions for free. You can use moxios to mock your axios requests, but that's kind of as sophisticated as it gets. You have unit testing, and you have mocks, and there you go. It's like that scene in The Blues Brothers, when they ask "what kind of music do you usually have here?" and the bartender says, "we got both kinds of music, country and western!"

With JS, there are many ways to do mocks and stubs, and there are many ways to do unit testing, but there's nothing as purpose-specific as spec/requests vs spec/controllers. You're also flying blind somewhat when it comes to distinctions as fine and precise as when to use a presenter, when to use a decorator, and when to use a serializer (which is basically just a presenter for APIs).

One of the benefits of something like React is that it's an improvement just to get a sub-community (or subculture) of JS devs who all use roughly the same application structure, because that common vocabulary permits the subculture to develop more sophisticated and specific testing libraries. As far as that goes, Angular and Vue and other frameworks have a similar positive influence. However, this is especially a strength of React in my opinion, and in fact, I need to update my book to reflect that. For some reason, when people talk about React, they talk a lot about FRP and virtual DOMs, but when you get past the hand-waving, the actual day-to-day work of React is very object-oriented. There are a lot of tradeoffs to consider with OOP, but one great positive is that OOP systems present very testable surfaces. It's really easy to see where you start testing, when you're dealing with objects.

Elm has this "common application structure" advantage as well, but with Elm, you have types, and types relieve a lot of the pressure on tests. That's not to say that you don't need tests when you have types. But since the compiler is guaranteeing you certain behavior, you don't need tests to secure that behavior. People use tests to design systems, to secure systems (and prevent regressions), to document systems, and for a few other purposes. Elm's types make it easy to get most of your security cheaply. Elm also pushes you towards better designs in ways that are more subtle and which I couldn't perfectly articulate just yet.

However, in both these cases, this isn't a solution, but rather a reason to be optimistic that the cacophony which has prevented solutions from arising will soon diminish, at least somewhat.

I'll have more to say about this in the future; for now, this is just an overview of how the problem space for JS testing in Rails apps differs quite a bit from Ruby testing in Rails apps.

Monday, November 28, 2016

ERROR: Permission denied for relation

Stack Overflow will have you mess around with GRANT all day long.

But if you're seeing this error, you're probably doing dev on your laptop and you probably don't need to care any further than this:



Saturday, November 5, 2016

Rails Isn't A Full-Stack Framework Any More

Rails is a terrific framework. Its emphasis on programmer happiness and convention over configuration makes building apps easy and fun. But it has some areas of cognitive dissonance. For instance, Rails bills itself as a full-stack framework, and I think that just isn't true any more.

The Rails solution for the front end is jQuery, unobtrusive JS, and CoffeeScript. These are great technologies, but each and every one of them is showing their age.

jQuery's the most egregious example. JavaScript systematically adopts its most successful open source projects and turns them into language features. Because of this, a lot of jQuery's functionality is now provided by browsers, including the fundamental thing of querying the DOM and modifying pieces of it. For sophisticated, modern use cases, jQuery can give you slow performance, especially on mobile devices, and it carries significant risks of brittle, unmaintainable code.

Unobtrusive JS is the idea that your JavaScript should provide decoration, not functionality. It came to Rails in 2010, the same year jQuery did, and each of these things was great for 2010. Rich single-page apps had existed since Gmail debuted in 2004, but in 2004, you kind of had to be a Google to pull it off. Even in 2010, unobtrusive JS was a reasonable, pragmatic default.

Today, however, it's just one of many examples in the history of Rails and its many ways to avoid writing JavaScript.

CoffeeScript is still pretty cool in my opinion, but here's the downside with CoffeeScript. A few years ago, Jen Fong-Adwent created a cool JavaScript project called Meatspace. It's a text chat client which creates an animated GIF of your face, using your webcam, every time you say anything in the chat. It's kind of dead now, but when she created it, it was full every night, with tons of people having fun. So I went in there and I decided I should read the code, to figure out how it worked. And I had a much harder time of that than I wanted, because Meatspace used new features of JavaScript that I hadn't seen before. Despite years of experience, I felt totally out to sea.

And that was a few years ago. Since then, ES6 has made JavaScript into an entirely new (and better) language. So while I still really like CoffeeScript, I also consider it kind of a dangerous choice, where you might be sacrificing your skill level for the sake of a little comfort.

This is the motivation behind my new book. If you're a Rails developer, and you want to do modern front-end work, you know Rails just basically skips the whole question. Which front-end tech should you use? How should you use it? You might be used to Rails setting you up with really useful defaults, but that's not what it does when it comes to front-end development. So my book gives you the necessary context to make these kinds of decisions, and walks you through several alternatives for modern front-end development.

Rather than sell you on it, though, I'd prefer to just tell you what I heard from somebody who bought it. Andrew Stewart, a Rails developer from the UK, wrote the PeepCode videos on Vim back in the day — they're now on PluralSight — and he said I could quote this awesome email he sent me. So here it is:
Hey Giles,

I just finished your book (v0.2) and I thoroughly enjoyed it.

I run a little SAAS Rails app and I'm always working on features, simplifications, i18n, support, etc.  I rarely find or make the time to upgrade the underlying stack.  As a result it's way behind; the front end is asset pipeline circa Rails 3.2 with jQuery v1.11.0, Coffeescript, SASS, some CSS I haven't converted to SASS yet, and a large amount of crappy 2007-style JavaScript I haven't touched for years.

Your book is exactly what I need!

I like how it's full of stuff I can do immediately to modernise everything.  I now intend to replace the asset pipeline with webpack, drop jQuery, and rewrite the old JavaScript in ES6.  Eventually I'll replace the Coffeescript with ES6 too.  Having said that, I'm looking forward to your Elm chapter...  (The React and Om chapters helped me decide to steer clear of them for now.)

Just one suggestion for v0.3 - a table of contents would be nice.

Anyway, I can safely say this is the most helpful programming book I've bought for years.  Thank you!


There is a table of contents coming, by the way, and the Elm chapter's about half-written. I have the code but it's not in a repo you can see yet. Sorry -- I write the code for these books in separate repos first, to figure out what I'm doing, and then re-write them for the book repo. This way I can make sure my commits are structured in a way that is more logical and easier to follow. I want reading the git repo to be as clear for my readers as reading the book itself.

Also, the React chapter might convince you to give React a shot. It's about thirty pages, and the last ten pages are all about the tradeoffs you have to consider when it comes to giving React a thumbs up or a pass. Those tradeoffs are different for different projects.

Regardless, I was obviously thrilled to get Andy's email, and I hope you found my own email useful.

Click here to learn more about my book.

Or, to buy it directly:

Wednesday, October 12, 2016

Modern Front-End Development with Ruby on Rails and Puppies

Update: I'm taking this book off the market for the moment. I never finished the Elm chapter because of some health problems I encountered, and the guilt about that is quite frustrating.

If you've found it baffling, stressful, or confusing to get started with modern JavaScript development, I have an easy solution for you. I wrote a book which not only walks you through the process, but does so with an adorable, reassuring puppy on every page.

The first, beta edition of my new book is now available. It costs $41. Here are a few sample pages.

This book has a lot more color and design than the average tech book. I did a lot of production work for ad agencies and design shops before I became a developer, and I did a little design work, too. I'm not going to lie, I'm a little bit rusty, but I think tech books should be fun to look at, especially when they're about the front end. This also happened because I was inspired by Lea Verou's terrific book CSS Secrets.

For related reasons, the code in this book uses syntax highlighting. I think syntax highlighting is important in tech books. For a lot of programmers, it's very rare on a day-to-day basis to ever even see code which doesn't have syntax highlighting. So if you're used to seeing code with syntax highlighting, and then a book shows you the code without syntax highlighting, you're actually getting less information from the code than normal. Books on code should give you more context, not less. And, if you're a newbie, and you're buying my book, I want you to get a realistic experience of what being a programmer is like. I can understand why books printed on physical paper in black and white don't use syntax highlighting, because ink costs money, but if you're publishing a PDF — and I am — then I really think syntax highlighting ought to be a given.

The book also comes with two git repos, because the repos allow you to follow along with my code as you go through the book. If you've ever done this with another book and found out that you had to figure out what the author forgot to tell you in order to make their code work, guess what? Non-issue when the author gives you a git repo. If they forgot to mention something, you just run a git show or a git diff to get the exact changes.

Plus, a lot of code books will give you code you can download, but they just give you the finished product, or a bare-bones sandbox to play in. A code base isn't a finished product, it's a continuum over time, and with a git repo, it's a carefully indexed continuum over time. I think that's an important aspect of making code clear, so I first did all my code-related research in other repos, and then built these repos so that the repos would themselves be easy to read.

Also, to continue the theme that I think a tech book should make use of modern technology, the book has about 200 links, in its current state. An upcoming chapter adds 27 more, and future chapters will add more still. So, if at any point you want more detail, there are plenty of opportunities to dive deeper.

What I'm putting on sale today is v0.1, and I'm expecting to do a few updates before I call it 1.0. Despite that, this early-release version has a lot to it. There are the two repos I mentioned: a small repo to help you understand a new CSS standard called Flexbox, and the main example app for the book. The book covers the example app's step-by-step journey from a front-end based on jQuery and CoffeeScript to one built using React, ES6, and JSX. The book also features a puppy on every page.

The example app is a simple flash cards app for learning and studying written Japanese. These pages feature screenshots:

This app is simple enough that I can cover every change in detail, but complex enough that it can drive some investigation of different strategies for the front end. In some ways, it's like reading ToDoMVC implementations, except there's an explanation for every decision, and these decisions get compared to each other. I use a 12-point checklist to compare different front-end strategies. (I trim that checklist down for some chapters, however; for instance, the Webpack chapter is about infrastructure, so it skips some unrelated considerations.)

The existing chapters cover setting up this app as a vanilla Rails app with jQuery and CoffeeScript, translating it to ES5 — because jQuery's features are basically built into every browser at this point, and using the language instead of a library has much better performance characteristics — then translating it again to ES6 via Babel, adding CSS styles (including Flexbox), replacing the asset pipeline with Webpack, and rebuilding the app in React. There's also an experimental ES8 feature. Every step in the process is clearly explained, and there's a GitHub repo where you can file issues if you encounter errors, typos, or ambiguities.

Although I haven't set up a formal roadmap, I have a pretty strong idea what version 0.2 will look like. I've already got the next chapter written. In it, I translate the React app into an Om app, using ClojureScript. I've also translated that app again into an Elm app. I've got the code for that, and I'll be writing an Elm chapter around it soon. That's probably what you get in version 0.3.

(The Om/ClojureScript code is included in the repo, if you want to skim ahead, but I'm expecting to release version 0.2 pretty soon anyway.) (Update: The version 0.2 release just went out!)

To be clear, if you buy now, you get the new versions when it's updated, unless I ever release a 2.0 or higher version. Buying the 0.1 version means you get all the 0.x and 1.x versions free. You might get grandfathered in for a 2.0 version also, but you might not; I want to leave that open, because it's too far off for me to make any realistic plans around it.

Of course, if you're not satisfied for any reason, just let me know and I'll happily give you a full refund. You literally don't even have to give me a reason. I just want my customers to be glad they bought from me.

Friday, October 7, 2016

More About Ruby And Japanese

My "Ruby is 和" blog post is about 8,500 words long. One of the things I cut, to get it down to that compact size, was a section about similarities between Japanese and Ruby, as languages. I also cut it because I spotted a problem with the argument.

First, the part I cut:

One way to understand 和 is as a harmonious blend of disparate elements. This is an accurate description of the Japanese writing system. Written Japanese employs kanji (for example, 和), which are Chinese letters that carry the same meaning they originally held in classical Chinese when Japan adopted them, plus new meanings that Japan has added in the subsequent centuries. Written Japanese further contains our entire alphabet, which, as it was originally the Roman alphabet, is called romaji in Japan. Written Japanese also has two syllabaries, which are like alphabets except each letter represents a complete syllable. The syllabaries are called kana. While kanji and romaji are both character sets in use outside of Japan, neither of the kana syllabaries are used by any other language. However, even with the kana, there's a magpie element in effect: both syllabaries evolved from streamlined, abbreviated ways of drawing kanji.

Each of these four different character sets has different uses in written Japanese, and despite the enormous complexity, it all kind of balances out. Different character sets have different purposes. For instance, one of the syllabaries, the katakana, is primarily used for loanwords — like English, Japanese has a lot of loanwords — but it also functions as an equivalent to italics. Romaji, meanwhile, allow you to type Japanese characters using Western keyboards. Kanji have to be individually memorized, which is a lot of work for Japanese schoolchildren, but kanji have some advantages over written English, which may spell two different pronounciations the same way — e.g., "read" in the present tense vs. "read" in the past tense — or introduce subtle and totally arbitrary distinctions in spelling, like "write" vs. "right." This is also just something that kids have to memorize in school.

Now, for comparison, here's a Python fan on Reddit, complaining about Ruby:

I started working on ruby with a friend and was amazed by the multiplying way they have of making methods. For instance, ruby has but it also has a .each. There are two different notations for ranges (inclusive and exclusive), and myriad unexpected methods for other things: unless (instead if not), upto (instead of the aforementioned range), etc.

It all seemed philosophically messy to me, like they just let whomever patch in whatever tool seemed most useful at the time, and did not imply the same rigid philosophical framework that the python community has adopted.

This could be a fan of English complaining about written Japanese, although English is perhaps a bad example, given that English has more corner cases than most languages. But it still has only one alphabet! And written Japanese definitely lacks unity by comparison.

I won't pick another example, as that could be a huge discussion. You get the idea. Ruby's cheerful willingness to accomodate multiple ways of doing the same thing has an obvious parallel in written Japanese. Fans of Sapir-Whorf will be thrilled to recognize this continuity, where a characteristic of the Japanese language reappears in a programming language created by a native Japanese speaker.

However, Matz himself has said that Ruby was not deliberately modelled on Japanese.

Japanese and Ruby? I try not to think too much about Japanese culture. The method chain looks like Japanese, but it’s just a coincident. Having said that, the support of M17N is heavily influenced by the use case of Japanese people. Otherwise, I wouldn’t spend too much time on such a hard problem.

This is from a tweet in Japanese. The translation comes from Makoto Inoue, a Japanese programmer based in London. He did a very interesting presentation about this, and wrote a blog post which summarizes it. He points out that Japanese sentence structure — subject first, verb last — works well with the Subject.verb syntax favored by object-oriented languages, and raises the same point I've made here about Ruby's versatility:

Doesn’t this “There are many ways to achieve one thing” concept [seem] familiar with Ruby’s philosophy?

Now for the problem with the argument, i.e., the second reason I didn't include this in my "Ruby is 和" post.

Here's a sentence in Japanese:


It's pronounced "shitsumon ga arimasu." You could translate it as "I have a question," and that's what it usually means in my extremely limited experience. I personally say this in emails to my Japanese teacher all the time. But that's not a literal translation.

This sentence has only three words. しつもん ("shitsumon") means a question, or some number of questions, since, like Cantonese, Japanese does not differentiate between the singular or the plural. が ("ga") is a modifying particle like the modifying particles in Attic Greek. It basically means "the word I just said is the subject of this sentence." Since it comes after しつもん, it means the sentence is about a question or some number of questions. The verb, あります ("arimasu"), literally means "exists," or "will exist." Japanese does not differentiate between the present and the future, at least not in the way that English does.
If the languages I'm using for comparison seem random, that's because they are. I'm not a linguist, I've just picked up a bunch of random knowledge here and there.

So the literal translation for しつもんがあります would be "a question, or some number of questions, exists, or will exist." It's up to the reader or listener to understand that, in context, this means roughly the same thing as "I have a question."

Because of all this ambiguity, idiomatic Japanese relies heavily on inference and implication. And this is the problem with the argument that Ruby mirrors Japanese in any meaningful way, in my opinion. Makoto Inoue has an interesting argument which relates this aspect of Japanese to Ruby's functional characteristics. But my concern is that you could imagine a possible programming language which emulated this aspect of Japanese, and Ruby differs from that imaginary language.

For example:

@user = User.find

Here, we have a bit of Ruby which finds a user and has that user register for something. We have to say @user twice. If Ruby were more like Japanese, it might implicitly capture the return value of each expression, and apply the next line to that return value — or resolve scope in the next line while favoring the return value — so that you would never have to say @user explicitly at all.

For example:


To my knowledge, no programming language does this. You can do stuff like that with method chaining:


And there was a with statement in Pascal and early versions of JavaScript that allowed something kind of remotely similar, but more explicit:

with(foo = 5) { console.log(foo); }

But it's not the same. I don't expect any programming language to ever really copy this aspect of Japanese, for the same reason that Google Translate basically sucks at Japanese, and I kind of expect that it always will: Japanese relies on context, i.e., information from outside the sentence, and writing code which can infer things from context, the way humans can, is not an easy thing to do. It's probably impossible to develop a programming language which properly emulates this aspect of Japanese without solving very hard problems in artificial intelligence and/or machine learning.

Update: I got a lot of responses to this about possible hacks to implement this, equivalents to with in other languages, and similar stuff. I plan to update with a new blog post semi-soonish but it's going to be after a slight delay, because some of the responses involve languages I'm not at all familiar with, so I want to investigate a little before I post the update.

Thursday, October 6, 2016

This Infographic Explains Why JS Dev Seems So Complicated

There's a web site called Hacker Noon which seems to specialize in complaining about JavaScript. One Hacker Noon piece resonated with a lot of people recently. It's all about how huge and overcomplicated the JS ecosystem is, and how hard it is to just sit down and get stuff done.

First of all, if what you want is a beautiful, omakase system for building modern front-end applications, just learn Elm. It's omakase as fuck. It's lovely to work with. And it's the type of project where you can do everything in one language, without ever needing to deal with other stuff.

But there's another option, too: just take a moment to understand why the JavaScript ecosystem is complicated.

It's not just because JavaScript is used by everybody who does anything on the web. It's not just because code on demand becomes a more powerful strategy each day, as the computers which run web browsers become smaller and more powerful.

It's also because JavaScript employs one of the oldest strategies of the Web: paving the cowpaths. Whenever people are already doing something, and it's working well, JavaScript seeks to lift up their practices into the language itself, rather than dictating something new.

Consequently, modern JavaScript is full of things which it got from open source projects.

Puppy picture from Daniel Stockman on Flickr.

bind was pioneered by Prototype, and showed up in Underscore and jQuery (as proxy) before landing in ES5. The Array methods each, map, and filter have a very similar backstory. CoffeeScript and Node are major contributors to ES6. When Angular and Ember first blew up, Object.observe briefly became a thing. People thought it would be a big deal, but this turned out to be premature paving of an unused cowpath, and Object.observe was deprecated, because people didn't use it. This is the output of a process where the people who shape JavaScript look at how open source projects use JavaScript, and the ways in which they seek to fix JavaScript, and use that as guidance to inform decisions about what JavaScript should be.

Modern front-end development is a lot of fun. In the 1990s and the 2000s, it was a nightmare. It was horrible. And paving the cowpaths is one of the reasons it got better. Because in the 1990s and 2000s, JavaScript implementers tried to just make up features and tell people what they wanted. It didn't work.

The chaos and churn of the JavaScript landscape is how we got a better JavaScript. It would never be practical to get such a big user base to agree on a hypothetical feature set. There's just too many people. It would be beyond impractical to expect any giganto-committee to be able to foresee the needs of such a colossal group. So JavaScript is a place where lots of projects get invented, and the language learns from its users.

That's fucking great. A huge landscape full of competing projects means that JavaScript will keep getting better.

If you'll forgive a little blatant self-promotion, this infographic is from my upcoming book, Modern Front-End Development with Ruby on Rails and Puppies. In this book, I build a simple UI in vanilla Rails, using CoffeeScript and jQuery. Then I throw out jQuery and turn the front-end code into ES5, because — as you can tell from the infographic — pretty much everything that made jQuery worthwhile has been integrated into JavaScript itself now. Anyway, then I turn it into ES6. Then I run it through Webpack and integrate Webpack with Rails. I implement the UI again in React with JSX, and then again with ClojureScript and Om. (I've also built it yet again in Elm, but I haven't written that chapter yet.) I even show how to pull in an experimental feature from ES8.

And yes, there's puppies. There's a puppy on every page. So if all this unfamiliar JavaScript scares you, all you have to do is look at the puppies and you'll feel better.

It's a very hand-hold-y book. It makes it easy for you to dive into all the complexity of modern front-end development. So there's no excuse for complaining! But the reason all this works is because this book teaches you things. In order to do front-end development, you have to learn stuff. There's no way around that. You can't expect one of the most lucrative fields on the planet to be a field which does not require you to obtain new information. That's just ridiculous.

Update: Version 0.1 0.2 of the book is now live! Also, when I made this infographic, I left out the most obvious and well-known example of JS adopting user ideas, namely, JSON. My bad!

Wednesday, October 5, 2016

Let The Asset Pipeline Die

日本じんRubyプログラマー、どうもありがとうございます! Rubyがだいすきです。日本語がよくわかりません。だから、これは推測です。すみません!
TLDR: The best front-end compilers come from the JavaScript world. Use words accurately. Eschew hate. Forget MINASWAN, because Ruby is 和. Learn Japanese.

don't save sprockets

The good folks at Arkency recently got in touch and asked if I wanted to include my book Rails As She Is Spoke in their Rails Book Bundle. I was delighted to say yes, and big shoutout to Arkency for that. It was a fun coincidence: I had actually been reviewing this book, with the goal of rewriting it, for a few different reasons.

First, Gary Bernhardt's Destroy All Software, in its first incarnation, included a ton of Rails content which I felt kinda put my book to shame, and I wanted to upgrade the whole book as a consequence. Second, I had realized that my book had accidentally downplayed the role of Kathy Sierra in the design of Rails. When Rails was new, Sierra's Creating Passionate Users blog was a centerpiece in the ongoing discussions about how to build the Web, not just in general among the Web community, but specifically on David Heinemeier Hansson's blog. DHH frequently linked to Sierra's blog, and frequently discussed how both Basecamp and Rails were founded on and/or in line with Sierra's thinking. But I left that out of my book, and when I re-read my own book, I realized that I had not only done Kathy Sierra a disservice there, but also made the story less coherent in the process.

Third, when I wrote the book, Rails was all the way back on Rails 3. It's now on version 5. So I also had some updating to do. In the original book, I had kind of skipped over the asset pipeline, and today it's one of the major hassles in Rails development, so I felt that this was a minor deficiency that had turned more serious. So I set out to find a definitive answer on that.

Which is how I ended up watching @schneems's talk Saving Sprockets. In this talk, @schneems explains why he's maintaining Sprockets, and why it's a lot of work. TLDR: the code base is a mess, it overuses modules, and ultimately, Sprockets is a single God object with 105 methods.

Saving Sprockets is an important talk. A lot of people need to see it. Programmers all over the world need to remember that open source is free; that filing a ticket means asking somebody else to do work and solve your problems for free; that open source maintainers do not owe you that free work and are not bad people for having lives beyond their GitHub Issues. I was really enjoying the talk, until there came a point when @schneems referred to Node.js as "the technology that shall not be named," and I nearly threw my computer out the fucking window.

rails hate against javascript is deranged

Rails was first created in 2004. Browsers were horrible in 2004. You still had to deal with IE6. You had quirks mode. At the time, "optimizing for programmer happiness" and "doing front-end development" were opposites. So Rails has a long history of creating adjunct technologies which enable you to build interactive web sites without ever having to touch a line of JavaScript.

In 2006, Rails introduced RJS, which allowed you to write Ruby which would generate JavaScript. It was a bad idea, and Rails retired it only one year later. Since then, Rails has introduced Sprockets and Turbolinks, and adopted CoffeeScript. Personally, of all these technologies, the only one I really love is the one that wasn't invented in-house.

But in 2006, at least, wanting to avoid JavaScript as much as Rails does was reasonable. Today it's like meeting somebody who thinks Czechoslovokia, which doesn't exist any more, is part of the USSR, which doesn't exist either. Browsers got better, the language got better, and you can now run it from the command line if you want. It's just a different world.
And indeed Czechoslovakia, when it did exist, was an independent country. Thanks to Jon Roesner for context on this.

Not only that, the Rails culture has had a problem with Node.js hate since Node.js was invented. For me personally, this has been a severe source of caremad since about 2011. For instance, here I am arguing with Avdi Grimm about it in 2012:

Avdi's prediction did not materialize, of course, but we'll get to that.

Ruby caremad is very durable for me. Rails itself might not even be the best tech for my needs any more, but Rails gathered a community around principles like beautiful code and programmer happiness. I think those principles are very important.

The thing that bothers me so much about this, however, is not that I love Node or love JavaScript. The thing that bothers me so much is that every time a Rubyist hates on JavaScript, they shit on all of the things that were said to me when I was first curious but doubtful about Ruby — that it's a polyglot community, that Ruby represents a synthesis of many great ideas from many different programming languages.

A little before I got into Ruby, I started going to Python meetups, but I stopped because I got sick of hearing them hate on Perl. It was constant; no Python meetup was complete without somebody saying something about how terrible Perl was. Perl is by no means a perfect language, but I'm just not the kind of person who likes to join groups who use hating another group as their key organizing principle. I consider that very unhealthy. I don't know if Python ever got over that shit, but I am very sad to see how sick the Rails culture has become.

So I started trying to figure out what went wrong, and I think I know.

Let's start by getting our terms right.

When people talk about Ruby programmers as a group of people, they usually say "the Ruby community." I often prefer Gary Bernhardt's term — "the Ruby culture" — but it's really a diaspora. Ruby originated in Japan, and most of the Ruby core team is Japanese to this day. Japanese Rubyists and Western Rubyists form distinct local cultures which share a common language and a common origin. That's a diaspora.

(Likewise, you might be tempted to say "the JavaScript community," but everybody uses JavaScript. It's like saying "the human beings who breathe community." It's too big to be a community or even a culture. It's a world.)

When programmers in the West discovered Ruby and told each other about it, they formed a distinct local culture of Western Rubyists. (Being Western, of course, they just called themselves Rubyists.) Every culture needs a story to tell about itself, and Western Rubyists were no different. But the story they came up with was so weak that the culture's current sickness was made inevitable.

The story was "Matz is nice, and so we are nice."

They abbreviate it MINASWAN.

ruby is 和

Just for the record, MINASWAN is at least half true. Matz is nice. I met him, and he's nice. I went to dinner with him, several other Rubyists, and the late, terrific, and much-missed James Golick at MountainWest RubyConf.

We were thrilled to get our Matz selfies.

But there are several problems with MINASAWN. It's simplistic hero worship, and it's so vague it inevitably leads to hypocrisy. "Be nice" is really general advice, and some of the people who say it don't actually do it.

I think there is a much, much better way to understand Ruby.

I've been studying Japanese for a few months, and one of the other students in my class brought in pictures of a Buddhist temple that he visited in Japan. One picture involved a Shinto shrine, even though Shinto and Buddhism are two completely different religions. This Buddhist temple simply included a space for Shinto worshippers, like if a church in America included a small corner that had been constructed as a tiny mosque. Our teacher told us that this is a characteristic of Japanese culture; where in America, you have a church over here and a mosque over there, in Japan, it's very common to find people reconciling or combining disparate elements in a harmonious blend.

This is what Ruby does. Ruby took disparate elements from Perl, Python, Lisp, and Smalltalk, and combined them in a harmonious blend.

I wasn't there when MINASWAN was first said. But what I imagine is that there were two people disagreeing with each other, and Matz, who is Japanese, looked at their points of view and sought to combine disparate elements from these two points of view in a harmonious blend. And all the American Ruby programmers looked at that and thought, "oh, he's nice." Like that was all there was to it.

In reality, this is a fundamental thing about Japanese culture. And Ruby comes from Japan. This is not a coincidence. The Japanese language even has a word for it: 和, which is roughly pronounced "wa."

It should hopefully be very obvious how hating on Node.js, or any language, is fundamentally irreconcilable with 和, and fundamentally inappropriate for any event or group of people organized around Ruby, if you regard Ruby as a product of 和.

I see Ruby that way. In fact I think we should go further, and say that Ruby is 和.

I hate to say this, but I believe this is a situation where we Western Rubyists are simply being very bad guests. Ruby was created in a spirit of 和. It should be used in a spirit of 和. It's just a matter of courtesy.

MINASWAN implies that Ruby on Rails does not exist

Now let's think about one of the major, glaringly obvious contradictions in MINASWAN.

By far the most famous Rubyist in the world is DHH. I would not call DHH nice. I think he wouldn't describe himself as nice either. In The Rails Doctrine, he describes Rails as "a deeply narcissistic endeavor." In 2006, at the first Rails conference ever, Canada on Rails in Vancouver — about a month before the first RailsConf — DHH gave a talk where he quoted some critics of Rails who had described him as arrogant, and he agreed with them.

A few years later, when someone else delivered a Rails conference presentation which contained porn in its slides, DHH defended them, saying this:
I've found that the fewer masks I try to wear, the better...

You're bound to upset, offend, or annoy people when you're not adding heavy layers of social sugarcoating...

What I'm not going to do, though, is apologize for any of these preferences and opinions...If you can deal with that, I'm sure we're going to get along just fine.
In other words, "I'm just being honest." The thing that only assholes say.

So, according to DHH, DHH is arrogant, doesn't mince words, and regards his own life's work as a narcissistic project. That's what this guy says about himself.

So if MINASWAN is really a basic truth about the Ruby culture, then how does DHH fit in at all? MINASWAN would predict that DHH was not involved in Ruby, since "we are nice" is a statement which probably cannot be made about a group of people which includes DHH. But MINASWAN's prediction is false.

DHH isn't just involved in Ruby, and he isn't just a very important figure in Ruby. He's also a huge Ruby fanboy. Half of The Rails Doctrine is just all about how much DHH loves Ruby. He wrote a long essay about the ideas behind his massively successful web framework, and despite it being a narcissistic project by his own admission, half of this essay just reads like a love letter to a programming language. And it's probably not a coincidence that his Ruby reads better than most people's Ruby.

Rails also fundamentally embraces 和. Think back to the example of a Shinto shrine within a Buddhist temple. Another way to describe 和 could be as a spirit of inclusive compromise. DHH famously hates RSpec — and I've recently come to understand his point of view — but you can not only use RSpec with Rails, rails new has a built-in --skip-test option which allows you to use RSpec more easily. There's a --skip-whatever option for nearly everything DHH thinks you should use in your Rails app; consider also the "no one paradigm" section of The Rails Doctrine.

Ever since DHH came along and transformed Ruby, in the West, from an esoteric hobby into a thing you could actually do for a living, MINASWAN's been this weird, illogical thing about pretending DHH doesn't exist. Which is already a weird red flag. But if you say "how can we carry forward Ruby's spirit of 和?" then DHH poses one answer. In his communication style, he's Western to a fault, individualistic and utterly uninterested in getting along with anybody, but when it comes to naming variables and methods, he's obsessed with balance and harmony. In other words, he chooses 和 when he's writing code, and he chooses a Western individualism when he's arguing on the Internet.

(And, sadly, when he's architecting systems, which is the core problem with the asset pipeline.)

DHH's answer might not be the best answer. It's certainly not the only answer. But it's an answer. MINASWAN implies that DHH just doesn't know what Ruby is about, as a culture, and that's a ridiculous position. It's the (Western) Ruby old school sticking its fingers in its ears and yelling "la la la I can't hear that Danish guy."

If we say "Ruby is 和," then we can acknowledge the obvious reality that DHH understands Ruby pretty well. You can still disagree with what he's chosen as the balance of Western individualism and Japanese 和, but that's fine — speaking as a Westerner myself, I feel like that's his decision to make — and you now have a sane, coherent, rational model of the Ruby culture. The model for itself that this culture has chosen instead, MINASWAN, implies absurdities. You can't structure a conversation around something like that. MINASWAN was simple enough when Ruby was a tiny scene of hobby programmers only, but when DHH made it a serious professional thing, MINASWAN should have evolved into something more nuanced. Instead, some Rubyists forgot about it, others never even heard of it, and others took it as a joke because it ignored DHH.

Still others employed it as a sort of snooty, superior, tragic thing. That's the old school Rubyists who tried to use MINASWAN to inhabit the moral high ground. But MINASWAN fails as an attempt to occupy the moral high ground because it implies DHH has no "real" place in Ruby. He's literally the only reason most of you have jobs! So it's not only terribly entitled and ungrateful, it's also passive-aggressive, since nobody ever addresses this huge contradiction that DHH somehow is just a footnote. But if you're passive-aggressive, entitled, and ungrateful, then you're not nice.

MINASWAN is garbage. It'd be more accurate to say, "Ruby showcases the Japanese value of 和, but we are arrogant Americans, so we reduce this to a really basic American idea, harshly compressing it in the process to a state where it cannot possibly mean anything any more, instead of bothering to learn something about the outside world for once." But MINASWAN was already a long acronym, so I guess they had to draw the line at RSTJVO和BWAAASWRTTARBAIHCIITPTASWICPMAAMIOBTLSATOWFO.

Instead of advocating this acronym myself, I have a simpler suggestion: shitcan MINASWAN, and say instead "Ruby is 和." That's pronounced, "Ruby is wa." Now you may be asking yourself, 和t the fuck? It's not enough that newbies have to learn OOP and some FP concepts, now we have to tell people that if they want to understand Ruby, they have to learn Japanese?

Well, every Japanese programmer has to learn English to program in Ruby, which was invented in Japan, so I don't really think that's as demanding as it might sound. I think it's actually pretty reasonable to ask that programmers be good at languages. But I'm not asking you to learn a whole language. I'm literally saying you should just learn this one word. It's a really important word.

Ruby's artful blend of disparate paradigms is not a fluke; it's just the most recent step in a tradition which stretches back at least a thousand years. This blending already has a name, and if you want to reason about it — say, for example, you're writing a new language, which aims to achieve a similar balance, as Jeremy Ashkenas did when he created CoffeeScript — then it makes sense to refer to it by name.

Alternatively, I suppose you could just say "Giles felt like his ideas were too obvious, so he started blogging in Japanese." But that would miss the point, because プログラマーはトランスレーターです.
"A programmer is a translator." In other words, translation is inherent to programming.


Around the same time I watched the Saving Sprockets presentation, I saw this (shorter) video too:

Take a second to watch it. It's just fantastic.

I also saw this remark on GitHub:

Since illuminating this murk was a major goal for my Rails As She Is Spoke rewrite, I dug into this some more, and found a great blog post about migrating from Sprockets to Webpack. It said:
I couldn't have imagined tools like Webpack or Gulp existing a few years ago, but today, Javascript asset packagers are becoming increasingly advanced and sophisticated. It seems to me that Sprockets will have a real tough run for its money in the very near future.
I agree.

This is the answer to all the confusion around the asset pipeline: don't fucking use it. Sprockets is poorly designed abandonware which causes endless problems. This is an easy choice.

The JavaScript world has a terrible and well-deserved reputation for changing too fast, but if Rails moves away from Sprockets and embraces Webpack, it'll be the first major change in the design of the asset pipeline since 2009. I think you can change your asset pipeline in fundamental ways, nearly eight years later, without deserving any criticism for moving too fast.

Plus, according to @searls, Ruby is still the best way to test browsers on a Node.js project:

Maybe it would be easier to sell Node programmers on a polyglot approach here if Rails itself hadn't turned its back on polyglot programming, aka Ruby's spirit of 和. Here's one way you might sell Node programmers on using a Ruby project: "hey, we talk shit about you all the time, but our Selenium wrapper is better than your Selenium wrapper, so suck a dick, dumbshits." Here's another way: "hey, we use a Node front-end compiler, because we're an open-minded, polyglot community, and we have this really good browser testing API, so why not be polyglots too?"
@searls showing up twice like this doesn't imply an endorsement from him, it just shows that he's freaking everywhere.

Also, Webpack is amazing. If you've read Ilya Grigorik's book on front-end performance — and frankly, if you have any opinion at all about Sprockets, then you should have read this book already — Webpack is a dream.

To dig up the Sprockets logo, I had to go to a 2009 blog post, because today, the Sprockets web site is a GoDaddy domain landing page.

Webpack doesn't just compile CoffeeScript, bundle up JavaScript files, gzip everything, and set up fingerprinting, aka cache-busting, like the asset pipeline does. Webpack can also embed CSS files in JavaScript — along with SVG, JPG, GIF, PNG, and even MP3 files — which allows you to reduce the number of network requests that your front-end code is making. Both Sprockets and Webpack can turn images into data URIs, but Webpack can transpile many other languages to JavaScript, not just CoffeeScript, and using Webpack means your front-end code can use the Node.js require() functionality which JavaScript proper still doesn't really support, at least not in every browser. So you can replace the hacky require_tree directives in the comments of your application.js with actual require() semantics.

Both Webpack and Sprockets have modular APIs which allow third parties to write plugins, but there are important differences. First, Webpack works more reliably than Sprockets, and its API doesn't flummox third-party contributors (or indeed its core team) the way the Sprockets API does. Second, as a technology favored by JavaScripters, Webpack gets more relevant third-party contributions than Sprockets does, it gets them faster, and they come from people who understand the front end better.

Community is probably the most important part of this. As long as you're using a JavaScript front-end compiler instead of a Ruby one, it doesn't even matter that much which one you use instead. I prefer Webpack, but there are strong arguments for Rollup, Browserify, and even Bower as well.

Node is like Ruby used to be back in the day, i.e., a new new hotness every three months, so you should find the balance that works best for you, but all of the serious JavaScript options are better for this than anything Ruby has ever produced, or ever could produce.

Webpack 2 has tree-shaking, and the Webpack team is looking for ways to integrate Rollup-style scope-hoisting as well. Could this ever happen with Sprockets? Can Sprockets solve CSS's namespace problems? How many refactors until this legacy code acquires an API clean enough to support third-party sunburst graphs? And how soon is Sprockets going to get its real-time dashboard?

The answer is either never or not any time soon. It's going to take a heroic amount of work just for Sprockets 4 to become more than one object. But despite a culture packed to the gills with OO gurus, Rails does not have a good track record with refactoring — remember Rails 3? — and even in the best-case scenario, Sprockets 4 is not going to deliver tree-shaking or scope-hoisting ever. It's just never going to happen. You need great JavaScript parsing for these features, and Ruby's JavaScript parsing libraries are not great.

I know this first-hand. I had this crazy idea to build a Ruby project which could automatically refactor JavaScript. Basically, a front-end compiler, except I hadn't even thought of compressing JavaScript. I just wanted to build a system which could automatically turn ugly code into more readable code. And I did that, sort of, a tiny bit. I built an MVP, really just a proof of concept. It was not easy, in fact it was excruciatingly painful, and the Ruby libraries which were available for the task were not great, or even good. The experience was so awful that only two things can possibly explain it: sheer stubbornness, and the ludicrous idea, which I believed at the time, that JavaScript parsers, compilers, and insane automatic refactoring experiments should all be written in Ruby.

In reality, the best JavaScript parsers are written in C or JavaScript. (And although Ruby has arguably good integration with C, the best JavaScript parsers for C are typically hidden inside JavaScript engines like V8, and tied too closely to those engines to be usable.) In general, it's reasonable to assume that the people who care the most about JavaScript and understand JavaScript the best are going to write the best JavaScript compilers and parsers, and further, that they will do so in JavaScript.

The first Google search result for "best css parser" also just happens to be written in the language that front-end coders use the most. I haven't done any CSS parsing, so I can't vouch for it. I just wanted to shock you with an astonishing coincidence. Because the intuitive, rational thing to assume, which I'm sure you assumed, would be that the best CSS parser was written in Ruby, right? This is what any rational person would assume. Yet for some inscrutable reason, it appears the best CSS parser's either written in JavaScript or C++. I guess this inexplicable phenomenon will remain an unsolved mystery until the end of time.

Unless, of course, you happen to believe that Ruby is a beautiful language which is good at some things but bad at others, in which case you might conclude that it's maybe not reasonable to be writing compilers in Ruby for anything other than a teaching example.

ok, save sprockets a little

At this point in the blog post, you might be expecting a caffeine-induced, fist-shaking rant, foaming at the mouth with impotent rage, but that's not what comes next. First I did a bunch of passive research, i.e., reading blog posts and watching videos. Then I did some active research, i.e., building stuff. Then I did a little bit of work on a few Sprockets tickets. We're not talking about a ton of work here. I answered a few questions, investigated a few bugs, and created a few example apps (which @schneems had requested, in the Saving Sprockets presentation).

But I did these things. I made a contribution of effort to Sprockets. Because Ruby is 和.

Although the anti-Node.js bigotry in Saving Sprockets annoyed me, and although I disagree with the fundamental premise — that Sprockets is worth saving — the presentation was a great presentation. The work of open source volunteers has made my life better, which is the real point of Saving Sprockets, and I feel grateful for that, partly because @schneems convinced me in his presentation that I should feel grateful for that.

Remember, one of my theories re 和 is that it explains DHH way better than MINASWAN can. I think this theory is obviously correct and nobody but an utter dipshit could possibly contest it. And that's something DHH himself might say, in his Western, non-和 mode. And my idea here is that his Ruby code, and his many writings about how to write Ruby code, display a very classically Rubyist 和. And that this is the balance he strikes between the two cultural currents in the Ruby diaspora.

His balance isn't the only balance. I decided to strike a different balance. Although I think Sprockets is the wrong direction overall, I decided I would show a little gratitude and respect for all the free hard work that other people have done on Rails, and Webpack, and so many other technologies, because it's made it possible for me to do stuff. Even though I think that Sprockets's time is ending, that piece of technology's been something useful which I got for free since 2009. @schneems said in his presentation that we should help the projects that have helped us, and that all programmers should do that, and I agree. So I contributed a tiny bit to this project even though I ultimately believe it should be wound down and retired. I feel this is 和.

When I cloned Sprockets, I even named my directory ~/code/和, but this caused 44 spec fails, all with error messages like URI::InvalidURIError: URI must be ascii only, because Sprockets builds its testing URLs from your directory name. I didn't file a ticket for this; I felt it was an unusual and inconsequential edge case. I just created another directory with a more run-of-the-mill name.

But I kind of hope @schneems uses his powers of persuasion in the service of a more current project next time. Because how on earth can Rails say it exalts beautiful code and optimizes for programmer happiness when it's keeping really bad legacy code alive on the spare time of volunteers?

It kinda feels like watching a bunch of mean teenagers telling lies to a younger kid. "Fix this legacy code, dude! You'll be a hero!" You might as well tell him that the Spanish word for "friend" is "pendejo" while you're at it. ("Pendejo" literally means "pubic hair," and if you use it as a term of address, it's not taken as a compliment.)

Imagine an alternate universe in which, instead of seeing Ruby programmers as expendable components we can burn out in order to extend the lifespan of terrible Basecamp side projects from 2009, we saw the evolution of Node.js as a way to expand the Ruby community. We used to need gem wrappers in order to get package management for our "JavaScripts" — app/assets/javascripts still uses this increasingly bizarre plural — but now we have npm. We used to need a bad asset pipeline made of bad code, but now we have a ton of excellent options to choose from. Rails was ahead of the curve; all you need to do now is be grateful that the JavaScript world finally caught up with you.

The whole reason JavaScript has so many front-end compilers is because Jo Liss wanted to have the Rails asset pipeline, but for JavaScript. So she built it, and she called it Broccoli. Other people liked the idea, saw their own ways to do it, and so, many other projects followed after. Webpack is part of a very long lineage that originates with Broccoli, and thus with Rails, and therefore with Sprockets. If you can't give Node.js credit for expanding on a Rails idea, that's just a cultural sickness. To use Sprockets in 2016 is to deny the profound influence it had back in the day.

(Although, to be fair, a lot of these projects were inspired by the Google Closure Compiler, too.)

The crazy irony of it is that the JavaScript world needs omakase so badly. (That's what's so great about Elm.) So many people try to jump into modern front-end code and just give up because it's got none of the luxuries Rails has, which ease the learning curve and keep you focused on building apps instead of duct-taping infrastructure together. There are starter/boilerplate projects, of course, which aim to serve a purpose similar to rails new or scaffolding, but it's not the same. A couple gems integrate Webpack and Rails, which is a step in the right direction, but really, gem wrappers need to die, preferably in a fire so hot and intense they melt to their component atoms. Imagine if Rails presented a nice, clean wrapper around Webpack, or something like it, in a uniquely calm corner of, which people could trust and consider authoritative. If Rails did that, instead of clinging to its legacy code, it'd be a tremendous public service.

Somehow, the Rails culture looked at this landscape and instead saw an opportunity to get talented people to mire themselves in legacy code for free. In my opinion, that's dysfunctional, and it won't end well. And it's not the only dysfunction involved.

Hating on Node.js might just be evil sexist bullshit

I hesitated before posting this rant, because of a tweet from Wes Bos:

And especially a blog post by Aurynn Shaw:
when I started programming in 2001, it was du jour in the communities I participated in to be highly critical of other languages. Other languages sucked, the people using them were losers or stupid, if they would just use a real language, such as the one we used, everything would just be better...

This sort of culturally-encoded language was really prevalent around condemning PHP and Java. Developers in these languages were actively referred to as less competent than developers in the other, more blessed languages.

And at the time, as a new developer, I internalised this pretty heavily...

[but] I was asked to consider who and what I was criticising... starting with Wordpress-based design backgrounds and moving from more simple themes to more complex themes where PHP knowledge is required, to plugin development is a completely valid narrative, but a path that is predominately for women.
With apologies for the pedantry, I'm fairly certain Ms. Shaw is using French incorrectly here, and that the phrase she wants is de rigeur rather than du jour. Du jour can mean trendy, which would imply that programmer tribalism was a thing of the past, which I would disagree with, although I would also wish it were true. De rigeur means something required by etiquette, which I think is what Ms. Shaw intended to convey.

So first, yes, OK, the asset pipeline works. I think a company which wants performance, and an asset pipeline that is easy to reason about, will have less trouble if they --skip-sprockets and install Webpack instead, but if you're learning Rails at a code bootcamp, by all means, Sprockets will get the job done.

(Also, Webpack is easy to use with Rails, and I've literally written a book which will show you how to do it, but we'll get to that.)

Second, I'm pretty damn sure @schneems had no sexist intent. He didn't even seem to hate Node.js himself, he seemed like he was apologizing for acknowledging the existence of Node.js, because other people hated it. That's not necessarily on him, that's on the audience. And I've already discussed why this alarms me.

But consider what Ms. Shaw said about the WordPress -> PHP path being a path which brings a lot of women into programming. Is it possible that a lot of women come into programming via front-end work, which in Silicon Valley's parlance is often considered nothing more than "making the app pretty"? Who might be tasked with the work of making things pretty?

My guess is that JavaScript has more women in it than Ruby. And which programming language community had a big fight over gendered pronouns? Which community banned Douglas Crockford from an event for allegedly sexist remarks and allegedly making women uncomfortable? Which community's package manager is allegedly overrun with so-called SJWs?
This has a link to the awful KotakuInAction. I tried to use, but it died? If you're up on the latest move to make re this, please LMK.

Every time a Rubyist distances themselves from Node.js, they're distancing themselves from a community where feminists play a more prominent role than they do in Ruby. I'm sure this is unintentional in some cases, but I'm equally sure that it's intentional in others. I don't know how you would calculate the ratio of intentional vs unintentional, but I very much doubt that intent matters as much as the practical outcome anyway. If I'm right about JavaScript having more women than Ruby, then every time a Rubyist sneers at JavaScript, they're not just building a wall around Ruby which keeps JavaScript programmers out, but also a wall which keeps women out.

It's obvious that "Matz is nice and so we are sneering a lot" just plain does not make sense, but before I read Ms. Shaw's blog post, I assumed the cognitive dissonance was based on insecurity. I assumed it was about Rubyists being mad that they weren't the hot new flavor any more.

This happened a long time ago, and I thought it was the main reason Rubyists hate on Node so much:

But now I have to consider an interpretation which paints the Ruby culture as more discriminatory than I had imagined it to be.

And by the way, that too has roots in Japan, although, again, not necessarily in a malicious sense.

why ruby has no code of conduct

Ruby's had a very difficult time establishing a code of conduct, and Ruby core's Japanese contingent have been staunch in their opposition to it. There might be some sexism to that, and there might not. But there's no ambiguity on one crucial point: 和 emphasizes working out differences via compromise, while codes of conduct tend to set relatively stringent rules, and use ostracism and exile as modes of punishment.

This is a sensible strategy in the West. Exile is a mild punishment in an individualistic culture — more a safeguarding mechanism than a punishment at all, really — but a harsh one in a culture like Japan's. One of the more important aspects of 和 is that you're supposed to work for the good of the group, not just yourself. Imagine that's how you're raised, how everyone around you was raised. If you get exiled, what do you even do with your life? In a culture where children are taught that helping your group is the whole point of doing things, ostracism is not a mild punishment. It means no longer having any reason to do things, or to exist.

和 is so fundamental to Japanese culture that 和 was the Japanese word for Japan about a thousand years ago, and still functions as an adjective meaning "Japanese" in many older words. As Michael Carr wrote in The International Journal of Lexicography, "[t]he notion that Japanese culture is based upon wa 和 'harmony' has become an article of faith among Japanese and Japanologists."

Unsurprisingly, Japanese members of the ruby-core mailing list, Matz included, were especially opposed to the ostracism aspect of a CoC. If you understand 和, the Japanese Rubyists' response to a CoC not only makes more sense, but sounds almost aghast at American barbarism.

Matz said, for instance:
The CoC contains banning members from the community as a punishment. This does not mean anything but hurting individuals... Besides that one can regret the previous act and change the attitude.
Matz's implied solution — atonement — is virtually never seen in discussions of code of conduct. It's not a coincidence that in Japanese, there are at least 11 different ways to say "I'm sorry". Let me tell you how that looks to me personally. I'm a first-generation American with roots in England — which, like Japan, is a formerly imperialist island nation, which has notoriously idiosyncratic etiquette, which is fond of compromise, and where the average resident apologizes constantly, frequently for things that are not even their fault. From my point of view, there's an evangelical, absolutist zeal to codes of conduct which I find distinctively American.

I support CoCs, in general, but I sometimes find it unsettling how certain CoC advocates are of their own righteousness. Likewise, perhaps again as a quasi-English person, the weirdest thing about CoCs to me is that virtually nobody ever even raises the possibility of apologizing, atoning, or learning anything. (Sorry to lean so hard on my tenuous claim to Britishness, I realize it's a bit disingenuous.)

Of course, most CoC discussions involve communities which are predominantly American, so this perspective might not even matter, overall. But the Ruby culture is a diaspora, and MINASWAN isn't worth shit if you're using it to decipher the drama around the failed attempt to adopt a Ruby Code of Conduct. "Matz is nice and so he is opposed to requiring that Rubyists be nice to each other" makes no goddamn sense at all, whereas "Ruby embodies the Japanese principle of harmonious balance, aka 和, and convincing Japanese Rubyists to abandon that spirit of inclusive compromise is exceedingly difficult" makes perfect sense. Let's say MINASWAN served its purpose, but it's time for a more nuanced point of view. Ruby is 和.

btw, buy my stuff

This blog post started out as a new chapter for the Rails As She Is Spoke sequel/revamp that I have planned, but along the way, I decided to write a new book entirely. It's a book about bringing modern front-end code to Rails applications. The TLDR will probably be "use Webpack, use Elm," but (as with many things) it's the journey, not the destination. The book starts out with a very simple, run-of-the-mill Rails app, with a front end using CoffeeScript and jQuery. I then carefully rebuild the front end for this app several times — in ES5, ES6, React, and ClojureScript (using Om) — to demonstrate modern front-end development and explore the tradeoffs that you have to consider when doing it. I also show you how to replace the asset pipeline with Webpack. These chapters are all written; I've also got a chapter on Elm which I'm still working on. (The code is written, but not the prose.) The book's working title is Modern Front-End Development with Ruby on Rails and Puppies, because there's going to be a puppy on every page.

btw, if you're like, "wait, Giles accused the entire Ruby community of sexism to sell a book," well, sure. I'm not above picking a fight now and then. Which reminds me: @schneems absolutely deserves his recognition as a Ruby Hero, but where's my recognition as a Ruby Villain? I know I'm not Lex Luthor, but I figure I'm at least that gorilla with a light bulb on his head. I don't even need an award ceremony, just throw me a t-shirt or something.

Anyway, my underrated villainy is actually kind of relevant to this discussion, because DHH isn't above picking fights either, and anybody who thinks otherwise is living in a dream world. Consider what Getting Real, a book DHH co-wrote, has to say on the subject:
Pick a fight

Sometimes the best way to know what your app should be is to know what it shouldn't be. Figure out your app's enemy and you'll shine a light on where you need to go....

One bonus you get from having an enemy is a very clear marketing message. People are stoked by conflict. And they also understand a product by comparing it to others. With a chosen enemy, you're feeding people a story they want to hear. Not only will they understand your product better and faster, they'll take sides. And that's a sure-fire way to get attention and ignite passion.
This could have been titled "The Troll's Guide To Marketing." Obviously, this is DHH in a Western mode. "Pick a fight" is not 和, but it's not MINASWAN either, is it? MINASWAN wants us to agree that Matz is nice, which is easy, but it also wants us to pretend DHH doesn't troll people all the time, which is ridiculous. The Rails Code of Conduct defines trolling as unacceptable behavior, which means Rails has to ban DHH from Rails, and Tenderlove as well! So the bad news is I guess I'm banned, but the good news is I'll be in excellent company.

I hate to provide MRAs with fuel for their arguments, but is there anything more ridiculous than the fact that the Rails Code of Conduct bans trolling? There are really only two elements to the Rails culture: trolling, and the color red.

One problem is that Internet generations are much shorter than regular generations, and "trolling" today means death threats, doxing, and harassment, while many Rubyists come from an Internet generation where "trolling" meant playful sarcasm like this:

But perhaps a much deeper problem is that sometimes people adopt a Code of Conduct in the spirit of lip service, and don't care too much about this kind of contradiction. Maybe they should.

Moving on, just because I'm saying it to sell a book doesn't mean it isn't true. In general, I like my books to contain true statements. I'd even go as far as to say that if you want to sell a book, telling people things for free is a good way to start, and those things you're saying for free should be true things.

So here are some true things: Rubyist contempt for Node.js might have plenty of non-sexist intent, but it also has a sexist effect — and it's also just impractical. You need front-end compilers. Sprockets is not a good front-end compiler. Webpack is. So are Rollup, Browserify, Google's Closure Compiler, and many other tools from the JavaScript world. Node.js in particular has much more overlap between systems programming and front-end work than the Ruby culture, by far, which is kind of what you should be looking for if you need somebody to write a front-end compiler, and it is so fucking weird that anybody would need to say that in the first place.

I saw my first Haskell talk at a Ruby conference and I saw my first Clojure talk at a Ruby conference too. Western Rubyists once had a magpie subculture, with lots of interest in other languages, and Ruby itself being a blend of many different ideas and influences. This is 和.

Imagine you were so foolish, as I once was, as to believe that the spirit of 和, which made Ruby what it is, was still alive and well over here in the West. When a Ruby project needed a front-end compiler, you would expect it to look at all the different ideas out there and pick a blend of the best options. And you'd expect at least some of those best options to come from the JavaScript world. Because the JS world is always going to produce better front-end compilers, and using one written in Ruby instead would already be a little eccentric even if that Ruby were beautiful and well-factored. But it is neither! Sprockets is one object with 105 methods!

What the fuck? No. Just no. Use the best tool for the job, and acknowledge the obvious fact that the best tool for the job might be written in another language. Get over this ridiculous Hatfields and McCoys bullshit. Otherwise, if we don't put a stop to this shit, the Rails culture is going to turn into an asshat pipeline.

We all know Rails is omakase.

And sure, omakase is all well and good, but here's a very related Japanese word: 旬, which I think is spelled しゅん. (My Japanese spelling is frequently wrong.) It's transliterated "shun" but it sort of rhymes with "moon." Its meaning is sort of like "freshest," except it also implies "most seasonal." Like you wouldn't serve a winter salad in July, no matter how fresh the ingredients were, because even if they were fresh, they couldn't be しゅん.

Sprockets isn't fresh, but it could be; @schneems is trying to make it fresh again, and even though I think it's a mistaken goal, he's having some success with it, and I respect that work. But today, Sprockets cannot be しゅん. That's just not possible any more. Whether we save it or not, its season has passed. When Sprockets was first written, I think only the Google Closure Compiler existed as a serious alternative. It couldn't handle CoffeeScript or CSS, and I don't think you could run it from the command line either (or if you could, you had to use Java, so it was painfully slow at best). Sprockets was a great innovation at the time, but that time is over. And if something is neither fresh nor しゅん, it doesn't belong on an omakase menu.

Still, I'm not actually proposing a change to the omakase stack. Basecamp appears to be developing NIH in its old age, but the omakase menu is up to the chef.

Let's just think about the real stack, the stack that every Rails app except Basecamp is using. The omakase concept might be arrogant, but it's not tyrannical. That --skip-sprockets option is right there, and the smart move is to use it.

And by the way, my new book shows you how to set up Webpack where Sprockets used to be.

Let's talk about this book

Here are a bunch of old tweets where people said nice things about my first book:

I've been doing this for a while, but recently, I decided to upgrade the design of my products. So I've created a new home for my books: is a new brand, and its goal is to signify a new standard. For example, in the past, I put no energy into design at all. Here's some screenshots from my old books:

And here are some design sketches for my new book:

In addition to a higher standard of design, this new book is longer than my other books, has more puppies than my other books, and ships with two git repos.

One repo is a quick overview of Flexbox, which simplifies a complex topic, while the other is the app I mentioned earlier, which teaches you all about the modern front end. It's a simple Rails app, with the same simple UI recreated in several different front-end technologies, including ES5, ES6, React with JSX, Webpack integration, ClojureScript and Om, and even a sprinkling of ES8 experimental features. (And with an Elm branch on the way quite soon.)

I also build a checklist of elements to consider when assessing the tradeoffs between each of these different front-end strategies, and run these different implementations through this checklist. The purpose is to make it easy for people to pick up new front-end technologies and evaluate them, because people who are new to front-end coding often have difficulty with that. This book aims to make that a lot easier.

Modern Front-End Development with Ruby on Rails and Puppies goes on sale soon, so sign up for my email list to learn more.

Update: early release version is on sale!

でも、この文を無視してください。私は外人がGoogle Translateに行きたいです。すみません!