What to do when you hit the system limit for number of file watchers

The emotion for this blog is frightening.

TL;DR (click here for solution below)

Basil Mohamed Gohar. A father of five. Working late as is not uncommon for him. He started out innocently enough just debugging a theme for a client. But he is about to discover that his seemingly innocuous actions dabbling with npm, webpack, and a Bootstrap admin theme will take him beyond the limits set forth by the Kernel Masters and into…the Hacker Zone.

Twilight Zone-esque narrator

My client requested the addition of a new theme to a product I am developing for him, and upon settling on the one he wanted, I got to work trying to integrate it into the Laravel app I was building for him.

I ran into a few issues related to how the Sass files referenced some relative paths, but after fixing those up, I got the whole thing to build (I am referring to compiling the assets with webpack). For those unfamiliar as I was only about a year or so ago, the tradeoff for today’s modern, desktop-like web applications with spiffy visuals, high interactivity, and more is a complicated build process that aggregates, minifies, and optimize web assets such as Javascript and CSS files. This, however, it also the reason that despite the fact that computers, network speeds, and browsers have all gotten orders of magnitude faster, most “modern” websites load slower than ever before. This is deserving of a fuller rant later, but let’s just say I’ve given up the good fight of hand-writing my Javascript and CSS in the interests of actually getting something done on a reasonable schedule for my client, because, let’s face it, in the end, it’s an uphill battle I don’t have the stamina to fight right now.

Having said that, I was…ahem…pleasantly surprised to see once I cleared the area of the asset compiling failing, my favorite npm run watch command, upon actually successfully building the asset pipeline and reporting success, died. This was a first for me in that, barring build errors, the watch subroutine always seemed to work flawlessly (and Laravel Mix by Jeffrey Way always gives a nice little desktop notification indicating it had succeeded by default).

The way Laravel Mix works is by watching the entire set of files that can affect your asset pipeline for changes. On a Linux system, the mechanism for doing this is through inotify. In a very simple way, Linux allows a user process to listen to changes to files. While immensely useful in many cases, it doesn’t take long for us to imagine that allowing this in an unfettered manner would easily overwhelm a system trying to track changes and report that to the listening applications.

The specific error I got, after filtering out all the stack tracing gibberish, was:

System limit for number of file watchers reached

Seems like the addition of the theme and its own dependencies pushed my web application past the system’s limit for watchers. Well, that definitely seems like Googleable material! Off to the tubes!

In my case, it seems the default limit for my system was 8192. The best resource I found was a Github wiki entry that succinctly explains how to solve this issue for users of their own software:

$ echo fs.inotify.max_user_watches=524288 | sudo tee -a /etc/sysctl.conf
$ sudo sysctl -p

Running these two lines solved the problem for me. Time will tell if any further danger lies in…the Hacker Zone!

Epilogue: This was written at around 2AM so please take the terrible style and jokes into account.

Update: Added a little explanation as to why this limit was even hit in the first place.

Update 2: Link to Hacker News discussion.