A single Node of failure

The web-development community was briefly thrown into chaos in late March when a lone Node.js developer suddenly unpublished a short but widely used package from the Node Package Manager (npm) repository. The events leading up to that developer's withdrawal are controversial in their own right, but the chaotic effects raise even more serious questions for the Node.js and npm user communities.

npm itself is a module repository for Node.js code, akin to the Python Package Index or similar repositories for other languages and frameworks. Users can install a package with a simple npm install foo, but the service is also widely used by Node.js developers to automatically fetch and install dependencies: projects list their dependencies in the package.json file, and they are recursively fetched from npm and installed when the package is built. Using npm in this manner is standard operating procedure, allowing complex JavaScript applications to be written on top of multiple third-party frameworks in minimal lines of code. The service, however, is run by a private company called npm, Inc., rather than by the Node.js project.

Kik starter

The trouble began with Azer Koçulu's module kik, a command-line tool designed to ask a few simple questions and set up a new Node.js project template based on the answers. According to Koçulu, a lawyer representing Kik Interactive (makers of the Kik instant-messaging app) contacted him by email and insisted he change the name of the package. When he refused, the company took its demand to npm, Inc. instead, copying Koçulu on the emails. npm founder Isaac Z. Schlueter then took ownership of the kik module on npm from Koçulu and unpublished it, without Koçulu's consent.

That action, it seems, frustrated Koçulu to the point where he decided to pull all of his other packages from npm as well; in his post on the subject, he commented that the situation "made me realize NPM is someone’s private land where corporate is more powerful than the people" and "NPM is no longer a place that I’ll share my open source work at". He concluded by apologizing for any effect the move might have downstream:

The story might have ended there, except that Koçulu was the owner of quite a few modules on npm. And, as luck would have it, one in particular, called left-pad, was part of the dependency chain for a host of high-profile Node.js projects and frameworks—including React.js, Babel, and Ember.js. When left-pad disappeared from npm, it caused a chain reaction of dependency failures that broke scores of JavaScript project builds.

Although no major interruptions appear to have hit live services, the widespread breakage of automated builds caused considerable panic; Daphne Maddox now famously described the impact as "This kind of just broke the internet." Reacting to the outcry, npm's Laurie Voss restored the unpublished left-pad module to the repository. On Twitter, he commented that "Un-un-publishing is an unprecedented action that we're taking given the severity and widespread nature of breakage, and isn't done lightly." Later, he said that Koçulu's decision to pull left-pad from the repository "puts the wider interests of the community of npm users at odds with the wishes of one author" and that the restoration was made because "I cannot see hundreds of builds failing every second and not fix it."

npm later posted its account of the events, citing the service's package name dispute resolution policy as the rationale for removing Koçulu as the owner of the kik module name. It also defended the re-publishing of Koçulu's left-pad module, noting that most of the downstream builds inherited their left-pad dependency through the line-numbers package, which was hard-coded to depend on left-pad 0.0.3. Even though another developer (Cameron Westland) created a new left-pad module shortly after the original was pulled, the new module was numbered version 1.0.0, which did not fix most of the dependencies.

For its part, Kik also posted its side of the story, quoting from the emails exchanged between Koçulu and the company representative, who it notes is the company's patent agent and not a lawyer. The post expresses regret for the unfolding drama:

Interestingly enough, Kik's representative evidently told Koçulu that the company would "have no choice" but to take down his accounts "because you have to enforce trademarks or you lose them." That is a common enough refrain, although as attorney Pamela Chestek recently pointed out, the "defend it or lose it" idea is not actually part of US trademark law. How one interprets the email exchanges between Kik and Koçulu will, no doubt, vary. It does seem, however, that Kik sought a compromise of some sort; Koçulu quoted a buyout price, which Kik did not accept.

Although many who commented on the string of events were clearly upset at Koçulu, he had his share of supporters as well, in light of the naming dispute with Kik. Tom McGee, for instance, wrote on GitHub: "Forget my broken build, stick it to the man!!" The past week has also seen a flurry of new protest projects appear on npm, using the name kik or otherwise referencing left-pad.

Left behind

As to the widespread cascade of disruption caused by the missing left-pad module, the community voiced several concerns about how npm is managed. First, providing the ability to unpublish a module is controversial. In its blog post, npm cited "historical reasons" for retaining the feature, but many other package repositories simply do not make it possible to remove an existing release on which other code depends. Mozilla's E. Dunham, for instance, noted that the Crates.io repository used by Rust projects allows developers to "yank" a release, which prevents it from being added to new builds, but does not break existing dependency relationships.

Furthermore, npm's package-publishing rules created yet another bind in that, once pulled, no other developer could publish another left-pad module utilizing an already-used version number. Thus, even though Westland published his replacement left-pad module just a few minutes after the Koçulu unpublished the original, it could not satisfy the hard-coded version dependency in line-numbers. That limitation is what forced Voss to take the "unprecedented action" of undoing Koçulu's withdrawal of the 0.0.3 release.

As several of the initial commenters on GitHub bug reports noted, another problem with npm is that modules are not namespaced by origin or user account. That made it impossible for downstream projects to simply switch from Koçulu's left-pad package to Westland's.

But perhaps the broadest issue raised is that npm itself is not managed by the Node.js community. Instead, it is run by a private entity and not subject to oversight. Koçulu's departure stemmed directly from what he saw as npm management siding with a for-profit company to take away his access to his own work. Many of the supportive comments from other developers seemed to agree—either expressing discontent with npm's unilateral action, or concern that no agreed-upon policy seemed to be at work.

There are, after all, many existing modules available in npm that use names trademarked by someone in the software industry, from well-known brands like Facebook to more generic words like square. The developer behind the newly minted not-kik package even challenged npm on the point, describing it with: "It's not kik, and never will be. But it has kik in its name. What now, NPM Inc.?"

Bryan Cantrill suggested that the "inescapable conclusion" was that the npm registry should be managed by the Node.js Foundation. So far, that idea does not seem to have gained much traction.

Lazyweb

The other side to the debate is concern that such a simple module as left-pad was widely enough used in the first place to wreak so much havoc when it disappeared. The module was eleven lines long, and served only to left-pad strings. Surely, the argument goes, such a simplistic function is too small to demand its own package—or, better yet, its purpose is so simple that any developers worth their salt should have implemented it themselves in mere minutes.

David Haney provided an in-depth critique of the Node.js community on precisely those grounds, calling the left-pad catastrophe symptomatic of a "stringing APIs together and calling it programming" mindset in the npm ecosystem, with developers "writing the smallest amount of code possible to string existing library calls together in order to create something new that functions uniquely for their personal or business need."

He added: "What concerns me here is that so many packages took on a dependency for a simple left padding string function, rather than taking 2 minutes to write such a basic function themselves." While the sentiment was echoed elsewhere on various discussion forums, Node.js developers as a whole, it seems, did not take the criticism lightly. The second comment on Haney's post chided him for "fundamentally misunderstanding the npm ecosystem and small module philosophy". Whether that refutes or underscores Haney's argument seems to be largely a matter of perspective.

Apart from the effort it takes to write a string-padding function, many people have pointed out that the hard-coded version dependency in the line-numbers module is bad form in its own right. Furthermore, even when it was unclear whether or not left-pad 0.0.3 could be restored to the npm registry, many developers complaining about broken builds seemed unaware of the fact that there are other ways (outside of npm) to specify a dependency in a Node.js package. One could even have pointed directly at Koçulu's left-pad code on GitHub.

Node War II?

On March 29, npm announced that it would, indeed, be changing its module-unpublishing policy. In the new policy, authors have a 24 hour window in which a published version can be unpublished. After 24 hours, authors will have to contact npm's support team, who will check whether or not the removal will break any dependencies for other packages. Furthermore, if every published version of a module is unpublished, npm, Inc. will take over the package name to prevent "malicious squatting". The post calls the new policy "a first step towards balancing the rights of individual publishers with npm’s responsibility to maintain the social cohesion of the open source community". Time will tell how well it will be received by the Node.js community.

The kik/left-pad debacle hinged largely on version numbering and package naming, but the incident is far from the only criticism leveled at npm. For instance, Google's Sam Saccone found that npm module installation poses a significant security risk. The npm command-line tool allows downloaded packages to run scripts with the full privileges as the current user; this means that a malicious script hidden in a dependency might run arbitrary commands as a user (or even as root). Among the possibilities is that such a script could publish updates of the user's modules directly to npm, allowing it to spread as a worm.

Others, like Armin Ronacher, warn that npm's "micro-dependency" approach leaves open several other possible exploits, starting with a "dependency explosion" that creates more packages than a reasonable person can be expected to maintain with scrutiny. One individual developer, he noted, maintains 827 separate modules on npm. Even if they are small, he said, "it's significantly harder for him to ensure that all of those are actually his releases". As recent events have shown, even a small module can be the source of considerable trouble. The open question is whether or not npm—and the larger Node.js development community—will learn from this experience and be able to self-correct.