Last week, I released npm@2.0.0 . If you’ve been using npm@1.4 , it’s a substantial update, but that’s not why it’s 2.0.0. npm@1.0.1 was released on April 30th, 2011 – three and a half years ago. 1 That’s basically the entire lifetime of Node as a viable platform. Why bump the major version now? The answer is pretty boring:

why 2.0.0?

In npm@2.0.0 , Ben changed npm run-script to allow you to pass arguments into scripts. That’s a breaking change. It’s as simple as that. Think of npm 2 as a step on the road towards getting npm right with semver. (There will be more. npm 3 will be out before the end of the year.)

But we did slip some other breaking and significant changes into npm 2, so please read on.

a note on npm’s release process

Those of you who follow me on twitter may have noticed that I’m doing something quirky with npm releases. This is because the npm CLI now has a release process based on a powerful, underused feature of npm: dist-tags. Every week, we publish the new version to the next dist-tag. After smoke-testing the release for a week, during the next week’s release, the release manager (me) promotes that version to the latest tag, which is what gets installed when you run npm install -g . 2

This means that every week, there are at least two versions of npm published – npm@latest , for everybody, and npm@next , for those who enjoy playing around with new things, or who want to help us test npm. (I would love it if more of you were running npm@next and giving me feedback on the next release. Just saying.)

revising node-semver

In npm 2, semver , the package that npm uses to deal with versioning, jumps straight from version 2 to version 4. Here’s why:

A while ago, we noticed that semver was at odds with the semver standard over how 0.x.y versions are interpreted. Isaac was fine with making node-semver follow the standard more closely, even though this was a breaking change and therefore meant that node-semver and the semver standard would have different major versions (this was for real his biggest objection to the change).

Isaac landed this change in npm@2.0.0-alpha.6 . At first, it seemed like this wasn’t going to be a very painful change (especially because Isaac also made 1.0.0 the default version for npm init ). It only affected the behavior of version ranges using ^ , and only for pre-1.0.0 versions. Unfortunately, we’d overlooked our old frenemy peerDependencies . 3

Consider grunt: pretty much the entire grunt ecosystem uses versions around 0.4.x . All it takes is a few plugins peer-depending on "grunt": "^0.4.2" while others are peer-depending on "grunt": "^0.4.5" to shove ordinary developers into dependency hell. Toolchains that were working fine for months suddenly couldn’t be upgraded, or even reinstalled, and npm started complaining constantly.

That wasn’t going to work. Isaac tried to reconcile the standard’s semantics for 0.x.y versions with common practice in the npm community. While the discussion was thoughtful, it was also slow. To get npm@2.0.0 out the door, we came up with our own semver that matches our understanding of how you, our users, think about semver (which, as Isaac says, was the intention of the standard’s authors all along). semver@4 is easy to describe:

Revert ^ behavior to semver@2 ’s. Exclude prerelease versions from matching ranges. You must pin against prerelease versions explicitly ( =1.2.0-alpha.5 ) to include them in package.json . Make the documentation easier to follow.

This is much better, except few package developers understand how prereleases work in semver. The thing about semver is that it’s semantic – almost all of the parts of a npm package version mean something. When you consistently use - to join the upstream version to the npm version of phantomjs, you’re (probably unintentionally) marking those versions as prereleases, and therefore not installable for people using ^ and ~ ranges. Getting maintainers to stop doing this is a much smaller problem than boiling the Grunt ocean, so we’re handling this by contacting the maintainers of affected packages directly.

We went through a lot of work to end up not that far from where we started. But the result is that we have a much better understanding of the implications of semver, a more consistent semver, and much more accessible documentation. 4 Not too bad.

scoped packages

The most prominent feature driving the release of npm 2 didn’t actually need to be in a new major version at all: scoped packages. npm Enterprise is built around them, and they’ll also play a major role when private modules come to the public npm registry.

Scopes themselves are simple, and the best way to learn about them is in the fine new documentation on scopes, npm install , and npm adduser that Laurie put together. One thing that might not be obvious from the docs is that because of the need to tie scopes to registries, you can now be logged in to multiple registries simultaneously. Most people don’t need this, but it’s handy for those who do.

improved reliability

Because npm is a node application from the ground up (or is it?), everything it does is a giant wad of highly-concurrent, continuation-based silly string. This makes npm really good at soaking up your machine’s available disk and network bandwidth, and given how much npm is doing, it’s surprisingly problem-free.

One fun thing about concurrency is that the more you increase a concurrent process’s efficiency, the more likely you are to discover its buried race conditions and deadlocks. 5 Another fun thing is that eliminating one race condition will make smaller, more obscure race conditions more likely to bite you. And so it’s been for us over the past few months:

#5453: Isaac fixed a bug where package.json files weren’t being written to the cache atomically, causing corruption when multiple processes were running npm installs simultaneously.

files weren’t being written to the cache atomically, causing corruption when multiple processes were running npm installs simultaneously. npm/lockfile#4: Isaac and Rebecca fixed a bunch of race conditions in lockfile , which fixed a host of lock contention issues.

, which fixed a host of lock contention issues. #6043: Isaac and I fixed a few install cases where npm would try to roll back failed optional dependencies before they’d finished unpacking. This closed a lot of issues.

#6182: Incredibly helpful first-time npm contributor Filip Weiss incorporated Rebecca’s write-file-atomic module into npm pretty much because he saw that it needed to be done (MORE OF THIS, PLEASE), closing a bunch of synchronization holes.

module into npm pretty much because he saw that it needed to be done (MORE OF THIS, PLEASE), closing a bunch of synchronization holes. #5920: Isaac and I cried a lot, and then eventually Isaac made stream writes inside the cache atomic, a simple resolution to a problem so difficult to troubleshoot that it defies hyperbole. The short version: last week sucked. (The last two fixes are only in npm@2.0.2 , currently npm@next .)

I am a pessimist, and also lack the hubris necessary to claim we’ve fixed everything. In fact, I know we have at least one more race condition lurking in there somewhere. I’m still very proud of the work the entire npm community has done in finding, analyzing, and fixing these bugs. If you’re using grunt, gulp, broccoli, Phonegap, Cordova, ember-cli , or Yeoman, you are now much less likely to run into baffling failures when using them. If you have complicated builds involving optionalDependencies or lifecycle scripts, they are much more likely to work right, every time. This is reason enough for me to celebrate npm 2’s release.

the full changelog

There are, of course, many other features (local dependencies! bearer-token auth!) and bug fixes (file permissions in the cache! allow % in passwords!). Here’s the semi-organized, complete changelog for npm 2.

BREAKING CHANGES

Semver:

4378a17 semver@4.0.0 : prerelease versions no longer show up in ranges; ^0.x.y behaves the way it did in semver@2 rather than semver@3 ; docs have been reorganized for comprehensibility (@isaacs)

: prerelease versions no longer show up in ranges; behaves the way it did in rather than ; docs have been reorganized for comprehensibility (@isaacs) 6e6a5fb prepare for upgrade to node-semver@4.0.0 (@isaacs)

prepare for upgrade to (@isaacs) 803da54 npm-registry-client@3.2.0 : prepare for node-semver@4.0.0 and include more error information (@isaacs)

: prepare for and include more error information (@isaacs) ea547e2 Bump semver to version 3: ^0.x.y is now functionally the same as =0.x.y . (@isaacs)

Bump semver to version 3: is now functionally the same as . (@isaacs) 60fe012 npmconf@2.0.6 : init.version defaults to 1.0.0 (@isaacs)

: init.version defaults to 1.0.0 (@isaacs) support for ‘init.version’ for those who don’t want to deal with semver 0.0.x oddities (@rvagg)

Arguments passed to npm run-script <script> :

df4b0e7 #5518 support passing arguments to run scripts (@bcoe)

#5518 support passing arguments to scripts (@bcoe) dbf0cab you can now pass quoted args to npm run-script (@bcoe)

you can now pass quoted args to (@bcoe) 021770b lifecycle: do not add the directory containing node executable (@chulkilee)

npm requires node >= 0.8:

c6ddb64 npm now assumes that node is newer than 0.6 (@isaacs)

large features

Scoped packages:

7f55057 install scoped packages (#5239) (@othiym23)

install scoped packages (#5239) (@othiym23) 0df7e16 publish scoped packages (#5239) (@othiym23)

publish scoped packages (#5239) (@othiym23) 0689ba2 support (and save) –scope=@s config (@othiym23)

support (and save) –scope=@s config (@othiym23) f34878f scope credentials to registry (@othiym23)

scope credentials to registry (@othiym23) a11c88b #6175 pack scoped packages correctly (@othiym23)

#6175 pack scoped packages correctly (@othiym23) 1b2ffdf #6097 document scoped packages (@seldo)

#6097 document scoped packages (@seldo) c92b8d4 #6004 manually installed scoped packages are tracked correctly (@dead-horse)

#6004 manually installed scoped packages are tracked correctly (@dead-horse) 21ca0aa #5945 link scoped packages correctly (@dead-horse)

#5945 link scoped packages correctly (@dead-horse) 16bead7 #5958 ensure that file streams work in all versions of node (@dead-horse)

#5958 ensure that file streams work in all versions of node (@dead-horse) cd422c9 #5748 link binaries for scoped packages (@othiym23)

#5748 link binaries for scoped packages (@othiym23) 4c3c778 #5758 npm link includes scope when linking scoped package (@fengmk2)

#5758 includes scope when linking scoped package (@fengmk2) fall back to _auth config as default auth when using default registry (@isaacs)

config as default auth when using default registry (@isaacs) a3a85dd --save scoped packages correctly (@othiym23)

scoped packages correctly (@othiym23) consistently use node-package-arg instead of arbitrary package spec splitting (@othiym23)

instead of arbitrary package spec splitting (@othiym23) eef4884 use the correct piece of the spec for GitHub shortcuts (@othiym23)

Fixes for race conditions:

ea515c3 #6043 slide@1.1.6 : wait until all callbacks have finished before proceeding (@othiym23)

#6043 : wait until all callbacks have finished before proceeding (@othiym23) 0b0a59d #6043 defer rollbacks until just before the CLI exits (@isaacs)

#6043 defer rollbacks until just before the CLI exits (@isaacs) 0583874 tar@1.0.1 : Add test for removing an extract target immediately after unpacking. (@isaacs)

: Add test for removing an extract target immediately after unpacking. (@isaacs) 22d72a8 fstream@1.0.2 : Fix a double-finish call which can result in excess FS operations after the close event. Part 1 of race condition leading to ENOENT errors. (@isaacs)

: Fix a double-finish call which can result in excess FS operations after the event. Part 1 of race condition leading to errors. (@isaacs) cdf3b04 lockfile@1.0.0 : Fix incorrect interaction between wait , stale , and retries options. Part 2 of race condition leading to ENOENT (@isaacs) errors.

Local dependencies:

4067d6b #5629 support saving of local packages in package.json (@dylang)

#5629 support saving of local packages in (@dylang) 16073e2 npm-package-arg@2.1.0 : support file URIs as local specs (@othiym23)

: support file URIs as local specs (@othiym23) ca0ef0e correctly interpret relative paths for local dependencies (@othiym23)

correctly interpret relative paths for local dependencies (@othiym23) 9164acb github-url-from-username-repo@1.0.2 : don’t match strings that are already URIs (@othiym23)

: don’t match strings that are already URIs (@othiym23) fa79413 #6119 fall back to registry installs if package.json is missing in a local directory (@iarna)

#6119 fall back to registry installs if package.json is missing in a local directory (@iarna) 5eb8db2 npm-package-arg@2.1.2 : support git+file:// URLs for local bare repos (@othiym23)

smaller features

0ac7ca2 capture and store bearer tokens when sent by registry (@othiym23)

capture and store bearer tokens when sent by registry (@othiym23) 66c7423 npmconf@2.0.7 : support -C as an alias for –prefix (@isaacs)

: support -C as an alias for –prefix (@isaacs) 685f8be npm-registry-client@3.1.3 : Print the notification header returned by the registry, and make sure status codes are printed without gratuitous quotes around them. (@isaacs / @othiym23)

: Print the notification header returned by the registry, and make sure status codes are printed without gratuitous quotes around them. (@isaacs / @othiym23) 4af0e71 make default error display less scary (@isaacs)

make default error display less scary (@isaacs) ab8dd87 swap out ronn for marked-man@0.1.3 (@isaacs)

swap out for (@isaacs) 78a1fc1 github-url-from-git@1.4.0 : add support for git+https and git+ssh (@stefanbuck)

: add support for git+https and git+ssh (@stefanbuck) b6bb746 build: add 'make tag’ to tag current release as latest (@isaacs)

build: add 'make tag’ to tag current release as latest (@isaacs) 27c4bb6 build: publish with --tag=v1.4-next (@isaacs)

build: publish with (@isaacs) cff66c3 build: add script to output v1.4-next publish tag (@isaacs)

build: add script to output publish tag (@isaacs) 1be4de5 build: remove unpublish step from make publish (@isaacs)

bug fixes

673d738 ensure permissions are set correctly in cache when running as root (@isaacs)

ensure permissions are set correctly in cache when running as root (@isaacs) ca791e2 restore a long (always?) missing pass for deduping (@othiym23)

restore a long (always?) missing pass for deduping (@othiym23) d2d4d7c #6082 don’t allow tagging with a semver range as the tag name (@isaacs)

#6082 don’t allow tagging with a semver range as the tag name (@isaacs) 9bac6b8 npmconf@2.0.8 : disallow semver ranges in tag configuration (@isaacs)

: disallow semver ranges in tag configuration (@isaacs) ed207e8 npm-registry-client@3.1.7 : Clean up auth logic and improve logging around auth decisions. Also error on trying to change a user document without writing to it. (@othiym23)

: Clean up auth logic and improve logging around auth decisions. Also error on trying to change a user document without writing to it. (@othiym23) 0dc6a07 #6059 run commands in prefix, not cwd (@isaacs)

#6059 run commands in prefix, not cwd (@isaacs) e8d75d0 #6057 read-installed@3.1.1 : properly handle extraneous dev dependencies of required dependencies (@othiym23)

#6057 : properly handle extraneous dev dependencies of required dependencies (@othiym23) 0602f70 #6064 ls: do not show deps of extraneous deps (@isaacs)

#6064 ls: do not show deps of extraneous deps (@isaacs) 5af493e ensure lifecycle spawn errors caught properly (@isaacs)

ensure lifecycle spawn errors caught properly (@isaacs) b4c717b npm-registry-client@3.1.4 : properly encode % in passwords (@isaacs)

: properly encode % in passwords (@isaacs) a8cb676 #5900 remove npm from its own engines field in package.json . None of us remember why it was there. (@timoxley)

#5900 remove from its own field in . None of us remember why it was there. (@timoxley) 6c47201 #5752, #6013 save git URLs correctly in _resolved fields (@isaacs)

#5752, #6013 save git URLs correctly in fields (@isaacs) 9243d20 lifecycle: test lifecycle path modification (@isaacs)

lifecycle: test lifecycle path modification (@isaacs) 1d5c41d install: rename .gitignore when unpacking foreign tarballs (@isaacs)

install: rename .gitignore when unpacking foreign tarballs (@isaacs) 9aac267 cache: detect non-gzipped tar files more reliably (@isaacs)

cache: detect non-gzipped tar files more reliably (@isaacs) f5a9434 test: fix Travis timeouts (@dylang)

test: fix Travis timeouts (@dylang) 9d73de7 remove unnecessary mkdirps (@isaacs)

remove unnecessary mkdirps (@isaacs) 33ccd13 Don’t squash execute perms in _git-remotes/ dir (@adammeadows)

Don’t squash execute perms in dir (@adammeadows) ac7a480 #5406 npm cache displays usage when called without arguments (@michaelnisi)

#5406 displays usage when called without arguments (@michaelnisi) f4554e9 Test fixes for Windows (@isaacs)

Test fixes for Windows (@isaacs) be06213 remove residual support for win log level (@aterris)

remove residual support for log level (@aterris) 375988b invalid package names are an early error for optional deps (@othiym23)

invalid package names are an early error for optional deps (@othiym23) 54cf625 fix handling for 301s in npm-registry-client@3.0.1 (@Raynos)

fix handling for 301s in (@Raynos) e410861 don’t crash if no username set on whoami (@isaacs)

don’t crash if no username set on (@isaacs) 0353dde respect --json for output (@isaacs)

respect for output (@isaacs) b3d112a outdated: Don’t show headings if there’s nothing to output (@isaacs)

outdated: Don’t show headings if there’s nothing to output (@isaacs) bb4b90c outdated: Default to latest rather than * for unspecified deps (@isaacs)

outdated: Default to rather than for unspecified deps (@isaacs) 63c3277 only delete files that are created by npm (@othiym23)

documentation fixes

e429e20 doc: add new changelog (@othiym23)

doc: add new changelog (@othiym23) 860a185 tweak docs to no longer advocate checking in node_modules (@hunterloftis)

tweak docs to no longer advocate checking in (@hunterloftis) 80e9033 add links to nodejs.org downloads to docs (@meetar)

add links to nodejs.org downloads to docs (@meetar) f9f58dd #5707 document generic pre- / post-commands (@sudodoki)

#5707 document generic pre- / post-commands (@sudodoki) e4e1223 #5936 document the use of tags in package.json (@KenanY)

#5936 document the use of tags in (@KenanY) 7b55f44 doc: Fix 'npm help index’ (@isaacs)

doc: Fix 'npm help index’ (@isaacs) f23f1d8 doc: update version doc to include pre-* increment args (@isaacs)

doc: update version doc to include increment args (@isaacs) 22abec8 build: remove outdated docpublish make target (@isaacs)

dependency upgrades

e4e48e0 #6121 read-installed@3.1.2 : don’t mark linked dev dependencies as extraneous (@isaacs)

#6121 : don’t mark linked dev dependencies as extraneous (@isaacs) d673e41 cmd-shim@2.0.1 : depend on graceful-fs directly (@ForbesLindesay)

: depend on directly (@ForbesLindesay) 9d54d45 npm-registry-couchapp@2.5.3 : make tests more reliable on Travis (@iarna)

: make tests more reliable on Travis (@iarna) 4fd9e79 npm-registry-client@3.2.1 : handle errors returned by the registry much, much better (@othiym23)

: handle errors returned by the registry much, much better (@othiym23) 0a67d53 #6007 request@2.42.0 : properly set headers on proxy requests (@isaacs)

#6007 : properly set headers on proxy requests (@isaacs) ae1d590 npm-package-arg@2.0.4 : accept slashes in branch names (part 1 of 3) (@thealphanerd)

: accept slashes in branch names (part 1 of 3) (@thealphanerd) 1d041a8 github-url-from-username-repo@1.0.0 : accept slashes in branch names (part 2 of 3) (@robertkowalski)

: accept slashes in branch names (part 2 of 3) (@robertkowalski) 65d2179 github-url-from-username-repo@1.0.1 : handle slashes in branch names (part 3 of 3) (@robertkowalski)

: handle slashes in branch names (part 3 of 3) (@robertkowalski) bf247ed columnify@1.2.1 (@othiym23)

(@othiym23) 4bbe682 cmd-shim@2.0.0 : upgrade to graceful-fs 3 (@ForbesLindesay)

: upgrade to graceful-fs 3 (@ForbesLindesay) b2f51ae semver@3.0.1 : semver.clean() is cleaner (@isaacs)

: semver.clean() is cleaner (@isaacs) 02c85d5 async-some@1.0.1 (@othiym23)

(@othiym23) 3f24755 readdir-scoped-modules@1.0.0 (@isaacs)

(@isaacs) 151cd2f read-installed@3.1.0 (@isaacs)

(@isaacs) 126cafc npm-registry-couchapp@2.5.0 (@othiym23)

(@othiym23) d987707 move fetch into npm-registry-client (@othiym23)

move fetch into npm-registry-client (@othiym23) 9b318e2 read-installed@3.0.0 (@isaacs)

(@isaacs) 48fd233 npm-package-arg@2.0.1 (@isaacs)

(@isaacs) update dependencies (@othiym23)

18a3385 npm-registry-client@3.0.2 (@othiym23)

(@othiym23) 4f54043 npm-package-arg@2.0.0 (@othiym23)

(@othiym23) 9e1460e read-package-json@1.2.3 (@othiym23)

(@othiym23) 719d8ad fs-vacuum@1.2.1 (@othiym23)

(@othiym23) 9ef8fe4 async-some@1.0.0 (@othiym23)

(@othiym23) a964f65 npmconf@2.0.1 (@othiym23)

(@othiym23) 113765b npm-registry-client@3.0.0 (@othiym23)