Nvm: Npm global modules

[pensierinmusica]

Npm saves global modules where Node is installed.

This is unconvenient when there are multiple versions of Node on the system, and one needs some npm modules globally available (afterall, that's the concept of "global", isn't it?)

For the same reason, in the current setup global modules get lost each time a new version of Node is installed.

Think about modules like Yeoman (and its generators), Grunt, Express, and so on.

Wouldn't it be way more convenient if we could keep global npm modules in a common folder in the system (something like /usr/local/lib/node_modules), that can be accessed by all Node versions?

This way it's easier to install, uninstall, or update things just once and have the same setting system-wide. If a project needs something different, then the module should be installed locally (like it happens for most modules).

It just seem to make much more sense to me, besides being more efficient.

Is there a way for nvm to implement this? What are your thoughts?

Cheers!

Alex

[pensierinmusica]

Thanks to this thread, I just realized that what I suggested can be achieved by adding this line to .bash_profile:

export NPM_CONFIG_PREFIX="/usr/local"`

Closing this issue :)

[ljharb]

Please don't do this, however - you absolutely do not want to share global modules across node versions, as that can cause incompatibilities.

Use nvm reinstall-packages or nvm install node --reinstall-packages-from=node instead.

[ljharb]

More to the point, in a future version, nvm will actively prevent you from doing this - including unsetting the environment variable, and force-clearing npm config --global get prefix. It's a horrible idea.

[adam-beck]

With the rapid releases of io.js, does it make sense to add this to the README?

[pensierinmusica]

@ljharb I completely disagree. Let the user have this option and decide for him/her self.

In my case global packages are just a few shared tools, and reinstalling/updating them each time in different places would be a total nightmare.

Single project packages must be locally installed instead, and that avoids any compatibility problem already.

I see your concerns, but forcing users to do things they absolutely don't want is an arguable principle in software design. Rather trying to understand what they need, and how it could be achieved in a more convenient way might be helpful.

In case it's useful to have a concrete example, here's a list of npm modules I have globally installed right now:

• bower
• generator-angular
• generator-angular-fullstack
• generator-karma
• grunt
• grunt-cli
• http-server
• jshint
• mocha
• nodemon
• sequelize-cli
• traceur
• yo

I often need to switch between node stable and iojs, and regularly update both pieces of software as new versions come out. Imagine the nightmare of re-installing and updating all these npm modules in many different places each time, having to check if they're available and what version I'm using. That to me seems the exact opposite of "global", besides being a huge waste of time and completely unpractical.

[kriswill]

@pensierinmusica many native modules do not work between versions of node, like execSync. Even between node 0.12.0 and io.js 1.5.1 there are some fundamental differences with how native modules behave, so you cannot simply reuse them.

I think this is the main rationale behind maintaining separate global modules between versions.

[ljharb]

@pensierinmusica nvm use iojs; nvm reinstall-packages node, or nvm install iojs --reinstall-packages-from=iojs etc. Not much of a nightmare. In addition, you should _not_ be sharing global modules across node installations using nvm - that's not its purpose.

I think I have changed my mind somewhat - I won't have nvm modify things on the user's system outside its purview. However, I _will_ make it simply refuse to operate entirely when guaranteed-incompatible settings are enabled - so the user will definitely have the choice to continue with their unwise approach and use something other than nvm, or, fix it (simple instructions would be provided).

[pensierinmusica]

@ljharb thanks for sharing the shortcut to reinstall packages across different Node versions. I also agree that showing the user a message on how to fix incompatible settings is a good approach! Cheers

I think it would be very interesting also to give the user an option when it installs or updates a global npm module to be able to perform the same operation for all available node versions on the system. Imagine something like nvm install-npm-g express and nvm update-npm-g express, which could run in series the appropriate npm install -g express or npm update -g express for each node version on the system.

Thoughts?

[ljharb]

Not every node version _can_ have the latest npm - try it in node 0.6, for example - and that could be a _lot_ of node versions. nvm ls-remote | wc -l gives me 223 versions, a great many of which won't support the newest npm.

I could see a nvm install --update-npm option that attempted to npm install -g npm after installing a single version, perhaps.

[chevtek]

nvm reinstall-packages works great. I'm glad there's an easy way to get a version-specific set of all my same global modules :D

Thanks!

PS - I also agree with your decision to not invasively configure things on the user's machine. That said, a message informing them of their unwise decision would be welcome :)

[ljharb]

Closing this - global modules should never be shared across different node versions.

[ELLIOTTCABLE]

I'd like to see this re-opened. For the reasons mentioned above, having _software you happened to install with npm_ (best not to think of these as modules, or even as being usually Node-related, for this conversation) disappear every time you switch nodes while working on another project is _really_ unintuitive and surprising behaviour.

I'd like to posit that I'm not a _complete_ idiot, but this ‘feature’ literally bites me in the ass monthly. I've been aware how nvm handles global packages since I started using it, and I've dealt with problems caused by this over and over; but I _still_ don't immediately think ‘I just installed a new Node to test on, shit, I'd better go reinstall all of my packages!’ every time. The behaviour is simply that unintuitive.

Here's my solution to the problems listed above:

1. Make what is currently the reinstall-packages behaviour _the default_ upon installation of a new Node, for ‘global binaries;’
2. when switching nodes, notify the user about any new binaries existing under $old_node but not unde $new_node (i.e. binaries installed since $new_node was last used), and provide a simple Y/N prompt to install all of those packages immediately;
3. and finally, provide a _new_ feature for node-version-specific ‘global’ packages (basically: nvm changes npm prefix to a _different_ node_modules directory specific to the current Node, installs package requested, then symlinks binaries into the nvm bin-dir).

---

I realize this may be controversial, but I think it's a _really_ serious failing of nvm. When I “install coffeescript on my development machine,” I don't expect it to disappear if I cd into a directory for somebody's project and it has an .nvmrc, or when I install a new Node version to run a buggy node-inspector, or etc etc etc. And, yes, I also seriously believe that should be the _default_ behaviour, not an option. npm install -g foo and then using foo on-demand, without worrying about other actions you're taking or projects you're working on, forevermore (or approximately such), is very much the most usually-expected behaviour; whereas it's only rarely that node_modules resolution or obscure-Node-version-specific binaries-outside-of-projects are dealt with.

[ljharb]

By installing nvm, you're intending to switch versions of node, so any node software comes with it. This happens with rvm and virtualenv as well. You aren't "installing coffeescript on your development machine" - you're "installing coffeescript into the current version of node on your development machine". Encouraging that line to remain blurred in the minds of developers will create more problems, not less.

The only way this would work safely is if foo could always be run with the version of node it was installed with, instead of the current one - and that's not likely the behavior you'd expect.

[ELLIOTTCABLE]

It's one thing with something like virtualenv, which _is_ primarily to install packages or project-specific global binaries for the Python environment; but npm has become a package manager for _all sorts_ of software. This is an issue unique to nvm, IMHO.

Here's another way to look at it: by its very nature, nvm is in a position to mess with _a user's computer's primary package manager_. Not just the Node project they work on occasionally.

[ljharb]

You make a very strong argument. However, the fact remains: If I install foo on node v0.10, and that's what it's tested with, and then I switch to node v0.12, and run foo - that can cause lots of surprise and bugs.

If, after switching to node v0.12, foo would run _with_ node 0.10, then I don't think it would be surprising, and I think that would allow the use case you're advocating. I'm open to exploring that, but I can't conceive of how it would work.

Typically what I do is have a system node installed, and install global packages there, and then do _not_ install them on nvm-managed nodes - that way my system's primary package manager is not part of a version manager.

[chevtek]

@ELLIOTTCABLE Just create a quick function in your .bash_profile or .bashrc.

function nvm-install {
    nvm install $1 --reinstall-packages-from=default
}

If your default node/iojs install is where your global modules are, then you never have to remember to include the flag again. Just start using your bash function.

Example:

$ nvm-install 0.10.33

The above would install node v0.10.33 and copy all the global modules from the default version.

[ELLIOTTCABLE]

@ljharb hm; you're definitely right about ‘continuing to use the Node it was installed with’ being an even more intuitive approach. Two things:

1. I still think my solution is a good one, even if not quite _as_ good as what you're pondering (i.e. the ‘problems solved’ to ‘complexity of implementation’ ratio is much higher if you take my suggestion); maybe as an interim, even, while trying to factorize your idea?
2. In terms of implementing, it'd basically require wrapping the binary in one fashion or another. If you did implement that, perhaps putting it behind a new operation _isn't_ a bad idea. nvm permanently blah, where any new binaries existing at the end of command are automatically replaced with wrapped versions that invoke the current version of Node at-time-of-install? Hm.

@chevex that's a good start; @ljharb is there any ‘hook’ system in nvm? I'd prefer to keep using nvm install and nvm use, but I could certainly attach my own ‘maintain global package’ semantics to those if necessary.

[ljharb]

There's not a hook system really - there's lots of internal functions, but it's dangerous to rely on those, and there's lots of functionality in the raw nvm function still.

[chevtek]

You could also modify the bash function I suggested to literally symlink the global node_modules directories in newly installed versions. Then you'd transparently be pointing at the same global modules directory as the default install. You'd even be able to install new modules or update existing modules no matter which version you are using, and they'd all end up in the same place.

[ptim]

There's a lot to digest in this thread, I'm definitely missing details on use of nvm reinstall-packages iojs in the Readme; a detailed explanation in the wiki would really help..

I've installed nvm primarily to alleviate permissions issues with npm install -g whatever. I guess I'd say my system was in an 'unknown state'; prob had node installed, then brew install nvm, then read that brew is unsupported, so did brew uninstall, rm'd ~/.nvm, and followed install proceedures.

I'd already done a npm list -g --depth=0 to track what I'd installed so I've installed nvm from a clean state according to the readme, then:

nvm install iojs
nvm alias default iojs
npm install -g <selected packages>
nvm install 0.10.33
nvm alias old 0.10.33
nvm use old
nvm reinstall-packages iojs

.. which gets me back to a shiny, familiar state.. many tx

This was good reading, too: https://groups.google.com/d/msg/nodejs/A9ku09F39-k/PkYynJOCmrIJ

[ljharb]

@ptim a PR to enhance the read me is always appreciated ;-)

[lalomartins]

So this doesn't really matters for modules as the title says, but, AIUI, for scripts in .../bin. Which is what I encountered within 30 minutes of installing nvm :wink: My homebrew solution was to replace the global scripts with wrapper scripts: https://gist.github.com/lalomartins/e04317bd953f954bb0a1

How about baking this feature into nvm? I'm not sure whether it should be automatic or manual, though.

Automatic could work like:

• nvm watches the current version's bin folder
• a global bin folder for wrappers is configured somehow, defaulting to whatever is the current official cool (apparently ~/.local/bin on linux)
• if any new scripts are added
  
  
  
  • if there's already something with that name in the global PATH and it's not nvm-managed, leave it alone
  
  
  • if there's already one and it's nvm-managed, add a new version
  
  
  • else create a new wrapper with only the current version
  
  
• each wrapper works like:
  
  
  
  • check in a registry for which versions this script exists for
  
  
  • if it exists for the current version, use that (although it should have run that instead of the wrapper, under normal circumstances, since nvm's PATH precedes everything else… but I can imagine a few corner cases where it wouldn't)
  
  
  • if it exists for the default version, use that
  
  
  • otherwise use the highest/newest version
  
  
  • or the one with the highest version of the package providing the script? Tough
  
  

Manual is a little less tricky — nvm wrap script — but there's still the question of whether to support just the current version as my script does, or a version registry as proposed above.

[ljharb]

The best is nvm reinstall-packages - there is _no_ guarantee that a global module will work across versions of node/npm, and attempting to enable that is inarguably doing the wrong thing.

If it's not installed for the current version, it shouldn't be run - it should be installed, perhaps, but that's it.

[lalomartins]

@ljharb you're missing the point. This is not about allowing node scripts/apps to find global modules, it's about being able to run globally-installed apps regardless of what version nvm is bound to.

Use case:

Let's say I'm using app xyz which only works on Node 0.12. So ~/.local/bin/xyz is what launches it and it's on my PATH. And I use xyz a lot when I'm developing, for some reason. (This would work better if I could come up with an actual app, but I kind of do have other things to do.)

But then I switch nvm to 0.10 because I'm developing an app that requires 0.10. Suddenly, I can't use xyz any longer, and the reason for it is completely unintuitive — it might be quite some time before I realise I installed it with npm and even longer before I figure out it's tied to a specific node version, and even if I do, the answer “you can't use it when nvm is bound to a different version” is really not at all acceptable.

Even if the app does in fact work across versions, it makes no sense that I need to manually keep it up to date in each nvm environment, if it has nothing to do with developing node itself. Mine include bower, cake, coffee, grunt, and istanbul; they're stand-alone apps that are part of my workflow, and have nothing to do with what node version nvm is bound to.

In fact, nowadays that's pretty much the only acceptable use of global installs, AFAIK

[lalomartins]

The thing is, the issue title is misleading, based on the OP's initial understanding of the problem and solution. A better understanding developed as the discussion progressed.

[ljharb]

ahhh i think i see what you're suggesting.

You're saying if I have package X installed on the system, Y on node 0.10, and Z on io.js, that you'd like to be able to run X, Y, and Z on everything - but run them in the correct node version?

If you know where they're installed, you can do that now, but it sounds like you're suggesting something like nvm find-global X which would print out the closest version to current in which it's installed, or N/A to stderr? and then figure out some sensible way of making it easy to run X in that found version (nvm exec "$(nvm find-global X)" X)?

[lalomartins]

No, what I'm suggesting is:

• I install bower using npm and I don't even know what version of node nvm is using
• later when I'm using a different version of node I type bower
• and it just works

… because that's what people expect :-P you install bower because you're installing an app, the fact that it comes from an npm package doesn't even cross your mind.

[lalomartins]

It's all a question (_only_) of scripts that npm puts in $PREFIX/bin; I agree with you that sharing actual modules between versions is nothing but a source of headaches. But things that go in $PREFIX/bin are apps, and we expect them to remain available no matter what; nvm in this case is an implementation detail.

[ljharb]

Right, https://github.com/creationix/nvm/issues/668#issuecomment-113965165 would be the path towards making that happen.

Although I think that it just works is a bit intrusive there, so while I'm on board with adding the primitives to make that work, I'd need more convincing to be OK with adding that type of approach.

Currently, nvm use warns you what globals you're leaving behind, so it's on you to understand that globals are part of a node version, and not the entire machine.

[lalomartins]

I think you have an UX design problem there :-)

Think from the user point of view. Something like bower or cake is in no way part of a node version, and it shouldn't be; treating it as if it were breaks it for no good reason. It's an app that just happens to have been installed with npm.

Also, I doubt we'll convince people to type nvm run-global bower rather than just bower. (Or even install an alias.)

An alternative is to make this a completely separate subsystem in nvm; something like nvm install-app bower instead of npm install -g bower. This has the benefit that it wouldn't need to bind to the current version; rather, get the package.json, find the latest/best version the app supports which is also installed, and use that one.

[lalomartins]

… then it would be conceivable to print a warning if you install anything globally while in a nvm version. “Scripts installed with npm -g will only be available in the current version of node; if you meant to use them as a system app, install them with nvm install-app instead.”

[lalomartins]

Personally though, I think it's redundant to make it a separate subsystem, since according to the npm faq the _only_ time you should use install -g is for apps: https://docs.npmjs.com/getting-started/installing-npm-packages-globally

If you want to use it as a command line tool, something like the grunt CLI, then you can want to install it globally. On the other hand, if you want to depend on the package from your own module using something like Node's require, then you want to install locally.

[ljharb]

In this case the user is a developer, so a higher bar may be assumed. Global modules are absolutely part of a node version, and it's a mistake to think about them otherwise. I'll still think about how this might be resolved though.

[lalomartins]

Well, rather think about whether “Global modules are absolutely part of a node version, and it's a mistake to think about them otherwise”. Sleep on it maybe. Since your absolutely is different than my obviously, considering my argument is the least courtesy you can extend, rather than mashing a reply in less than a minute.

[lalomartins]

Bear in mind the npm faq pretty much agrees with me…

[ljharb]

The npm faq is written assuming you have a single node version installed, so it's irrelevant here.

[lalomartins]

The problem is you're thinking in terms (and the issue title refers to) “global modules”. In real-life node usage (and npm best practises), there's no such a thing as a global module, only a global app. Yes you can use those as a global module by fiddling with NODE_PATH, but that's recommended against.

[ljharb]

This may be some people's real-life node usage, but it is NOT mine, and it is not most nvm users'. In addition, there are no "npm best practices" here since npm's guide is _irrelevant_ in a multiple-node situation.

I'd certainly prefer to make using nvm as intuitive as using a single node version through traditional means, but to assume that's a) most people's usage, or b) something nvm users are entitled to, or nvm contributors are obligated to provide, is a bit shortsighted.

[lalomartins]

¯_(ツ)_/¯

[chevtek]

When you run a "global app" it runs a shebang script where the first line tells it to run "node" or "iojs". Your shell will then look in your PATH and find an executable for node or iojs, run it, and pass in the rest of the script to be interpreted. At no point in this process is it able to reach out to nvm, figure out what version of node/iojs is installed, download the correct package for that version and run the exact needed engine or "app". It uses whatever version you currently have installed. As @ljharb explained, npm is not built to understand a multi-install environment. If you upgrade node and suddenly a "global app" stops working then your only solution is to downgrade node to a supported version or upgrade the "app" to a version that supports the version of node you have installed.

If you use nvm to install multiple versions of node/iojs then it makes it easy to install a "global app" under one version and then later try to use it under another version. If nvm allowed this to happen it could cause it to break, or worse, run with unexpected side effects. There's no way for the shebang script for the "global app" to fire up the right version of node/iojs without some complex communication between bash and nvm. Bash would have to know what versions of node are installed and where. All nvm does is swap out which version of node/iojs is in your path so that bash can operate as it already expects to. It's not nvm's responsibility to magically make things work in different versions of node/iojs as you switch them around.

Trying to run scripts under engines that the script does not say it supports shouldn't be taken lightly because the side effects of running code in outdated engines or engines that are too new can be very unpredictable. You also can't just be swapping out your node/iojs version automatically whenever you want because you could be doing it underneath a running process that may potentially call into it again, say, to spin up child processes and whatnot.

Now if nvm functionality were baked into npm itself then I could see this discussion having a lot more merit because node and npm could work together to help scripts that are dependent upon a specific version of node, fire up the right engine during runtime. Nvm isn't part of npm though, and so must try to stay as hands-off as possible, letting node & npm operate as they normally would without ever having to be aware other versions are being swapped in at will. The best way to do that is to keep global modules compartmentalized with the proper node versions, and to make version switching be a manual user-intended operation. No automatic switching or version-agnostic globalizing just to make a single task slightly less inconvenient.

[arcseldon]

NVM is fantastic, but this is the single issue I have every time I upgrade even a dot version of Node. I usually run npm ls -g --depth=0 and copy / pipe that into a new file, do a little vim macro magic to get it all lined up as list of npm install -g module_name then paste that into the bash window. And the fact that some modules globally installed require sudo (eg. sails) whilst others don't adds its own headaches. There must be a better way?

What is the final conclusion in this thread? Do we really have to resort to workarounds like custom functions in our bash profiles etc? Does nvm reinstall-packages actually do what it says on the tin? If so, why the copious debate. I just want to upgrade node, and say to nvm "Yes, I know what I am doing just give me all the globally installed modules I was working with before." I don't want to have to mess about with symlinks to node_modules, bash scripts, etc etc. Ideally, NVM just takes care of any (heavy) lifting. Am I right in thinking nvm reinstall-packages default is what I am looking for?

Thanks.

[ljharb]

@arcseldon nvm install node --reinstall-packages=node is usually how i upgrade. That is exactly what nvm reinstall-packages and the --reinstall-packages flag are meant to provide.

[arcseldon]

@ljharb - appreciate the confirmation, thanks very much. I will make sure my Clients using Node are also aware of this.

[bahmutov]

Maybe my tool could help people a little: https://github.com/bahmutov/all-nvm - runs the same command (including install) across all Node versions managed by NVM

[ljharb]

@bahmutov if it's npm-installed that would install it in one of the nvm-managed node versions - that doesn't make any sense to me.

[bahmutov]

@ljharb the original post lamented the lack of "global" tools when using NVM. My tool solves this problem by installing the same tool in each, while avoiding shared / global installation problem that people were upset about in this thread.

[sidneys]

I solved this with a small script that uses some nvm core functions.

It upgrades nodejs to the latest version whilst migrating all global modules. Afterwards it updates every global npm package to "latest" as an added benefit.

# node_updater.sh
#
# This script makes keeping NVM-managed NodeJS and its global packages up-to-date a breeze.
# Requires nvm, the Node Version Manager (https://github.com/creationix/nvm)
# 
# First load it: _$: touch node_updater.sh
# Then run it: node_updater
#
# Protip: Add to your .bash_profile for hassle-free updating.
#

node_updater () {
    export NODE_CURRENT=$(nvm_ls_current)
    export NODE_LATEST=$(nvm_remote_versions | grep '^v' | tail -1)

    # Update global NodeJS to 'latest'. Migrates all installed global packages into the new Version.
    echo "Updating global NodeJS installation from ${NODE_CURRENT} to ${NODE_LATEST}."
    nvm install ${NODE_LATEST} --reinstall-packages-from=${NODE_CURRENT} && echo "Upgrade of global NodeJS installation complete."

    # Remove previous NodeJS versions
    nvm use ${NODE_LATEST}

    # Update global NodeJS packages to 'latest'
    for PACKAGE in $(npm --global outdated --parseable --depth=0 --loglevel silent | cut -d: -f3 | cut -f1 -d'@');
        do npm --global install --loglevel silent ${PACKAGE} && echo "Updated global package '${PACKAGE}' to 'latest'.";
    done

    # Uncomment to delete previous NodeJS  installations after update
    #find ${NVM_DIR}/versions/node -type d -mindepth 1 -maxdepth 1 -not -name ${NODE_LATEST} -exec sh -c 'printf "Removed NodeJS version: "' \; -exec basename {} \; -exec rm -r {} \;
}

Gist @ https://gist.github.com/sidneys/e27eb239fd022d3e5204

[pensierinmusica]

Thanks @SidneyS, looks cool :)

The only part I'm not 100% convinced about is:

# Remove previous NodeJS versions
    nvm use ${NODE_LATEST}

Would it make sense to change it with this?

# Update "default" alias and switch to it
    nvm alias default ${NODE_LATEST}
    nvm use default

Cheers!

[Jpunt]

For everyone interested.. I've made a little tool for this: npmg. It stores a list of packages that you want installed globally, and when you end up in a new environment (like a different node version) you can run npmg install to install them all 🙂

[waldyrious]

@Jpunt that's already possible natively with nvm. It supports a $NVM_DIR/default-packages file with a list of packages that will be automatically installed for any node version you install using nvm.

[Jpunt]

That's amazing! Unfortunately it doesn't work for me though 😢 I've added a package to that file, installed a new version of node, and that package isn't installed. Ah well, will take a look at it some other time.

[ljharb]

@Jpunt make sure you’ve updated to the latest version of nvm.