pnpm install should support monorepo

[KSXGitHub]

Why?

It is more convenient than having to look at files to decide whether it is a monorepo.

How?

If pnpm-workspace.yaml exists and/or link-workspace-packages = true in .npmrc, pnpm install should redirect to pnpm recursive install.

Some Clarifications

I only concern with pnpm install, not pnpm install <pkg>.

---

For reference, yarn uses yarn/yarn install for both monorepo and single repo.

[zkochan]

What about pnpm install foo? Currently pnpm recursive install foo means that foo is installed into every package in the repo.

[KSXGitHub]

It should throw an error and tell user to explicitly specify dependants via pnpm recursive ... command.

[zkochan]

but that would not be very intuitive if there was a package.json in the root.

[zkochan]

Maybe it is a good idea but I am not sure. I think users should get what they'd expect to get. I am not 100% sure everyone would expect pnpm install to behave as pnpm recursive install.

I was also thinking about how this can be made easier to type. One of my ideas was to use an -r flag for running commands recursively, so pnpm i -r would do pnpm recursive install. A poll I did on Twitter has shown that people like it https://twitter.com/pnpmjs/status/1011360303941148673

Also, we can use recursive when filtering is used, so pnpm i -- foo... == pnpm recursive i -- foo...

[KSXGitHub]

How about preventing user from pnpm install in a monorepo?

[zkochan]

but pnpm install works fine in a monorepo. For instance, if you run pnpm install inside a package of the monorepo, it will link all the dependencies to that package and also run install inside them.

Currently pnpm install inside foo is same as pnpm recursive install -- foo...

[KSXGitHub]

In my opinion, when people run pnpm install, I think they run it in context of the repo, not their current working directory. In other words, people ask _"in which repo should I run pnpm install"_ rather than _"in which directory/subdirectory"_.

[zkochan]

cc @vjpr @pgonzal @jvanbruegge

what are your thoughts about this?

the idea is basically changing the behavior of pnpm install to make it work as pnpm recursive install

it might be a good idea. Though I don't think pnpm install foo shoud work as pnpm recursive install foo

[zenmumbler]

My 2 cents: I did recently run pnpm install in the root when I should have done pnpm m install and was a bit surprised, but there are valid reasons — and Yarn begrudgingly allows this as well in their monorepo setup — to have an actual set of dependencies only for the root, e.g. for builds or deploys or other repo-wide functionality. Not everyone will have this, some have a build package inside the repo for example, but I'm not sure if either setup is more "default" than the other, esp. since tooling for JS monorepos is stil relatively new. Yarn also has the luxury of breaking completely from NPM's idioms.

[zkochan]

Thanks, @zenmumbler, any feedback is important.

I will create an RFC with my personal vision. It will be easier to discuss the aspects in a PR

[zkochan]

Here is my suggestion: https://github.com/pnpm/spec/pull/3

[jvanbruegge]

At the moment I can't really give input on this as we have not yet updated to 2.17 and we do not have a workspace yaml yet

[zenmumbler]

Reading the intro text and the examples make me feel it's a bit contradictory to have install in the root install only on the root, but install in libs will search downward for all packages in libs. Intuitively, I would expect these commands to scan up, not down, though I understand the idea of running a command on a group of packages without mucking about with filtering.

[zenmumbler]

In general, while I sympathise with the issue posted here, the package setups, even/especially(?) for monorepos are fractured to such an extent preventing any setup to be made canonically the "default". pnpm's shorthand pnpm m install is short enough, IMO.

What pnpm can do is check for e.g. empty/missing dependency lists in the root package.json and then issuing a "Did you mean 'pnpm recursive install' instead?" or some such and _not_ updating the (global) shrinkwrap, as currently it adds a . entry to it that is empty.

On the other hand, for a CWD inside a package directory, pnpm install could auto-scope to that package, but should print a notice that it did so in the output. So, my suggestion is sort of a hybrid.

[octogonz]

1. Running pnpm install in the root will only create node_modules for the root package.json.
2. Running pnpm install -r (or pnpm install -- .) in the root will install dependencies of all workspace packages.
3. Running pnpm install -r (or pnpm install -- .) in libs/ directory will only install dependencies of lib-a and lib-b

In our monorepos, normally nobody would ever want to install a single project in isolation. It wouldn't make any sense, because nearly all our projects depend on the current (unpublished) versions of other projects in the repo.

In some cases it might be useful to install a subset of projects. However PNPM doesn't know what the meaningful subsets are, and the person isn't going to want to explain that using the PNPM command-line. Instead, we would use a wrapper such as rush install --version-policy X which invokes PNPM behind the scenes with the right list of packages to install.

There is a scenario where the docker deployment wanted PNPM to locally install a single package (by copying its dependencies), but this is a somewhat special case, and it again would be invoked by an automated script.

[zkochan]

ok, so seems like most developers expect that pnpm install === pnpm multi install

In that case I agree with the majority.
My only suggestion would be to keep pnpm install something to work with the current package.json only. Otherwise, pnpm install something would add something to every package in the repository.

I will update the specs today

[KSXGitHub]

My only suggestion would be to keep pnpm install something to work with the current package.json only. Otherwise, pnpm install something would add something to every package in the repository.

How about this: When user issues pnpm install <pkg> in a monorepo, pnpm should print an error and a suggestion.

[vjpr]

1. Running pnpm install in the root will only create node_modules for the root package.json.
2. Running pnpm install -r (or pnpm install -- .) in the root will install dependencies of all workspace packages.
3. Running pnpm install -r (or pnpm install -- .) in libs/ directory will only install dependencies of lib-a and lib-b.

@zkochan I am very used to pnpm m i. pnpm i -r is not nice to type.

pnpm i foo installs/workspace-link + saves package into closest package.json.

It is very rare that I want to install a package in multiple repos, so filters are good for that.

ok, so seems like most developers expect that pnpm install === pnpm multi install

Yeh I agree with this. But pnpm install foo should install to closest package.json.

[octogonz]

Agreed, I would suggest for the spec to be organized around realistic problems a developer would encounter, and the list of commands they would run, with the goal to make those commands as simple and intuitive as possible. This is generally a better approach than organizing it to reflect the implementation details that are happening behind the scenes.

For example:

3. Running pnpm install -r express (or pnpm install express -- .) in the root will install express as dependency in all workspace packages.

This scenario seems fairly unlikely -- for a small repo, you can just run the PNPM command several times. For a large repo, you would never want to affect on "all" packages, because you probably don't even know what half the packages are.

The pnpm install express -- . variant seems a bit awkward. If I polled random developers familiar with PNPM, and asked them what -- means, they probably would have no idea. And since it's not a named command-line parameter, it's difficult know how to search for help: pnpm install -h doesn't explain it. Googling for "pnpm --" won't return any useful results either.

Monorepos tend to have lots of developers working in them. Some of them make one or two PRs and move on. So having a simple "on the rails" user experience and very easy learning curve is pretty important.

[KSXGitHub]

The pnpm install express -- . variant seems a bit awkward. If I polled random developers familiar with PNPM, and asked them what -- means, they probably would have no idea. And since it's not a named command-line parameter, it's difficult know how to search for help: pnpm install -h doesn't explain it. Googling for "pnpm --" won't return any useful results either.

The ... -- <args> syntax convention is quite common in posix world. It is often used to pass arguments start with - as normal arguments (instead of options/flags). Git also has this (for instance, git checkout -- <files> will undo changes made to <files>).

[zkochan]

@pgonzal filtering is described on the website https://pnpm.js.org/docs/en/pnpm-recursive.html#lt-package-selector-filter-lt-package-selector (also, we had an RFC for the filtering syntax https://github.com/pnpm/spec/pull/1)

and it is also described in help of the recursive command: pnpm recursive -h

I personally use filtering frequently and I think it is a useful feature. It allows to do complex tasks on a subset of packages. On a big repo (with hundreds of packages), I don't want to do everything on everything because that would be slow. With filtering you can do things like pnpm test --filter ...foo that will run tests of foo and tests of dependents of foo.

[octogonz]

Is -- simply a synonym for --filter?

[zkochan]

You can list any number of filters after --. With --filter you should put a --filter before each one

pnpm m i --filter foo --filter bar... === pnpm m i -- foo bar...

[vjpr]

ok, so seems like most developers expect that pnpm install === pnpm multi install

Is this still the consensus?

I am opposed to this. Most devs are used to working with single packages where they expect pnpm i to just install for that package.

If I run pnpm i in the root, I only want the root to install (say after I add a dep to the root package.json).

[KSXGitHub]

I am opposed to this. Most devs are used to working with single packages where they expect pnpm i to just install for that package.

If I run pnpm i in the root, I only want the root to install (say after I add a dep to the root package.json).

It works the same in single package repo.

[zkochan]

In this case, I think we have a consensus on:

1. any command + filtering should work (pnpm i -- foo == pnpm m i -- foo)
2. filtering options should be shown on pnpm i -h
3. info messages should tell the user about the scope of the commands

We can come back to what should be the scope of pnpm install later

---

I'd also love some fun shortcuts like:

pnpm ii == pnpm multi install
or
pnpm I === pnpm multi install

[zenmumbler]

Just my opinion, but please no distinction between lower and uppercase letters. That is, IMCO, the worst aspect of UNIXness. (Cue the people to correct me it's actually POSIX, or GNU, or what have you ;) I will reiterate by questioning again: how is pnpm m i too long? I understand I don't have to use the shortcut-shortcuts, but these things tend to create a theme over time. I may be overreacting, I don't know.

I also still stand with pnpm install in the root == root only install or warning without actions if the root has no package.json or has one with empty/missing dependencies.

[KSXGitHub]

but please no distinction between lower and uppercase letters. That is, IMCO, the worst aspect of UNIXness.

The other posix tools already have that distinction, it's already a common convention, going against it wouldn't make any sense. Besides, pnpm already has that distinction, because it's computer's default behavior.

how is pnpm m i too long? I understand I don't have to use the shortcut-shortcuts, but these things tend to create a theme over time.

It's not about "too long", it's about sensible default. And pnpm install is a standard/default command, it should make sense in all situations including single-package repos and monorepos.

Imagine cloning a monorepo without knowing it is a monorepo, run pnpm install, expecting all npm scripts work out-of-the-box but it doesn't because of missing dependencies.

We should also consider external tools that work with pnpm. External tools (such as CI, Renovate, etc.) wouldn't have to know if a repository is a monorepo, just run pnpm install and everything will just work.

[zkochan]

I created a PR for one thing that seems like nobody was against.

If filters are used, the command is automatically recursive. So pnpm i -- foo === pnpm m i -- foo

[zkochan]

From v2.18, it is enough to specify a filter and the command will be recursive. So you can do things like pnpm install -- foo or pnpm test --filter bar

[morganfree]

I have a package.json file in the root of the monorepo with its own devDependencies. When I type pnpm install I expect it will install modules for the root only - similar as pnpm install some-package will install some-package to the root only. The proposed change does not seem to be coherent with other commands.

We should also consider external tools that work with pnpm. External tools (such as CI, Renovate, etc.) wouldn't have to know if a repository is a monorepo, just run pnpm install and everything will just work.

I think this is a place where the npm scripts should be used and customised, e.g. default install overwrite. Default commands' behaviour should be coherent.

[KSXGitHub]

To clarify, when I opened this issue, I only cared about pnpm install, not pnpm install <pkg>. So how pnpm install <pkg> works is irrelevant to me. As long as pnpm install installs all packages recursively, I am happy.

[zkochan]

I think we can create a config that will make pnpm install === pnpm recursive install.

Something like prefer-recursive-commands=true

[KSXGitHub]

I think we can create a config that will make pnpm install === pnpm recursive install.

Something like prefer-recursive-commands=true

It's more about sensible default, that's the reason I opened this issue. I originally only concern of the default command (that is pnpm install, not pnpm install <pkg>).

It is true that I can easily set prefer-recursive-commands=true, but it is also true that I can assign pnpm recursive to a bash alias/command instead, so this flag is unnecessary, it adds no extra benefits but needless complexity.

Again, what I want to discuss is about the default pnpm install and only the default pnpm install, no extra arguments, no extra flags, no extra configurations.

[zkochan]

I am now convinced that this should be the default but this is a breaking change. Till v4 it will be on a config called use-beta-cli.

[zkochan]

PR is in progress. See #1913

[zkochan]

:ship: 3.6.0-0

use the use-beta-cli=true config