Volta: Support for pnpm

[willsoto]

Seems to be a fairly popular alternative to npm and yarn. Apologies if this has already been suggested but I couldn't find an existing issue. It would be great to be able to volta pin pnpm.

https://pnpm.js.org/en/

[charlespierce]

Agreed, this is definitely something we’ve discussed on Discord, I’m surprised there wasn’t an issue already either. Thanks!

[arxpoetica]

What would need to take place for this to happen? Point me to the right place in the code, and I'll see if I can get around to a PR.

[charlespierce]

Hi @arxpoetica, we would definitely welcome that! Adding support for pnpm—or really any unsupported package manager—is a bit involved, though hopefully not too difficult. There are 3 main areas that need to be updated:

• Fetching / installing / pinning the version, which is all contained within the tool module in crates/volta-core (https://github.com/volta-cli/volta/tree/main/crates/volta-core/src/tool)
  
  
  
  • The changes here can mostly be based on what is done for npm, the only caveat I'm aware of is that instead of editing the launcher files, we need to create them (so that the bare command pnpm works as expected).
  
  
• Reading / writing pnpm settings, contained in the platform, project, and toolchain modules.
  
  
  
  • The changes here should mostly be adding the pnpm key alongside npm and yarn
  
  
• Running pnpm through the shim, contained in the run module.
  
  
  
  • Here the logic should be almost identical to the yarn logic.
  
  

That's a very high-level overview of the needed changes, but I'll take a look at write some more this week about any corner cases / difficulties I can predict. I'm also happy to pair on this for a while if that would help.

[arxpoetica]

@charlespierce this ended up being something I just haven't had time to get around to yet. It's definitely still on my radar, but if someone feels like giving it a head start (or even attacking totally)...

...I think you'll find it happening more quickly. Sorry for my lack of involvement (re: time).

[charlespierce]

@arxpoetica No worries at all, thanks for updating!

[willsoto]

@charlespierce slightly related to this, has any thought been given to pinning arbitrary dependencies that would useful to have available on the command-line? For example, it might be useful to have mocha or tsc installed and available directly on the command line at a specific version without telling users to volta install it themselves. Another example is something like lerna or rush which are the primary entry-points for repos that are managed by those tools.

Under the hood, maybe something specific happens when pnpm is installed but in some ways it is just another command line tool being managed by Volta.

Just wondering if you consider this out of scope since the documentation for pin has something specifically about this.

[charlespierce]

Hi @willsoto, are you talking about something like #861? In that issue there's some discussion about the complications, however that is definitely something we'd like to support. Or do you mean a setting in package.json that says "Use Volta to manage running lerna"?

If the latter, there's a couple reasons that's not really in scope for Volta:

1. Technically speaking, Volta operates with shims in a shell-agnostic way, there's no way for us to shim tools that we don't know about ahead of time (if a user runs lerna and the shell says "Command not found", that never goes through anything Volta controls). Individual shells may have a way to control that behavior, but it's unlikely to be universally applicable across OS and shell combinations.
2. Philosophically, we decided against creating automatic shims for everything that had been installed, because it caused issues and raised some security concerns (see #235 for the discussions). We ultimately decided that the end user should have direct control over which tools we do or do not manage, so it would go against that to allow packages to take that control away from the user.

[willsoto]

Of course I didn't see #861 when I went looking for existing issues 🤦

After reading through it a couple of times, it does _seem_ like that would also solve my problem, just in a different way.

Or do you mean a setting in package.json that says "Use Volta to manage running lerna"?

Yeah, something like this is what I had in mind:

// package.json
{
  "volta": {
    "lerna": "2.0.0"
  }
}

I totally understand if this is considered out of scope for Volta, but I thought since pnpm is something that gets installed via npm (I don't think they have an alternative way to download?) then you could really get any tool in this way. I'll subscribe to #861 and see where that goes so I don't clutter this issue up. Thanks

[zkochan]

@muuvmuuv suggested us to document Volta as the preferred tool with pnpm. Unlike Yarn, pnpm currently doesn't have the ability to seamlessly switch between different versions of pnpm. So if Volta was to support pnpm, I think we could just recommend using the pin feature of Volta.

Also, @arcanis might be interested in this discussion, he created corepack

[zkochan]

I think we can offer a $50 bounty from our opencollective funds (if this is enough, I don't know how much work it involves).

[charlespierce]

@zkochan I took a look at it before our 1.0 release late last year, and it actually was somewhat involved, as our models and abstractions around the package managers aren't as clean as they could be (some room for tech debt cleanup). However, it was mostly straightforward. The main sticking point was around the global package directory: As I recall, there isn't a clean way to redirect _both_ the installation and bin directories to a custom location in pnpm using only environment variables.

Volta does that redirection when installing global packages so that we can sandbox them to a specific Node version, and I recall struggling to get that working. I _think_ I could do it with a command-line flag, but that gets a little tricky if there's already one specified and so I didn't dig in too deeply. There also appeared to be a "layout version" type directory that was always appended, which we would need to take into account, but I think I had worked out how we can handle that on our side.

I determined most of that with some experimentation and poking around the source of pnpm, so if there's something I missed about how to define the global directories, that would be much appreciated!

So in all, I think it's probably a larger task than a $50 bounty, however if someone was interested and wanted to collect that on top of the experience, I'd be more than happy to mentor them through it. Working on Volta is also part of my day-to-day, so if I can get it onto the roadmap that would be the other way.

[zkochan]

Is there something we could implement on pnpm's side to simplify the task?

You may set the NPM_CONFIG_GLOBAL_DIR env variable to change the global directory used by pnpm. The global bin directory will currently be selected automatically from one of the directories in PATH. pnpm has to have write access to the directory and the directory should have npm, pnpm, nodejs, or node in the path

https://github.com/pnpm/pnpm/blob/9c41e11b0bfa3f4ece2b7d4d67690496f1c9b9c0/packages/global-bin-dir/src/index.ts#L35-L40

[charlespierce]

Is there something we could implement on pnpm's side to simplify the task?

I don't think so, only because those changes would necessarily be in only newer versions of pnpm, while we would ideally want to support a larger range of versions.

You may set the NPM_CONFIG_GLOBAL_DIR env variable to change the global directory used by pnpm. The global bin directory will currently be selected automatically from one of the directories in PATH. pnpm has to have write access to the directory and the directory should have npm, pnpm, nodejs, or node in the path

https://github.com/pnpm/pnpm/blob/9c41e11b0bfa3f4ece2b7d4d67690496f1c9b9c0/packages/global-bin-dir/src/index.ts#L35-L40

Interesting, that is jogging my memory a bit. I think I may have been able to get that to work by modifying the PATH, but I don't recall the specific issues I was hitting any more (and I appear to have failed to write them down anywhere). I don't think any of them were insurmountable, just that they were decent amount of work to fit into everything, taking into account our current model.

[muuvmuuv]

I would add another 30€.

Just saying, but I'm already using pnpm with Volta since three months without any problems but would also love to see a "clean" way of implementation for future package managers/binaries so Volta will stay on top of the other managers.