Package-maintenance: Publishing Strategy for PkgJS packages

[wesleytodd]

I would like to propose that we publish all PkgJS projects under the npm scope @pkgjs. We can dual publish on the global scope, but I think there is value in also always publishing under @pkgjs. There are approaches we could take to the dual publish, but I first wanted to align on the naming.

[ljharb]

What would be the advantages of dual publishing?

[wesleytodd]

I personally would always use @pkgjs/* when referencing them. The alignment between the GH org and the npm org is a branding opportunity, and one to make sure folks know where to go with issues or questions. When on the call @dominykas mentioned the only reason to publish wiby on the global scope was the ease of running npx on it. Personally I like npm @pkgjs/wiby, but there are some config issues I will let him describe (as they have not hit me).

EDIT: and to be clear, I would be a fan of just publishing under the scope, but if there are blockers to that I would rather be pragmatic.

[ljharb]

If we did that, I'd expect wiby to be nothing but a single dependency on @pkgjs/wiby: *, and to expose a bin file named wiby that proxies through to @pkgjs/wiby's bin file, and then we'd never need to update wiby again.

[wesleytodd]

Yeah, I am also fine with that. @dominykas proposed the other way around, but since I would like to use the @pkgjs version I had hoped that it would expose the entire package contents (hence why I proposed dual publishing). But I think we should agree on the general strategy before we make concrete decisions for each of the libraries.

[ljharb]

I'm confused, why would you care whether you use the scoped or unscoped version?

[wesleytodd]

Well, I would prefer we never publish anything under the global scope (as in perhaps in the future it shouldn't exist). It is traditionally harder to manage access, discovery, and configuration with multiple registries.

I recognize that I am not making a strong case, nor do I care to sell folks on my vision for this. So, if there are strong and compelling reasons to use the global scope (like npx issues, or UX of not typing the scope in a cli) then I think we should use it, but for me I would prefer to have orgs and scopes for things with shared ownership like this.

[ljharb]

That seems like a relatively uncompelling reason to me to add a bunch of complexity by having to manage two packages. The global scope will always exist, and new packages will continue to be created in it.

Unscoped packages can be managed by npm orgs identically as well as scoped ones; as far as I'm aware the difference only appears with non-public packages, or with the local "registry per scope" feature that requires a scope.

[wesleytodd]

Unscoped packages can be managed by npm orgs identically

This is why I phrased it above as "traditionally harder". This feature is relatively new and was a concern which is not applicable anymore. It still does mean the extra step of moving it under the org after the first publish. But I agree, this is not a compelling reason.

local "registry per scope" feature that requires a scope.

This also is something which will become a historical side note when https://github.com/npm/rfcs/pull/217 lands.

[Eomm]

It is traditionally harder to manage access, discovery, and configuration with multiple registries.

Agree on that

I had hoped that it would expose the entire package contents (hence why I proposed dual publishing)

I agree on dual publishing since exposing the content of a package could lead to misbehaviour - if exports will be used in future.

That seems like a relatively uncompelling reason to me to add a bunch of complexity by having to manage two packages.

Could a GHaction do the work?
I agree that adding tasks could be unproductive

[mhdawson]

My preferences are (in order)

1) just publish once under the pkgjs scope.
2) just publish in global scope

Edited to remove dual publish as I agree its more likely to cause problems than help.

[ljharb]

I don't think dual publishing ever makes sense - I think the proper thing to do if we want two packages is have one of them be the source of truth, and the other be a thin wrapper. There should never be two sources of truth.

My preference, thus, is:

1. global scope
2. pkgjs scope

[wesleytodd]

Ok, so would guidance like this be agreeable to everyone?

Projects in the pkgjs GitHub org will seek consensus among the collaborators for what to publish on the registry. When a package is published to the global scope, ownership should be transferred to the pkgjs org right after the first version is published.

The extra step will be required if teams decide to use the global scope, but we will land back with simple permission management via the org.

---

In addition to this, we should consider how we indicate the package is managed by the PMWG/Node.js project. This is one of the benefits of using the @pkgjs scope on the registry, and one which we will not get "for free" if the packages are spread out. Should we do something like add the logo to all the readmes? Add just a text line?

[ljharb]

What I typically do is create a user for the dual purpose of being a backup owner, as well as a visible link between org packages, to be an owner on all of the org's packages.

[dominykas]

So, the issue with scoped packages and npx is as follows:

• you npm i @scoped/pkg
• if @scoped/pkg exposes a bin - the bin will be symlinked into the node_modules/.bin
• the bin will be _unscoped_, i.e. even if you have bin: { '@scoped/pkg': './bin/scoped' }, you will end up with just the node_modules/.bin/pkg
• this means that if you have an unscoped package called pkg, and it has a bin, and you have it installed locally, and you npx @scoped/pkg, it will actually call the bin from the unscoped package, rather than from the scoped one - npx never checks where does the node_modules/.bin/pkg come from - if it exists, it will be executed
  
  
  
  • note that in effect, you can expose _any_ bin and it will symlink, so evil-package can expose bin: { webpack: 'evil-script' } and spoof things that way, although arguably this is not a security issue, because evil-package, if it's in your node_modules, presumably has already hacked you and does not need to wait for you to npx webpack
  
  
• most of the time this is not an issue, but it does cause some interesting effects, esp. if you have private registries and naming collisions (you very much do not want to npx @private/package and get a package from global namespace under any circumstance).

Either way, if you're exposing a bin for npx, you're better off squatting the globally namespaced package name, simply to avoid prevent naming collisions. While something like @pkgjs/detect-node-support is unlikely to collide, something like @pkgjs/support has an equivalent.

And if you're going to squat a globally scoped package, then you may as well just publish the whole thing there and save people some typing.

I don't have a strong opinion about dual publishing, but if it's just doing it for the sake of having everything under a scope, then I'm not sure that's a worthy goal.

Some might argue that this needs to be fixed in npx, and I'd agree, but that's a different topic altogether.

[ljharb]

Note that any package can expose any bin name it wants - if there's a bin conflict in your graph, you've got an unsolved problem. Using scoped packages doesn't alter this landscape (except perhaps that if you've never tested your scoped package and you've chosen a scoped bin name, which i expect to be quite uncommon, then you might not realize the conflict exists).

[dominykas]

Yup, the bin collisions are an old problem. I'm just highlighting the fact, that npx does not solve that problem and it does not check the origin of the bin - moreover the scope is dropped, so you end up with a chance of collision even if you name the bin the same as your [scoped] package.

[wesleytodd]

This sounds like something we need to work with npm on. Also, since this has moved into npm directly for npm@7 (along with changed behaviors) we should make sure this is still the behavior. @darcyclarke @ruyadorno @isaacs?

[wesleytodd]

a user for the dual purpose of being a backup owner

And for this, we should use the the openjs user, right?

[mhdawson]

Its currently https://www.npmjs.com/~nodejs-foundation which is used as the backup user for Node.js modules like node-addon-api

[dominykas]

Let's recap this during a meeting and see if we can close this?

[wesleytodd]

To recap: maintainers are free to choose what makes the most sense for their packages. Publish scoped or on the global.

Last remaining item is to document the fact that ownership has to be transferred when new packages are published. I will look for where to add it and submit a PR later today.

Done: #419

[wesleytodd]

Also, ensure required 2fa is enabled on the packages when you onboard them.

[dominykas]

419 is merged - closing this, please reopen if there's anything else we need to address.