Yarn: Yarn.lock shouln't contain any information about the registry used

[victornoel]

Hi,

When trying to change the registry I was using (from default one to a Nexus internal to my company), I realised that yarn.lock was containing the URL of the registry:

"@angular-cli/ast-tools@^1.0.0", "@angular-cli/ast-tools@^1.0.1":
  version "1.0.9"
  resolved "https://registry.yarnpkg.com/@angular-cli/ast-tools/-/ast-tools-1.0.9.tgz#be2bf235e03a956c053a1f65661abc981c7ffe38"
  dependencies:
    "@angular/tsc-wrapped" "^0.3.0"
    denodeify "^1.2.1"
    rxjs "5.0.0-beta.12"
    typescript "~2.0.3"

I don't think it is a good idea to honour that information when downloading packages (which is the case, see #2557), because the yarn.lock is meant to be committed to versioning system!
How would you manage situations where the CI, for example, is using a different registry than the developer's? What about developers working remotely from their company?

Yes there should be some md5sum for example to verify we are actually using the same artefact, but that's all.

[aalexgabi]

I guess there are two ways to think about packages: package-name@version or registry/package-name@version so two registries may not refer to the same source code for a package having the same name.

Today there is one well known registry: npm but there are also alternative or private ones that are not mirrors of npm. I guess registries are identified by their URL so when locking the registry name should be there. This poses some problems with mirrors because mirrors url's should not be in lock files (due to private subnetworks), instead the source registry's url should be there. This implies some type of configuration of yarn mirrors such as:

yarn mirror registry.yarnpkg.com 192.168.1.46

When yarn locks it should use the source registry URL but when downloading it should use the mirror instead.

My opinion is that we should make the distinction between these two notions: registry and mirror. One publishes and the other replicates.

What do you think about this?

[victornoel]

I don't really agree, for example the question has been existing in the Maven community, and nobody never has used the repository as a way to identify a package. Still, they do have the difference between mirrors and repository, but this is transversal.
It is the same with Debian-based distributions: packages have a name and a version, the place where you downloaded them does not matter!

What you CAN do when you lock a package/version, is also to register the hash of the source, so that a different source is not used. But the registry? it does not make sense!
We live in a globalised world and names (and version) are ways to uniquely identify artefacts: if you are working on a fork of an artefact that you store in your private registry, then simply change its name!

On the other hand, if you look at the Docker community, they DO use the registry as part of the identification of an image: but they do not have the problem we do here! They don't do version management in the same way it is done with Yarn, NPM or Maven.

[aikar]

@victornoel mentioned to include the hash of the file.

Verifying that the hash matches on the file solves the concern over 1 registry giving a different file for the same name@version than the other.

if the hash doesnt match, fail.

[victornoel]

@kittens @bestander I hope it's ok to ping you on that, I feel there is something important on this issue and the sooner it is processed, the easier it will be to fix it… Thanks :)

[bestander]

A fresh idea how to handle this use case is discussed here: https://github.com/yarnpkg/yarn/issues/579.
An RFC and a following implementation based on that approach is welcome.
"Up for grabs", we expect the community to help with this feature.

[victornoel]

@bestander thanks for your time.
Indeed @Kovensky does talk about the problem in a comment, but I don't see so much more mention to it later in the discussion, so I guess we can keep this issue open for that particular matter?

I will keep this RFC thing in mind, I'm not sure I am up to it :)

[victornoel]

FYI, as a workaround, the following script can be helpful to keep references to the official yarn registry in the yarn.lock while taking advantage of a mirror:

#!/bin/bash

{
  sed -i 's#https://registry.yarnpkg.com#http://private-registry/repository/npm#' yarn.lock
  yarn $@
} || true

sed -i 's#http://private-registry/repository/npm#https://registry.yarnpkg.com#' yarn.lock

[supnate]

In China it's very slow to use official npm repository so we usually use a mirror. But the mirror is also very slow when used out of China. So it's really a trouble to share a project both in and out of China when using yarn.

IMO, since yarn has allowed to config a registry url, it should not store it in lock file redundantly. It's developer's concern about what a package name means but not yarn's.

[ghost]

There is now a pending RFC to address this issue - yarnpkg/rfcs#64

[victornoel]

Closing this issue in favour of #3330 (since it was opened by the rfc's author).

Thanks @destroyerofbuilds