Yarn: Lock file ignores `yarn-offline-mirror` config for git+ssh dependencies

[albertfdp]

Do you want to request a feature or report a bug?

Bug

What is the current behavior?

When installing a dependency using git+ssh, the generated lock file doesn't reference the yarn offline mirror, but keeps pointing to the external dependency. The dependency however is correctly added to the offline mirror.

When installing a dependency using git+ssh, the first time the dependency is correctly added to the offline mirror and yarn.lock correctly references it. However, installing that same dependency again in a different project (so, resolved from yarn cache), it points to the external registry.

If the current behavior is a bug, please provide the steps to reproduce.

1. Add a yarnrc config to your project with the yarn-offline-mirror config
2. Run yarn add webpack@git+ssh://[email protected]/webpack/webpack.git#v2.2.0-rc.2 or any private git dependency
3. Recreate the lock file, or install in a different project so that dependencies points to the mirror
4. Now references of all npm packages point to the mirror, except the git+ssh webpack dependency

I have created a git repository showcasing the issue with the reproduction steps. See that the lock file has all dependencies pointing to the offline mirror except this one https://github.com/albertfdp/yarn-offline-cache-ssh/blob/master/yarn.lock#L1673

What is the expected behavior?

That the lock file points to the offline mirror tarball. This feature is probably used in combination with the install --offline flag, so access to the network makes it fail.

Please mention your node.js, yarn and operating system version.

• node 6.9.1
• yarn 0.18.1
• macOS 10.12

Note

23/02/2017 Edited to reflect a better description of the issue

[staugaard]

Getting this fixed would be especially nice for private git dependencies. Then CI and deployments systems don't need to have git credentials.

[pauloppenheim-gingerlabs]

I am investigating a workaround, by untar-ing the yarn-offline-mirror copy of the git ref into the cache myself. Notably, a normal cached version has a .yarn-metadata.json file. Here's how to workaround for now (you might need to change some paths):

yarn_cache_untar_hack() {
  # BAD BUG HACK temporarily manually fill the yarn cache with our git deps
  local YARNCACHEDIR=$1
  local project=$2
  local hashcode=$3
  local version=$4

  local cachedest="npm-$project-$version-$hashcode"
  local vendorsrc="$project.git-$hashcode"
  echo "yarn_cache_untar_hack $YARNCACHEDIR $cachedest $vendorsrc"
  mkdir -p "$YARNCACHEDIR/$cachedest"
  tar -C "$YARNCACHEDIR/$cachedest" -xf "vendor/npm-packages-offline-cache/$vendorsrc"
  cat > $YARNCACHEDIR/$cachedest/.yarn-metadata.json <<EOF
{
  "artifacts": [],
  "remote": {
    "resolved": "git+ssh://[email protected]/Ginger-Labs/$project.git#$hashcode",
    "type": "git",
    "reference": "git+ssh://[email protected]/Ginger-Labs/$project.git",
    "hash": "$hashcode",
    "registry": "npm"
  },
  "registry": "npm",
  "hash": "$hashcode"
}
EOF
}

I started to look into where the bug might be in yarn, but couldn't locate the source of the logic that decides to fill the cache from git when offline quickly enough.

[albertfdp]

I added a failing test for this https://github.com/yarnpkg/yarn/pull/2775

[agibralter]

Any progress on this? I'm having a lot of issues with yarn's offline mirror: https://github.com/yarnpkg/yarn/issues/3154...

[kavi87]

It works without the pruning option, see #3154

[yukw777]

Just checking in to see if there's any progress on this. As @staugaard mentioned, this would help a lot with private repos.

[BYK]

This should be fixed after Yarn 1.0. Since the issue is quite old and there are no clear repro steps, I'm closing it. Please file a new issue with reproduction steps if you think this is a problem for you.