yarn install --offline with jenkins

[vesterij]

build environment: linux-x64
node: v7.8.0
yarn version v0.24.5

Hi,

I am trying to make build that runs with Jenkins in server that has no access to public internet.
Building local npm cache works and offline build works on my machine but in the offline build server this happens

09:04:56 + yarn install --verbose --offline
09:04:56 yarn install v0.24.5
09:04:56 verbose 0.169 Found configuration file "/var/lib/jenkins/jobs/MYJOB/jobs/MYJOB-offline/workspace/.yarnrc".

09:04:56 verbose 0.174 current time: 2017-05-22T06:04:56.948Z

09:04:57 [1/4] Resolving packages...
09:04:57 [2/4] Fetching packages...

09:04:58 verbose 2.075 Error: Can't make a request in offline mode
09:04:58     at RequestManager.request (/usr/share/yarn/lib/yarn-cli.js:56073:59)
09:04:58     at NpmRegistry.request (/usr/share/yarn/lib/yarn-cli.js:30492:32)
09:04:58     at TarballFetcher.fetchFromExternal (/usr/share/yarn/lib/yarn-cli.js:54412:21)
09:04:58     at /usr/share/yarn/lib/yarn-cli.js:54470:29
09:04:58     at Generator.next (<anonymous>)
09:04:58     at step (/usr/share/yarn/lib/yarn-cli.js:91:30)
09:04:58     at /usr/share/yarn/lib/yarn-cli.js:102:13
09:04:58 error Can't make a request in offline mode

If I go to the workspace dir and run the command again with sudo, it works.

If I run command once without offline I get the lines

09:14:01 verbose 1.611 Copying "/var/lib/jenkins/.cache/yarn/v1/npm-@gulp-sourcemaps/identity-map-1.0.1-cfa23bc5840f9104ce32a65e74db7e7a974bbee1/.yarn-tarball.tgz" to "/var/lib/jenkins/jobs/MYJOB/jobs/MYJOB-offline/workspace/npm-packages-offline-cache/@gulp-sourcemaps-identity-map-1.0.1.tgz". 09:14:01 verbose 1.621 Copying "/var/lib/jenkins/.cache/yarn/v1/npm-@gulp-sourcemaps/map-sources-1.0.0-890ae7c5d8c877f6d384860215ace9d7ec945bda/.yarn-tarball.tgz" to "/var/lib/jenkins/jobs/MYJOB/jobs/MYJOB-offline/workspace/npm-packages-offline-cache/@gulp-sourcemaps-map-sources-1.0.0.tgz". 09:14:02 warning [email protected]: The platform "linux" is incompatible with this module.

After this --offline also works without sudo or network

EDIT: it seems that Jenkins also caches yarn so after one install with internet connection everything was installed from Jenkins cache instead of offline cache

EDIT2: While running with sudo yarn was also using different cache instead of yarn-offline-mirror "./npm-packages-offline-cache". I'll have to test this with different environment later.

[vesterij]

It seems to be that problem is either with "yarn-offline-mirror-pruning true" option or Yarn's cache

When some packages get newer version and you just run "yarn" new packages are fetch for Yarn's cache but not for the yarn-offline-mirror.

Then yarn-offline-mirror-pruning removes old packages but it doesn't replace them with new ones and you end up with missing packages from yarn-offline-mirror.

You can fix this by removing everything and rebuilding your offline-mirror again.

It would be nice if this would work automatically but now I know what to look for and how to fix it, so I will close this issue.

[brianhorst]

I just started with yarn to be able to use offline mode, but ran into the same issue you describe here. It doesn't seem like this issue is addressed in any way. Do you just need to constantly rebuild the offline mirror? Maybe this issue should be re-opened if that's the case.

[vesterij]

In my case CI build happens with internet and I only need to rebuild offline mirror just before pushing version to production environment. So I just rebuild offline mirror from scratch roughly twice a month.

But I agree that this is not working properly and should be fixed or at least explained better in documentation.

I don't have time to study this or try to narrow problem down. I'll reopen the issue.

[brianhorst]

Hey I looked into this a bit more and my issue was caused by a scoped package not being saved in the offline cache. See #3509

[vesterij]

It seems that while buildin your mirror you have to use
yarn-offline-mirror-pruning false

Otherwise some packages are missing, even if you build everything from scratch

If you run following with and without yarn-offline-mirror-pruning you might get different amout of packages in your npm-packages-offline-cache folder

rm -rf node_modules npm-packages-offline-cache
yarn cache clean
yarn install

I was using latest version "yarn install v0.24.6"

[bestander]

Tracking the prune issue here https://github.com/yarnpkg/yarn/issues/3509

[kappa-gooner]

I'm hitting this issue regardless of whether my yarn-offline-mirror-pruning setting. Any pointers?