yarn does not ensure offline cache is in a good state after install

[agibralter]

(Updated)

If someone forgets to commit a file in the offline mirror by mistake, running yarn again does not fix the problem. You have to run yarn install --force. I guess this is just confusing because Ruby's bundler ensures that the offline cache is always in a good state after every run. Wouldn't that be nice behavior?

Previous issue description:
I've written a script here that demonstrates this behavior: https://github.com/agibralter/yarn-test

Basically, yarn will only add packages to the offline cache when it is installing the package for the first time.

This is frustrating for large teams:

• Let's say someone has a package installed already (npm install package-name).
• That person decides this would be a nice package to add for the project and adds it to package.json.
• That person runs yarn install.
• The person checks in the change to the repo, but the offline cache version of the package isn't there!
• Chaos ensures.

Basically we've gotten around this by making everyone run a yarn wrapper script we've written called yarn.sh. This script runs yarn install --force to make sure that the offline cache is in the right state.

[arcanis]

I'm not sure I understand - your testcase is basically this, is that correct?:

yarn install     # ok, classnames is put into the offline mirror
rm -rf cache
yarn install     # not ok, classnames should be put into the offline mirror

Is that correct? If this is the case, then this is to be expected - Yarn tries to be smart and to only run if it detects that something changed in the package.json or yarn.lock since the last time the node_modules has been updated.

What I don't understand is how you get from:

That person runs yarn install.

to

The person checks in the change to the repo, but the offline cache version of the package isn't there!

If you've added the package to the package.json file, then Yarn should pick up the change and run the full install, populating the offline mirror in the process. So the testcase should be:

rm -rf foo

(
    mkdir -p foo && cd foo

    echo 'yarn-offline-mirror "./cache"' > .yarnrc

    echo '{}' > package.json
    ~/yarn/bin/yarn

    echo '{"dependencies":{"classnames":"*"}}' > package.json
    ~/yarn/bin/yarn

    ls cache
)

And it seems to work correctly.

[agibralter]

Ah you're right—actually the issue is more this: if someone forgets to commit the file in the offline mirror by mistake, running yarn again does not fix the problem. You have to run yarn install --force. I guess this is just confusing because Ruby's bundler ensures that the offline cache is always in a good state after every run. Wouldn't that be nice behavior? I guess this is more of a feature request then? 😄 🙏

[BYK]

@agibralter I think you are on to something nice. Would you mind clarifying your feature request so someone may start working on it if they are interested? This also covers you if you are up for submitting a PR :)

[agibralter]

I updated the description above. I'm not sure I'll be able to submit a PR anytime soon... any hints on where to get started?

[siebertm]

You have to run yarn install --force

actually, not (at least not in my case). I had to run yarn cache clean; yarn install --force, as yarn tries to be REALLY smart on my and takes the package it can't find in the offline cache (in ./cache) from the "global" cache (~/.cache/yarn)

[homburg]

We'ere using the following Makefile target to avoid blowing away ~/.cache/yarn:

update-yarn-cache:
    set -e ;\
    TMP_YARN_CACHE_FOLDER=$$(mktemp -d) ;\
    YARN_CACHE_FOLDER=$$TMP_YARN_CACHE_FOLDER yarn --force ;\
    rm -rf $$TMP_YARN_CACHE_FOLDER