Berry: [Bug] PnP Module Resolution Fails in CI

[danielstreit]

Describe the bug

I am bootstrapping a new project to use yarn 2 and am running into an issue running tests in CI.

The test command works as expected locally (OSX).

The CI stage is running in the node:12.18.2 docker image. Here is the error I get:

$ yarn run test
➤ YN0000: Error: Qualified path resolution failed - none of the candidates can be found on the disk.
➤ YN0000: 
➤ YN0000: Source path: /builds/my-repo/.yarn/unplugged/core-js-pure-npm-3.6.5-c634b57d82/node_modules/core-js-pure/stable/object/define-property
➤ YN0000: Rejected candidate: /builds/my-repo/.yarn/unplugged/core-js-pure-npm-3.6.5-c634b57d82/node_modules/core-js-pure/stable/object/define-property
➤ YN0000: Rejected candidate: /builds/my-repo/.yarn/unplugged/core-js-pure-npm-3.6.5-c634b57d82/node_modules/core-js-pure/stable/object/define-property.js
➤ YN0000: Rejected candidate: /builds/my-repo/.yarn/unplugged/core-js-pure-npm-3.6.5-c634b57d82/node_modules/core-js-pure/stable/object/define-property.json
➤ YN0000: Rejected candidate: /builds/my-repo/.yarn/unplugged/core-js-pure-npm-3.6.5-c634b57d82/node_modules/core-js-pure/stable/object/define-property.node
➤ YN0000: 
➤ YN0000: Require stack:
➤ YN0000: - /builds/my-repo/.yarn/cache/@babel-runtime-corejs3-npm-7.10.4-4aab60b0c9-5621e6b5c6.zip/node_modules/@babel/runtime-corejs3/core-js-stable/object/define-property.js
➤ YN0000: - /builds/my-repo/.yarn/cache/xregexp-npm-4.3.0-ffd2086dab-2dcef4888e.zip/node_modules/xregexp/lib/index.js
➤ YN0000: - /builds/my-repo/.yarn/cache/decamelize-npm-3.2.0-a7d59351fd-dbe98e2e5f.zip/node_modules/decamelize/index.js
➤ YN0000: - /builds/my-repo/.yarn/cache/yargs-npm-15.4.0-ff836e3caf-b4163b7e8f.zip/node_modules/yargs/build/lib/usage.js
➤ YN0000: - /builds/my-repo/.yarn/cache/yargs-npm-15.4.0-ff836e3caf-b4163b7e8f.zip/node_modules/yargs/build/lib/yargs.js
➤ YN0000: - /builds/my-repo/.yarn/cache/yargs-npm-15.4.0-ff836e3caf-b4163b7e8f.zip/node_modules/yargs/yargs.js
➤ YN0000: - /builds/my-repo/.yarn/cache/yargs-npm-15.4.0-ff836e3caf-b4163b7e8f.zip/node_modules/yargs/index.js
➤ YN0000: - /builds/my-repo/.yarn/cache/jest-cli-npm-26.1.0-91142dcbda-c520c0040d.zip/node_modules/jest-cli/build/cli/index.js
➤ YN0000: - /builds/my-repo/.yarn/cache/jest-cli-npm-26.1.0-91142dcbda-c520c0040d.zip/node_modules/jest-cli/bin/jest.js
➤ YN0000: - /builds/my-repo/.yarn/cache/jest-npm-26.1.0-bd255a3889-95638b5947.zip/node_modules/jest/bin/jest.js
➤ YN0000:     at internalTools_makeError (/builds/my-repo/.pnp.js:14444:34)
➤ YN0000:     at resolveUnqualified (/builds/my-repo/.pnp.js:15438:13)
➤ YN0000:     at resolveRequest (/builds/my-repo/.pnp.js:15462:14)
➤ YN0000:     at Object.resolveRequest (/builds/my-repo/.pnp.js:15522:26)
➤ YN0000:     at Function.external_module_.Module._resolveFilename (/builds/my-repo/.pnp.js:14676:34)
➤ YN0000:     at Function.external_module_.Module._load (/builds/my-repo/.pnp.js:14540:51)
➤ YN0000:     at Module.require (internal/modules/cjs/loader.js:1026:19)
➤ YN0000:     at require (internal/modules/cjs/helpers.js:72:18)
➤ YN0000:     at Object.<anonymous> (/builds/my-repo/.yarn/cache/@babel-runtime-corejs3-npm-7.10.4-4aab60b0c9-5621e6b5c6.zip/node_modules/@babel/runtime-corejs3/core-js-stable/object/define-property.js:1:18)
➤ YN0000:     at Module._compile (internal/modules/cjs/loader.js:1138:30)
➤ YN0000: Done in 0.54s

yarn run test simply calls the test script in each package:

root/package.json:

  "scripts": {
    "format-check": "yarn workspaces foreach run format-check",
    "lint": "yarn workspaces foreach run lint",
    "test": "yarn workspaces foreach run test"
  },

root/packages/test-package/package.json:

  "scripts": {
    "format-check": "prettier --check .",
    "lint": "eslint './src/**/*.ts'",
    "test": "jest"
  },

The format-check and lint scripts work as expected in CI and locally.

I have poked around the zip archives locally and can see that the expected file does seem to exist.

I'm using the Zero Installs approach, all dependencies are checked into the repo in the .yarn/cache directory.

I'll work on a Sherlock repro, but if you have any quick thoughts on what I'm doing wrong from the error message, I'd appreciate it!

[paul-soporan]

I assume (from a quick look, feel free to correct me if I missed something) that the problem is that you're not running yarn install on CI, which is still needed for unplugged dependencies even when using Zero Installs, because they have build steps that Yarn needs to execute for everything to work correctly.

There are 2 ways to proceed here:
1) Always run the install, including on CI.
2) If you want to avoid running the install, you can just add "dependenciesMeta": { "core-js-pure": { "built": false } } to your top-level manifest, which tells Yarn to ignore core-js-pure's build scripts, which cause it to be unplugged (IIRC core-js and probably also core-js-pure have postinstall scripts that only show a prompt for funding, so it's safe to disable them). You can also toggle the enableScripts configuration option, which makes Yarn not run any build scripts by default. (And you can enable build scripts on a per-package basis via dependencyMeta, if needed)

[danielstreit]

Thanks @paul-soporan !

1. This fixed the issue.
2. I tried various iterations on this but wasn't able to get it to work. I think that I'd prefer this solution to avoid calling yarn install in CI, but it isn't a big deal given the speed of the command now: Done with warnings in 2.02s. I noticed that this is the approach taken by yarn itself, so it seems like it should work. Not sure what I'm doing differently 🤷 . I'm going to punt on it for now.

Thanks for the quick response! And for building a great package manager!