Yarn: YarnPnP breaks workspace links when packages have bin files

[KwanMan]

Do you want to request a feature or report a bug?
Bug

What is the current behavior?
In yarn workspaces, if one package depends on another package with bin files (and present as a package in the workspace), the bin file is symlinked to the dependent package so it can be run. This behaviour is broken when PnP is turned on. The bin file is no longer symlinked, which is expected as it should use the PnP resolver to find them, but it doesn't seem to be able to locate the binary.

If the current behavior is a bug, please provide the steps to reproduce.
Clone this repo: https://github.com/KwanMan/yarn-pnp-workspaces

# bootstrap
cd ./with-pnp
yarn

cd ./packages/package-a
## bin files are not symlinked as expected, as doing `yarn run [cmd]` should use the pnp resolver
ls node_modules/.bin

yarn run build ## but it doesn't resolve correctly and can't find the bin file

What is the expected behavior?
It should correctly resolve bin files of packages in the same workspace

The linked repo contains the same setup without pnp which works

# bootstrap
cd ./without-pnp
yarn

cd ./packages/package-a
## bin files from `build-script` is symlinked in so we can use them here
ls node_modules/.bin

yarn run build ## runs fine

Please mention your node.js, yarn and operating system version.

Node v10.11.0
Yarn 1.12.1
MacOS High Sierra 10.13.6

[arcanis]

The node_modules doesn't exist at all when operating under PnP. To run a script, you must use yarn run <bin-name> which will automatically run the binary from whatever is its true location with the PnP hook correctly setup.

In case you need to call it yourself for some reason, using yarn bin <bin-name> --silent will print the path to the real binary (but you then have to make sure to call it using the yarn node command, or to setup the PnP hook yourself).

[KwanMan]

@arcanis I'm trying to run it with yarn run already but it can't find the binary

[lroszak]

We ran into same issue wen we switched on TFS builds to Yarn 1.12.1. Links in .bin directory were not working properly, and we had problems with recursive search through folders in solution.
Switching back to 1.10.1 solved it for us

[arcanis]

@lroszak I think yours is a different issue, already solved in the 1.12.3 release (released yesterday). Can you double check?

[lroszak]

@arcanis you are right, it works in 1.12.1.

[mufasa71]

@arcanis With latest release when I try to run babel-node I'm getting

/home/foo/.cache/yarn/v4/[email protected]/node_modules/@babel/node/lib/babel-node.js:98
    if (err.code !== "MODULE_NOT_FOUND") throw err;
                                         ^

Error: Package "@babel/[email protected]" (via "/home/eagle/.cache/yarn/v4/[email protected]/node_modules/@babel/node/lib/babel-node.js") is trying to require the package "kexec" (via "kexec") without it being listed in its dependencies (@babel/polyfill, @babel/register, commander, fs-readdir-recursive, lodash, output-file-sync, v8flags, @babel/node)
    at makeError (/home/foo/app-pnp/.pnp.js:55:17)
    at Object.resolveToUnqualified (/home/foo/app-pnp/.pnp.js:1462:17)
    at Object.resolveRequest (/home/foo/app-pnp/.pnp.js:1533:31)
    at Function.Module._resolveFilename (/home/foo/app-pnp/.pnp.js:1713:30)
    at Function.Module._load (/home/foo/app-pnp/.pnp.js:1631:31)
    at Module.require (internal/modules/cjs/loader.js:643:17)
    at require (internal/modules/cjs/helpers.js:22:18)
    at /home/foo/.cache/yarn/v4/[email protected]/node_modules/@babel/node/lib/babel-node.js:94:19
    at /home/foo/.cache/yarn/v4/npm-v8flags-3.1.1-42259a1461c08397e37fe1d4f1cfb59cad85a053/node_modules/v8flags/index.js:131:14
    at /home/foo/.cache/yarn/v4/npm-v8flags-3.1.1-42259a1461c08397e37fe1d4f1cfb59cad85a053/node_modules/v8flags/index.js:39:14
error Command failed with exit code 1.

Also husky and nodemon are not working properly with pnp enabled.

[KwanMan]

@arcanis Can you confirm if it's a bug I'm seeing or am I doing something wrong? Happy to look into it if there's a bug but need to know if I'm understanding the intended behaviour

[goodmind]

I tried yarn unplug with binary package, but there is no .bin folder in .pnp/unplugged/npm-.../node_modules

[ngryman]

I'm running into the same issue with Yarn 1.15.2 with the repo provided by @KwanMan:

$ yarn build
yarn run v1.15.2
$ build-script
/bin/sh: build-script: command not found
error Command failed with exit code 127.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

yarn bin doesn't work either:

$ yarn bin build-script
error Couldn't find a binary named "build-script"

@arcanis Do you have some news about this issue? We would like to migrate our monorepo to PnP and this particular issue is one of the blockers for us right now.

If it's not a priority for the Yarn team, I would be glad to try to fix it and open a PR for that. Some context on binaries resolution with PnP would greatly help if so. Thanks.

[arcanis]

We would definitely welcome a PR! However, note that most of the PnP development now occurs in the experimental Yarn 2 repository, which benefits from an all-new implementation with much less workarounds than what was needed on the v1.

If you want to use PnP, I'd suggest to give a shot at the v2 (which uses it by default). Plus it would give us valuable feedback regarding how simple are migrations and how we could make them simpler (I unfortunately haven't wrote the migration guide yet 😔).

So if you get the chance, can you check whether this use case is satisfied by the v2 and open an issue on its repo if it doesn't?