Berry: [Bug] yarn add --peer after --dev doesn't add peer dependency

[andreialecu]

• [ ] I'd be willing to implement a fix

Describe the bug

I have a library that depends on rxjs as a peer dependency, but it also has tests inside it.

If I add rxjs as a normal dependency, tests run properly. If instead I run:

yarn add --peer --dev rxjs

My tests will complain about: Cannot find module 'rxjs' from '...'

Note that I'm also using typescript and ts-jest.

Ideally, the dependency should be available to both peer and dev dependencies.

To Reproduce

``js repro const {promises: {readFile}} = require(fs`);

await packageJsonAndInstall({
devDependencies: {},
});

await yarn(add, --dev, rxjs);
await yarn(add, --peer, rxjs);

const pkgJson = JSON.parse(await readFile(package.json, utf8));
expect(pkgJson).toHaveProperty(peerDependencies);
expect(pkgJson).toHaveProperty(devDependencies);
expect(pkgJson.peerDependencies).toHaveProperty(rxjs);
expect(pkgJson.devDependencies).toHaveProperty(rxjs);
```

Additional context

This is also a bug in yarn v1: https://github.com/yarnpkg/yarn/issues/5287

[yarnbot]

The reproduction case in your issue seems broken (ie it neither pass nor fail due to throwing an unmanaged exception):

Error: Command failed: /usr/bin/node /github/workspace/scripts/actions/../run-yarn.js add --peer --dev rxjs

Internal Error: No project found in the initial directory
    at Function.find (/github/workspace/packages/yarnpkg-core/sources/Project.ts:83:13)
    at AddCommand.find [as execute] (/github/workspace/packages/plugin-essentials/sources/commands/add.ts:79:48)
    at processTicksAndRejections (internal/process/task_queues.js:93:5)
    at AddCommand.validateAndExecute (/github/workspace/.yarn/cache/clipanion-npm-2.1.5-aef0ebbd62-1.zip/node_modules/clipanion/lib/advanced/Command.js:182:26)
    at Cli.run (/github/workspace/.yarn/cache/clipanion-npm-2.1.5-aef0ebbd62-1.zip/node_modules/clipanion/lib/advanced/Cli.js:85:24)
    at Cli.runExit (/github/workspace/.yarn/cache/clipanion-npm-2.1.5-aef0ebbd62-1.zip/node_modules/clipanion/lib/advanced/Cli.js:94:28)

    at ChildProcess.exithandler (child_process.js:295:12)
    at ChildProcess.emit (events.js:210:5)
    at maybeClose (internal/child_process.js:1021:16)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:283:5)

Remember: any non-Jest exceptions will cause the test to be reported as broken. If you expect something to pass without throwing, you must wrap it into something like await expect(...).resolves.toBeTruthy(). If you instead expect something to throw, you need to wrap it into await expect(...).rejects.toThrow().

[yarnbot]

The reproduction case in your issue seems broken (ie it neither pass nor fail due to throwing an unmanaged exception):

Error: expect(received).toHaveProperty(path)

Matcher error: received value must not be null nor undefined

Received has value: undefined
    at module.exports (evalmachine.<anonymous>:12:33)
    at async /github/workspace/.yarn/cache/@arcanis-sherlock-npm-1.0.36-028b2a4919-1.zip/node_modules/@arcanis/sherlock/lib/executeRepro.js:56:13
    at async executeInTempDirectory (/github/workspace/.yarn/cache/@arcanis-sherlock-npm-1.0.36-028b2a4919-1.zip/node_modules/@arcanis/sherlock/lib/executeRepro.js:17:16)
    at async Object.executeRepro (/github/workspace/.yarn/cache/@arcanis-sherlock-npm-1.0.36-028b2a4919-1.zip/node_modules/@arcanis/sherlock/lib/executeRepro.js:24:12)
    at async ExecCommand.execute (/github/workspace/.yarn/cache/@arcanis-sherlock-npm-1.0.36-028b2a4919-1.zip/node_modules/@arcanis/sherlock/lib/commands/exec.js:25:38)
    at async ExecCommand.validateAndExecute (/github/workspace/.yarn/cache/clipanion-npm-2.0.0-rc.16-b9444aaf89-1.zip/node_modules/clipanion/lib/advanced/Command.js:161:26)
    at async Cli.run (/github/workspace/.yarn/cache/clipanion-npm-2.0.0-rc.16-b9444aaf89-1.zip/node_modules/clipanion/lib/advanced/Cli.js:74:24)
    at async Cli.runExit (/github/workspace/.yarn/cache/clipanion-npm-2.0.0-rc.16-b9444aaf89-1.zip/node_modules/clipanion/lib/advanced/Cli.js:83:28)

Remember: any non-Jest exceptions will cause the test to be reported as broken. If you expect something to pass without throwing, you must wrap it into something like await expect(...).resolves.toBeTruthy(). If you instead expect something to throw, you need to wrap it into await expect(...).rejects.toThrow().

[yarnbot]

This issue reproduces on master:

Error: expect(received).toHaveProperty(path)

Expected path: "devDependencies"
Received path: []

Received value: {"peerDependencies": {"rxjs": "*"}}
    at module.exports (evalmachine.<anonymous>:12:17)
    at async /github/workspace/.yarn/cache/@arcanis-sherlock-npm-1.0.36-028b2a4919-1.zip/node_modules/@arcanis/sherlock/lib/executeRepro.js:56:13
    at async executeInTempDirectory (/github/workspace/.yarn/cache/@arcanis-sherlock-npm-1.0.36-028b2a4919-1.zip/node_modules/@arcanis/sherlock/lib/executeRepro.js:17:16)
    at async Object.executeRepro (/github/workspace/.yarn/cache/@arcanis-sherlock-npm-1.0.36-028b2a4919-1.zip/node_modules/@arcanis/sherlock/lib/executeRepro.js:24:12)
    at async ExecCommand.execute (/github/workspace/.yarn/cache/@arcanis-sherlock-npm-1.0.36-028b2a4919-1.zip/node_modules/@arcanis/sherlock/lib/commands/exec.js:25:38)
    at async ExecCommand.validateAndExecute (/github/workspace/.yarn/cache/clipanion-npm-2.0.0-rc.16-b9444aaf89-1.zip/node_modules/clipanion/lib/advanced/Command.js:161:26)
    at async Cli.run (/github/workspace/.yarn/cache/clipanion-npm-2.0.0-rc.16-b9444aaf89-1.zip/node_modules/clipanion/lib/advanced/Cli.js:74:24)
    at async Cli.runExit (/github/workspace/.yarn/cache/clipanion-npm-2.0.0-rc.16-b9444aaf89-1.zip/node_modules/clipanion/lib/advanced/Cli.js:83:28)

[yarnbot]

This issue reproduces on master:

Error: expect(received).toHaveProperty(path)

Expected path: "devDependencies"
Received path: []

Received value: {"peerDependencies": {"rxjs": "*"}}
    at module.exports (evalmachine.<anonymous>:12:17)
    at async /github/workspace/.yarn/cache/@arcanis-sherlock-npm-1.0.36-028b2a4919-1.zip/node_modules/@arcanis/sherlock/lib/executeRepro.js:56:13
    at async executeInTempDirectory (/github/workspace/.yarn/cache/@arcanis-sherlock-npm-1.0.36-028b2a4919-1.zip/node_modules/@arcanis/sherlock/lib/executeRepro.js:17:16)
    at async Object.executeRepro (/github/workspace/.yarn/cache/@arcanis-sherlock-npm-1.0.36-028b2a4919-1.zip/node_modules/@arcanis/sherlock/lib/executeRepro.js:24:12)
    at async ExecCommand.execute (/github/workspace/.yarn/cache/@arcanis-sherlock-npm-1.0.36-028b2a4919-1.zip/node_modules/@arcanis/sherlock/lib/commands/exec.js:25:38)
    at async ExecCommand.validateAndExecute (/github/workspace/.yarn/cache/clipanion-npm-2.0.0-rc.16-b9444aaf89-1.zip/node_modules/clipanion/lib/advanced/Command.js:161:26)
    at async Cli.run (/github/workspace/.yarn/cache/clipanion-npm-2.0.0-rc.16-b9444aaf89-1.zip/node_modules/clipanion/lib/advanced/Cli.js:74:24)
    at async Cli.runExit (/github/workspace/.yarn/cache/clipanion-npm-2.0.0-rc.16-b9444aaf89-1.zip/node_modules/clipanion/lib/advanced/Cli.js:83:28)

[andreialecu]

Note that trying to add the dev dependency once the peer one is in package.json also doesn't work:

``js repro const {promises: {readFile}} = require(fs`);

await packageJsonAndInstall({
peerDependencies: {"rxjs": "*"},
});

await yarn(add, --dev, rxjs);

const pkgJson = JSON.parse(await readFile(package.json, utf8));
expect(pkgJson).toHaveProperty(peerDependencies);
expect(pkgJson).toHaveProperty(devDependencies);

expect(pkgJson.peerDependencies).toHaveProperty(rxjs);
expect(pkgJson.devDependencies).toHaveProperty(rxjs);
```

In yarn v1 it was possible to force the dependency to be installed to node_modules and get running (usually by using npm instead), but with PnP this workaround doesn't seem easy to do any more.

Edit: Seems like sherlock doesn't run out of comments. Not sure how to run this repro.

[arcanis]

yarn add only adds to a single field at a time; I'm not sure for the second repro you put, it seems to work for me:

[yarnbot]

We couldn't reproduce your issue (all the assertions passed on master).

[arcanis]

^ I've updated your original repro to call yarn add twice, and things seem to work fine. Feel free to fix the repro and reopen if I'm missing something 🙂

[yarnbot]

We couldn't reproduce your issue (all the assertions passed on master).

[yarnbot]

This issue reproduces on master:

Error: expect(received).toHaveProperty(path)

Expected path: "peerDependencies"
Received path: []

Received value: {"devDependencies": {"rxjs": "^6.5.4"}}
    at module.exports (evalmachine.<anonymous>:12:17)
    at async /github/workspace/.yarn/cache/@arcanis-sherlock-npm-1.0.36-028b2a4919-1.zip/node_modules/@arcanis/sherlock/lib/executeRepro.js:56:13
    at async executeInTempDirectory (/github/workspace/.yarn/cache/@arcanis-sherlock-npm-1.0.36-028b2a4919-1.zip/node_modules/@arcanis/sherlock/lib/executeRepro.js:17:16)
    at async Object.executeRepro (/github/workspace/.yarn/cache/@arcanis-sherlock-npm-1.0.36-028b2a4919-1.zip/node_modules/@arcanis/sherlock/lib/executeRepro.js:24:12)
    at async ExecCommand.execute (/github/workspace/.yarn/cache/@arcanis-sherlock-npm-1.0.36-028b2a4919-1.zip/node_modules/@arcanis/sherlock/lib/commands/exec.js:25:38)
    at async ExecCommand.validateAndExecute (/github/workspace/.yarn/cache/clipanion-npm-2.0.0-rc.16-b9444aaf89-1.zip/node_modules/clipanion/lib/advanced/Command.js:161:26)
    at async Cli.run (/github/workspace/.yarn/cache/clipanion-npm-2.0.0-rc.16-b9444aaf89-1.zip/node_modules/clipanion/lib/advanced/Cli.js:74:24)
    at async Cli.runExit (/github/workspace/.yarn/cache/clipanion-npm-2.0.0-rc.16-b9444aaf89-1.zip/node_modules/clipanion/lib/advanced/Cli.js:83:28)

[andreialecu]

@arcanis interesting, it seems the order in which --peer and --dev installs are ran matters. Adding the --dev one first one makes the --peer one not do anything.

// this doesn't work
await yarn(`add`, `--dev`, `rxjs`);
await yarn(`add`, `--peer`, `rxjs`);

// this works
await yarn(`add`, `--peer`, `rxjs`);
await yarn(`add`, `--dev`, `rxjs`);

Would also be nice to throw some sort of error if --peer --dev is ran in the same command.

[yarnbot]

This issue reproduces on master:

Error: expect(received).toHaveProperty(path)

Expected path: "peerDependencies"
Received path: []

Received value: {"devDependencies": {"rxjs": "^6.5.4"}}
    at module.exports (evalmachine.<anonymous>:12:17)
    at async /github/workspace/.yarn/cache/@arcanis-sherlock-npm-1.0.36-028b2a4919-1.zip/node_modules/@arcanis/sherlock/lib/executeRepro.js:56:13
    at async executeInTempDirectory (/github/workspace/.yarn/cache/@arcanis-sherlock-npm-1.0.36-028b2a4919-1.zip/node_modules/@arcanis/sherlock/lib/executeRepro.js:17:16)
    at async Object.executeRepro (/github/workspace/.yarn/cache/@arcanis-sherlock-npm-1.0.36-028b2a4919-1.zip/node_modules/@arcanis/sherlock/lib/executeRepro.js:24:12)
    at async ExecCommand.execute (/github/workspace/.yarn/cache/@arcanis-sherlock-npm-1.0.36-028b2a4919-1.zip/node_modules/@arcanis/sherlock/lib/commands/exec.js:25:38)
    at async ExecCommand.validateAndExecute (/github/workspace/.yarn/cache/clipanion-npm-2.0.0-rc.16-b9444aaf89-1.zip/node_modules/clipanion/lib/advanced/Command.js:161:26)
    at async Cli.run (/github/workspace/.yarn/cache/clipanion-npm-2.0.0-rc.16-b9444aaf89-1.zip/node_modules/clipanion/lib/advanced/Cli.js:74:24)
    at async Cli.runExit (/github/workspace/.yarn/cache/clipanion-npm-2.0.0-rc.16-b9444aaf89-1.zip/node_modules/clipanion/lib/advanced/Cli.js:83:28)

[yarnbot]

This issue reproduces on master:

Error: expect(received).toHaveProperty(path)

Expected path: "peerDependencies"
Received path: []

Received value: {"devDependencies": {"rxjs": "^6.5.4"}}
    at module.exports (evalmachine.<anonymous>:12:17)
    at async /github/workspace/.yarn/cache/@arcanis-sherlock-npm-1.0.36-028b2a4919-1.zip/node_modules/@arcanis/sherlock/lib/executeRepro.js:56:13
    at async executeInTempDirectory (/github/workspace/.yarn/cache/@arcanis-sherlock-npm-1.0.36-028b2a4919-1.zip/node_modules/@arcanis/sherlock/lib/executeRepro.js:17:16)
    at async Object.executeRepro (/github/workspace/.yarn/cache/@arcanis-sherlock-npm-1.0.36-028b2a4919-1.zip/node_modules/@arcanis/sherlock/lib/executeRepro.js:24:12)
    at async ExecCommand.execute (/github/workspace/.yarn/cache/@arcanis-sherlock-npm-1.0.36-028b2a4919-1.zip/node_modules/@arcanis/sherlock/lib/commands/exec.js:25:38)
    at async ExecCommand.validateAndExecute (/github/workspace/.yarn/cache/clipanion-npm-2.0.0-rc.16-b9444aaf89-1.zip/node_modules/clipanion/lib/advanced/Command.js:161:26)
    at async Cli.run (/github/workspace/.yarn/cache/clipanion-npm-2.0.0-rc.16-b9444aaf89-1.zip/node_modules/clipanion/lib/advanced/Cli.js:74:24)
    at async Cli.runExit (/github/workspace/.yarn/cache/clipanion-npm-2.0.0-rc.16-b9444aaf89-1.zip/node_modules/clipanion/lib/advanced/Cli.js:83:28)

[arcanis]

Oh indeed - I think the problem comes from here, the two blocks should likely be inverted so that CLI flags take precedence over the detection heuristic:

https://github.com/yarnpkg/berry/blob/master/packages/plugin-essentials/sources/commands/add.ts#L249-L257

[talentlessguy]

Same happens with installing Next.js. I tried @andreialecu's solution but it still drops a missing webpack dependency error.