Yarn: Yarn workspace not hoisting dependencies correctly

[jackyef]

Do you want to request a feature or report a bug?
Possible bug.

What is the current behavior?
Yarn workspace seems to be counting the references from external modules to decide whether to hoist a module or not. This causes an unexpected behavior in my opinion. A minimal reproduction can be looked at this repository.

In the repo, there are 3 packages, called serviceA, serviceB, and serviceC.

1. serviceA has dependency to [email protected], [email protected] and [email protected]
2. serviceB and serviceC have dependency to [email protected], [email protected]

After running yarn install in the project root, we end up with [email protected] and [email protected] in the root node_modules. This mismatching version can cause problems in react apps.

What is the expected behavior?
I expected that [email protected] and [email protected] to be installed in serviceA's node_modules, and [email protected], [email protected] to be hoisted to root node_modules because there are more local packages that depends on them.

Even when I run yarn why react, yarn itself seems to be expecting [email protected] to be hoisted, not [email protected].

yarn why v1.17.3
[1/4] 🤔  Why do we have the module "react"...?
[2/4] 🚚  Initialising dependency graph...
[3/4] 🔍  Finding dependency...
[4/4] 🚡  Calculating file sizes...
=> Found "[email protected]"
info Has been hoisted to "react"
info Reasons this module exists
   - "workspace-aggregator-0de2b996-5873-43fd-bf58-f1a0a092304c" depends on it
   - Hoisted from "_project_#serviceb#react"
   - Hoisted from "_project_#servicec#react"
info Disk size without dependencies: "248KB"
info Disk size with unique dependencies: "436KB"
info Disk size with transitive dependencies: "524KB"
info Number of shared dependencies: 3
=> Found "servicea#[email protected]"
info This module exists because "_project_#servicea" depends on it.
✨  Done in 0.13s.

Please mention your node.js, yarn and operating system version.
Node version: 10.16.2
Yarn version: 1.17.3
OS version: macOS 10.14.6

[jackyef]

Sorry, bumping this issue to keep it alive.

[exogen]

I'm seeing both incorrect resolution and hoisting as well with at least v1.17.3, v1.18.0, and v1.19.0.

packages/demo-app/package.json declares a dep on next: 9.0.7 (exact).
packages/next-server/package.json declares a peerDep and devDep on next: ^9.0.0.

First off they both should resolve to 9.0.7, because that satisfies ^9.0.0 just fine. Instead, version 9.0.0 is also installed to satisfy next-server for some reason.

Additionally, Yarn's own output of yarn list does not match how it was actually hoisted.

├─ @techstyle/[email protected]
│  └─ [email protected]
└─ [email protected]

In reality, 9.0.0 was placed in the root and 9.0.7 was placed in demo-app. Yarn is confused about both what should be installed and the tree it actually installed.

[attilavago]

Having the same issue with @material-ui
I seem to be ending up with it in the root of my monorepo for no good reason. 😮

[faudisio]

Same issue here. Can this issue be assigned for resolution?

[christ776]

Same issue here as well.

[Siegrift]

I have also problem with dependency hoisting. From https://classic.yarnpkg.com/en/docs/workspaces/ I assumed that the hoisting is done only when all of the modules specify the same dependency...

In my case I have two workspaces A and B. I want to add @types/lodash to A, but it is automatically hoisted and used in B which is just wrong (My intention is to have this dependency only for A).

For now, my solution is to forbid hoisting at all. E.g. change "workspaces": ["A", "B"] to:

  "workspaces": {
    "packages": [
      "A",
      "B"
    ],
    "nohoist": [
      "**"
    ]
  },

I would say this workaround is a fix for the problem above...

Am I misunderstanding something? Or this is intended behaviour? If so, can you explain the reasoning behind this?

[Fallup]

I have also problem with dependency hoisting. From https://classic.yarnpkg.com/en/docs/workspaces/ I assumed that the hoisting is done only when _all_ of the modules specify the same dependency...

In my case I have two workspaces A and B. I want to add @types/lodash to A, but it is automatically hoisted and used in B which is just wrong (My intention is to have this dependency only for A).

For now, my solution is to forbid hoisting at all. E.g. change "workspaces": ["A", "B"] to:

  "workspaces": {
    "packages": [
      "A",
      "B"
    ],
    "nohoist": [
      "**"
    ]
  },

I would say this workaround is a fix for the problem above...

Am I misunderstanding something? Or this is intended behavior? If so, can you explain the reasoning behind this?

@Siegrift According to the issue it obviously does not matter whether all packages have the dependency or just one or few if external references are taken into account.
Currently the best approach is to imo assume that Yarn workspaces can hoist anything (also because the hoisting algorithm can change in the future).

Using noHoist is completely valid way of preventing the issue you mentioned. noHoist of everything is a brute force (but valid solution), you just loose the disc space saving.
I mostly prefer to be selective of packages which should not get hoisted:

  "workspaces": {
    "packages": [
      "packages/*"
    ],
    "nohoist": [
      "**/@types/lodash",
      "**/@types/lodash/**"
    ]

The bigger problem is then preventing Yarn from installing deps for other packages when you want to build e.g. one package on your CI (this is related https://github.com/yarnpkg/yarn/issues/4099)

But it would be great if someone one could clarify the hoisting strategy.

[Siegrift]

@Fallup Yeah, there are a few specific points that would be nice to address:
1) Introduction to workspaces should mention this explicitly
2) I don't understand why yarn can't hoist only if the dependency is exactly the same (or when the major version is the same similarly how node_modules are resolved). But this is basically clarifying the resolution strategy you mention.

With selective approach you never know if you workspace doesn't accidentally pick up unwanted dependency. I will gladly trade off space for this guarantee. And I guess it also resolves #4099, because the workspace has all of it's modules installed inside it's node_modules.

[Fallup]

@Siegrift Possible issues should be definitely communicated in a clearer, more explicit way. Although they were partially mentioned in Introduction to workspaces - limitations and caveats first point:

The package layout will be different between your workspace and what your users will get (the workspace dependencies will be hoisted higher into the filesystem hierarchy). Making assumptions about this layout was already hazardous since the hoisting process is not standardized, so theoretically nothing new here. If you encounter issues, try using the nohoist option

I'd also suggest to read Dependencies done right which explains few issues related to hoisting which might partially answer your 2. question.
Still I wish the hoisting behavior would be described a bit clearer as it is in case of Lerna.

Regarding the selective approach, this is the quote directly from yarn nohoist blog:

While nohoist is useful, it does come with drawbacks. The most obvious one is the nohoist modules could be duplicated in multiple locations, denying the benefit of hoisting mentioned above. Therefore, we recommend to keep nohoist scope as small and explicit as possible in your project.

Nohoisting everything does not resolve #4099 as "nohoist everything" does not equal to installing "one package in isolation" - it rather install all packages, but with deps. not getting hoisted to the root.

Hoisting has always been an issue and if your monorepo is small enough and you can afford it, then nohoisting everything might save you a few headaches.

[eliw00d]

I just ran into some strange issues hoisting with the following:

structure

apps/mobile
apps/web
libs/shared

package.json

  "workspaces": {
    "nohoist": [
      "**/react-native",
      "**/react-native/**",
      "**/react-native-*",
      "**/*-react-native"
    ],
    "packages": [
      "apps/*",
      "libs/*"
    ]
  },

All three have the same version of react as a dependency. libs/shared is meant to be transpiled by babel and used as a package in apps/*.

react is correctly hoisted for apps/web and libs/shared, but not hoisted for apps/mobile.

yarn why v1.22.4
[1/4] 🤔  Why do we have the module "react"...?
[2/4] 🚚  Initialising dependency graph...
[3/4] 🔍  Finding dependency...
[4/4] 🚡  Calculating file sizes...
=> Found "[email protected]"
info Reasons this module exists
   - "_project_#mobile" depends on it
   - Hoisted from "_project_#mobile#react"
   - Hoisted from "_project_#web#react"
   - Hoisted from "_project_#shared#react"
info Disk size without dependencies: "244KB"
info Disk size with unique dependencies: "432KB"
info Disk size with transitive dependencies: "520KB"
info Number of shared dependencies: 5
✨  Done in 0.93s.

This causes invalid hook errors due to having multiple react instances.

I was able to get it to work by moving react to peerDependencies, but that seems like something I should not have to do.

Any thoughts?

[Fallup]

I just ran into some strange issues hoisting with the following:

structure

apps/mobile
apps/web
libs/shared

package.json

  "workspaces": {
    "nohoist": [
      "**/react-native",
      "**/react-native/**",
      "**/react-native-*",
      "**/*-react-native"
    ],
    "packages": [
      "apps/*",
      "libs/*"
    ]
  },

All three have the same version of react as a dependency. libs/shared is meant to be transpiled by babel and used as a package in apps/*.

react is correctly hoisted for apps/web and libs/shared, but not hoisted for apps/mobile.

yarn why v1.22.4
[1/4] 🤔  Why do we have the module "react"...?
[2/4] 🚚  Initialising dependency graph...
[3/4] 🔍  Finding dependency...
[4/4] 🚡  Calculating file sizes...
=> Found "[email protected]"
info Reasons this module exists
   - "_project_#mobile" depends on it
   - Hoisted from "_project_#mobile#react"
   - Hoisted from "_project_#web#react"
   - Hoisted from "_project_#shared#react"
info Disk size without dependencies: "244KB"
info Disk size with unique dependencies: "432KB"
info Disk size with transitive dependencies: "520KB"
info Number of shared dependencies: 5
✨  Done in 0.93s.

This causes invalid hook errors due to having multiple react instances.

I was able to get it to work by moving react to peerDependencies, but that seems like something I should not have to do.

Any thoughts?

I was in the same situation and went for noHoist of react as well.

[eliw00d]

Is it better to only have react, or any other shared dependency, in the package.json of libs/shared knowing it will be hoisted?

[JaosnHsieh]

fixed electron-builder install-app-deps error by nohoist in a mono repository built with lerna.

commit