Berry: [Bug] Error: Assertion failed: The dependent package could not be found

Created on 26 Nov 2019  ·  21Comments  ·  Source: yarnpkg/berry
  • [x] I'd be willing to implement a fix

Describe the bug

When I attempt to run yarn in my workspace I receive this error:

Error: Assertion failed: The dependent package could not be found

The error itself is rather unhelpful and doesn't tell you what the dependent package was, however I tweaked the source code a little and got it to tell me that in my case it is react-dom@npm:16.8.6 that it is choking on. The dependency tree for react-dom in my workspace is as follows:

  • Root

    • resolution: "react-dom": "16.8.6"

    • workspaces:



      • a





        • dependencies







          • j




          • l







        • devDependencies:







          • k







        • peerDependencies:







          • "react-dom": ">=16.8.0"









      • b





        • dependencies







          • j




          • l




          • m







        • devDependencies:







          • "react-dom": "^16.8.6"




          • k







        • peerDependencies:







          • "react-dom": ">=16.8.0"









      • c





        • dependencies







          • j




          • l







        • peerDependencies:







          • "react-dom": ">=16.8.0"









      • d





        • dependencies







          • j




          • l







        • devDependencies:







          • k







        • peerDependencies:







          • "react-dom": ">=16.8.0"









      • e





        • dependencies







          • j




          • l







        • devDependencies:







          • "react-dom": "^16.6.8"




          • k







        • peerDependencies:







          • "react-dom": ">=16.8.0"









      • f





        • dependencies







          • j




          • l







        • devDependencies:







          • k







        • peerDependencies:







          • "react-dom": ">=16.8.0"









      • g





        • dependencies







          • j




          • l







        • devDependencies:







          • "react-dom": "^16.8.6"




          • k









      • h





        • dependencies







          • j




          • l







        • devDependencies:







          • k







        • peerDependencies:







          • "react-dom": ">=16.8.0"









      • i





        • dependencies







          • j




          • l







        • devDependencies:







          • "react-dom": "^16.8.6"




          • k







        • peerDependencies:







          • "react-dom": ">=16.8.0"









      • j





        • dependencies:







          • "react-dom": "^16.9.0"









      • k





        • dependencies:







          • "react-dom": "16.8"




          • j









      • l





        • dependencies







          • j







        • peerDependencies:







          • "react-dom": ">=16.8.0"









      • m





        • dependencies







          • j




          • l







        • devDependencies:







          • "react-dom": "^16.8.6"




          • k







        • peerDependencies:







          • "react-dom": ">=16.8.0"









      • n





        • dependencies:







          • "react-dom": "^16.8.6"




          • l









      • o





        • dependencies:







          • "react-dom": "^16.8.6"









      • p





        • dependencies:







          • "react-dom": "^16.8.6"




          • a




          • b




          • c




          • d




          • e




          • f




          • g




          • h




          • i










This is just a single pass

To Reproduce

Run yarn in https://github.com/georeith/berry-609

Screenshots

If applicable, add screenshots to help explain your problem.

Environment if relevant (please complete the following information):

  • OS: [e.g. OSX, Linux, Windows, ...]
  • Node version [e.g. 8.15.0, 10.15.1, ...]
  • Yarn version [e.g. 2.0.0-rc1, ...]

Additional context

We have a lot of peer dependencies in the workspace packages that are provided as dependencies by the root package.json but these spit out a lot of warnings, e.g.,:

➤ YN0002: │ @marvelapp/ui@workspace:src/packages/marvel-ui [85b8c] [56877] doesn't provide react-dom@* requested by @testing-library/react@npm:8.0.1 [92105] [1a0f7]
➤ YN0002: │ @marvelapp/ui@workspace:src/packages/marvel-ui [85b8c] [56877] doesn't provide @babel/core@^7.0.0 requested by babel-jest@npm:24.9.0 [92105] [1a0f7]
➤ YN0002: │ @marvelapp/ui@workspace:src/packages/marvel-ui [85b8c] [56877] doesn't provide react-dom@>15 requested by react-div-100vh@npm:0.3.4 [92105] [1a0f7]
➤ YN0002: │ @marvelapp/ui@workspace:src/packages/marvel-ui [85b8c] [56877] doesn't provide react-dom@>15.4.2 <17.0.0 requested by react-intl-tel-input@npm:7.0.1 [92105] [1a0f7]
➤ YN0002: │ @marvelapp/ui@workspace:src/packages/marvel-ui [85b8c] [56877] doesn't provide react-dom@^16.1.1 requested by react-lightweight-tooltip@npm:1.0.0 [92105] [1a0f7]
➤ YN0002: │ @marvelapp/ui@workspace:src/packages/marvel-ui [85b8c] [56877] doesn't provide react-dom@^16.3.2 requested by react-pose@npm:4.0.2 [92105] [1a0f7]
➤ YN0002: │ @marvelapp/ui@workspace:src/packages/marvel-ui [85b8c] [56877] doesn't provide react-dom@^16.8.1 requested by react-select-search@npm:0.9.6 [92105] [1a0f7]
➤ YN0002: │ @marvelapp/ui@workspace:src/packages/marvel-ui [85b8c] [56877] doesn't provide react-dom@^15.0.0 || ^16.0.0 requested by react-textfit@npm:1.1.0 [92105] [1a0f7]
➤ YN0002: │ react-apollo@npm:3.0.1 [85b8c] [56877] doesn't provide apollo-utilities@^1.3.2 requested by @apollo/react-common@npm:3.0.1 [5d567]
➤ YN0002: │ react-apollo@npm:3.0.1 [85b8c] [56877] doesn't provide apollo-cache@^1.3.2 requested by @apollo/react-components@npm:3.1.2 [5d567]
➤ YN0002: │ react-apollo@npm:3.0.1 [85b8c] [56877] doesn't provide apollo-link@^1.2.12 requested by @apollo/react-components@npm:3.1.2 [5d567]
➤ YN0002: │ react-apollo@npm:3.0.1 [85b8c] [56877] doesn't provide apollo-utilities@^1.3.2 requested by @apollo/react-components@npm:3.1.2 [5d567]
➤ YN0002: │ @apollo/react-hooks@npm:3.0.1 [de1d7] [21625] doesn't provide apollo-utilities@^1.3.2 requested by @apollo/react-common@npm:3.0.1 [803f1]
➤ YN0002: │ @apollo/react-hoc@npm:3.1.2 [5d567] [8ed8c] doesn't provide apollo-utilities@^1.3.2 requested by @apollo/react-common@npm:3.0.1 [777d3]
➤ YN0002: │ @apollo/react-hoc@npm:3.1.2 [5d567] [8ed8c] doesn't provide apollo-cache@^1.3.2 requested by @apollo/react-components@npm:3.1.2 [777d3]
➤ YN0002: │ @apollo/react-hoc@npm:3.1.2 [5d567] [8ed8c] doesn't provide apollo-link@^1.2.12 requested by @apollo/react-components@npm:3.1.2 [777d3]
➤ YN0002: │ @apollo/react-hoc@npm:3.1.2 [5d567] [8ed8c] doesn't provide apollo-utilities@^1.3.2 requested by @apollo/react-components@npm:3.1.2 [777d3]
➤ YN0002: │ @apollo/react-hooks@npm:3.0.1 [c3205] [a250a] doesn't provide apollo-utilities@^1.3.2 requested by @apollo/react-common@npm:3.0.1 [e75e6]
➤ YN0002: │ @apollo/react-hooks@npm:3.0.1 [5d567] [8ed8c] doesn't provide apollo-utilities@^1.3.2 requested by @apollo/react-common@npm:3.0.1 [dcbbc]
➤ YN0002: │ @apollo/react-ssr@npm:3.1.2 [5d567] [8ed8c] doesn't provide @types/react@^16.8.0 requested by @apollo/react-common@npm:3.0.1 [a1528]
➤ YN0002: │ @apollo/react-ssr@npm:3.1.2 [5d567] [8ed8c] doesn't provide apollo-client@^2.6.4 requested by @apollo/react-common@npm:3.0.1 [a1528]
➤ YN0002: │ @apollo/react-ssr@npm:3.1.2 [5d567] [8ed8c] doesn't provide apollo-utilities@^1.3.2 requested by @apollo/react-common@npm:3.0.1 [a1528]
➤ YN0002: │ @apollo/react-ssr@npm:3.1.2 [5d567] [8ed8c] doesn't provide graphql@^14.3.1 requested by @apollo/react-common@npm:3.0.1 [a1528]
➤ YN0002: │ @apollo/react-ssr@npm:3.1.2 [5d567] [8ed8c] doesn't provide @types/react@^16.8.0 requested by @apollo/react-hooks@npm:3.0.1 [a1528]
➤ YN0002: │ @apollo/react-ssr@npm:3.1.2 [5d567] [8ed8c] doesn't provide apollo-client@^2.6.4 requested by @apollo/react-hooks@npm:3.0.1 [a1528]
➤ YN0002: │ @apollo/react-ssr@npm:3.1.2 [5d567] [8ed8c] doesn't provide graphql@^14.3.1 requested by @apollo/react-hooks@npm:3.0.1 [a1528]
➤ YN0002: │ @apollo/react-hooks@npm:3.0.1 [a1528] [eaa30] doesn't provide apollo-utilities@^1.3.2 requested by @apollo/react-common@npm:3.0.1 [116f6]
➤ YN0002: │ react-apollo@npm:3.0.1 [682af] doesn't provide apollo-utilities@^1.3.2 requested by @apollo/react-common@npm:3.0.1
➤ YN0002: │ react-apollo@npm:3.0.1 [682af] doesn't provide apollo-cache@^1.3.2 requested by @apollo/react-components@npm:3.1.2
➤ YN0002: │ react-apollo@npm:3.0.1 [682af] doesn't provide apollo-link@^1.2.12 requested by @apollo/react-components@npm:3.1.2
➤ YN0002: │ react-apollo@npm:3.0.1 [682af] doesn't provide apollo-utilities@^1.3.2 requested by @apollo/react-components@npm:3.1.2
➤ YN0002: │ @apollo/react-hooks@npm:3.0.1 [6863a] doesn't provide apollo-utilities@^1.3.2 requested by @apollo/react-common@npm:3.0.1
➤ YN0002: │ @apollo/react-hoc@npm:3.1.2 [dc44d] doesn't provide apollo-utilities@^1.3.2 requested by @apollo/react-common@npm:3.0.1
➤ YN0002: │ @apollo/react-hoc@npm:3.1.2 [dc44d] doesn't provide apollo-cache@^1.3.2 requested by @apollo/react-components@npm:3.1.2
➤ YN0002: │ @apollo/react-hoc@npm:3.1.2 [dc44d] doesn't provide apollo-link@^1.2.12 requested by @apollo/react-components@npm:3.1.2
➤ YN0002: │ @apollo/react-hoc@npm:3.1.2 [dc44d] doesn't provide apollo-utilities@^1.3.2 requested by @apollo/react-components@npm:3.1.2
➤ YN0002: │ @apollo/react-hooks@npm:3.0.1 [6d993] doesn't provide apollo-utilities@^1.3.2 requested by @apollo/react-common@npm:3.0.1
➤ YN0002: │ @apollo/react-hooks@npm:3.0.1 [dc44d] doesn't provide apollo-utilities@^1.3.2 requested by @apollo/react-common@npm:3.0.1
➤ YN0002: │ @apollo/react-ssr@npm:3.1.2 [dc44d] doesn't provide @types/react@^16.8.0 requested by @apollo/react-common@npm:3.0.1
➤ YN0002: │ @apollo/react-ssr@npm:3.1.2 [dc44d] doesn't provide apollo-client@^2.6.4 requested by @apollo/react-common@npm:3.0.1
➤ YN0002: │ @apollo/react-ssr@npm:3.1.2 [dc44d] doesn't provide apollo-utilities@^1.3.2 requested by @apollo/react-common@npm:3.0.1
➤ YN0002: │ @apollo/react-ssr@npm:3.1.2 [dc44d] doesn't provide graphql@^14.3.1 requested by @apollo/react-common@npm:3.0.1
➤ YN0002: │ @apollo/react-ssr@npm:3.1.2 [dc44d] doesn't provide @types/react@^16.8.0 requested by @apollo/react-hooks@npm:3.0.1
➤ YN0002: │ @apollo/react-ssr@npm:3.1.2 [dc44d] doesn't provide apollo-client@^2.6.4 requested by @apollo/react-hooks@npm:3.0.1
➤ YN0002: │ @apollo/react-ssr@npm:3.1.2 [dc44d] doesn't provide graphql@^14.3.1 requested by @apollo/react-hooks@npm:3.0.1
➤ YN0002: │ @apollo/react-hooks@npm:3.0.1 [83318] doesn't provide apollo-utilities@^1.3.2 requested by @apollo/react-common@npm:3.0.1

Is this set up no longer supported?

bug
Source
picture georeith

Most helpful comment

I think I found the problem - we're doing a delete here, so if a virtualised package is deduplicated while at the same time depending on a peer dependency that will be deduplicated, we're going to hit the assertion here. I'll try to make a repro and submit a fix.

It takes me about 16 minutes per yarn to get to this error which is frustrating when you are making tweaks.

Is it the "Docker is super slow problem"? Can you share the timings for each step? Which one is the slowest?

That would be good, would be definitely helpful to add more context to this error message too.

Btw I forgot to mention it, but by running yarn set version from sources --no-minify you'll get a build straight from master, without minification. It's easier to find out what is crashing and where, and put console.log around.

picture arcanis on 26 Nov 2019
🚀1 🎉1 👍1

All 21 comments

Is this set up no longer supported?

The v1 workspace implementation was broken in that even if you were listing peer dependencies inside a workspace, it wasn't actually up to their dependent to provide it (because we had no way to force a workspace to be instantiated multiple times depending on their effective dependency set).

It kinda worked when you listed the peer dependencies inside the top-level, but it wasn't meant to behave that way - more a side effect of the hoisting than anything else.

Working on this, I'm running in a large private repo so need some time to recreate the case.

I wonder if we could make an error reducer plugin ... copy all the workspaces somewhere, then remove them one by one until the error doesn't repro anymore, then do the same for individual dependencies ... 🤔

arcanis picture arcanis on 26 Nov 2019

It kinda worked when you listed the peer dependencies inside the top-level, but it wasn't meant to behave that way - more a side effect of the hoisting than anything else.

I was worried you would say that 😛. I'm not really sure how I would go about specifying peer dependencies in workspace packages so that they could be resolved in yarn 2. Do I just have to specify it as a dev dependency also?

I wonder if we could make an error reducer plugin ... copy all the workspaces somewhere, then remove them one by one until the error doesn't repro anymore, then do the same for individual dependencies ... 🤔

That would be good, would be definitely helpful to add more context to this error message too.

PS: Thanks for the really swift replies on here.

georeith picture georeith on 26 Nov 2019

Do I just have to specify it as a dev dependency also?

Yep! This will act as the default dependency when the workspace is accessed from its own folder (and it will pick up the right peer dependencies if another workspace depends on it).

Note that you can use yarn up lodash to upgrade lodash across all your workspaces at once. It makes it less tedious to manage duplicated dependencies (plus yarn upgrade-interactive with the @yarnpkg/interactive-tools plugin).

arcanis picture arcanis on 26 Nov 2019
👍1

OK so I've resolved all the react-dom peer dependency issues, I still receive this error but I'm not really sure how to go about debugging it yet. It takes me about 16 minutes per yarn to get to this error which is frustrating when you are making tweaks.

georeith picture georeith on 26 Nov 2019

I think I found the problem - we're doing a delete here, so if a virtualised package is deduplicated while at the same time depending on a peer dependency that will be deduplicated, we're going to hit the assertion here. I'll try to make a repro and submit a fix.

It takes me about 16 minutes per yarn to get to this error which is frustrating when you are making tweaks.

Is it the "Docker is super slow problem"? Can you share the timings for each step? Which one is the slowest?

That would be good, would be definitely helpful to add more context to this error message too.

Btw I forgot to mention it, but by running yarn set version from sources --no-minify you'll get a build straight from master, without minification. It's easier to find out what is crashing and where, and put console.log around.

arcanis picture arcanis on 26 Nov 2019
🚀1 🎉1 👍1

I think I found the problem - we're doing a delete here, so if a virtualised package is deduplicated while at the same time depending on a peer dependency that will be deduplicated, we're going to hit the assertion here. I'll try to make a repro and submit a fix.

🙏 legend. Really appreciate you looking into this especially without a reproduction case.

Is it the "Docker is super slow problem"? Can you share the timings for each step? Which one is the slowest?

It spends a super long time on react-native which one of the packages in the workspace uses. Often fails here with certificate errors like:

Hostname/IP does not match certificate's altnames: Host: registry.yarnpkg.com. is not in the cert's altnames: DNS:*.facebook.net, DNS:*.facebook.com, DNS:*.m.facebook.com, DNS:facebook.com, DNS:*.fb.com, DNS:*.xy.fbcdn.net, DNS:*.xz.fbcdn.net, DNS:fb.com, DNS:*.messenger.com, DNS:*.fbsbx.com, DNS:*.xx.fbcdn.net, DNS:messenger.com, DNS:*.fbcdn.net
    at ClientRequest.<anonymous> (/Users/george/Documents/marvel/mkiii/.yarn/releases/yarn-berry.js:174:13324)
    at Object.onceWrapper (events.js:288:20)
    at ClientRequest.emit (events.js:205:15)
    at ClientRequest.e.emit (/Users/george/Documents/marvel/mkiii/.yarn/releases/yarn-berry.js:360:55257)
    at TLSSocket.socketErrorListener (_http_client.js:402:9)
    at TLSSocket.emit (events.js:205:15)
    at emitErrorNT (internal/streams/destroy.js:91:8)
    at emitErrorAndCloseNT (internal/streams/destroy.js:59:3)
    at processTicksAndRejections (internal/process/task_queues.js:84:9)

It's possible its going through some very slow build during pack there as it is spending a loooong time on individual packages in the resolution step, I did yarn unplug react-native.

Btw I forgot to mention it, but by running yarn set version from sources --no-minify you'll get a build straight from master, without minification. It's easier to find out what is crashing and where, and put console.log around.

That is super useful! I am ashamed to say I modified the minified version of the package to log out which package this was failing on.

georeith picture georeith on 26 Nov 2019

I think I found the problem - we're doing a delete here

Can confirm that commenting out that line lets it get further!

image

If I reintroduce that line now it takes 3 seconds to get back to that error as it doesn't need to download everything anymore (made it past that stage for the first time)

image

Progress!

georeith picture georeith on 26 Nov 2019

Damn, you're breaking all my assertions! 😄 I haven't managed to write a test for the previous one yet, still working on that.

Can you try to rather replace the assertion I linked with a continue if the package cannot be found? It still needs to be removed from allPackages, otherwise Yarn will try to link it even though it shouldn't exist anymore (hence the second assertion).

arcanis picture arcanis on 26 Nov 2019

@arcanis That triggers the same issue as when I comment out the delete:

image

Assuming this is what you meant:

for (const descriptorGroup of descriptorGroups) {
  const [primaryDescriptor, ...duplicates] = descriptorGroup;

  for (const duplicateDescriptor of duplicates) {
    const dependents = allVirtualizedDescriptorsDependents.get(duplicateDescriptor.descriptorHash);
    if (!dependents) throw new Error(`Assertion failed: The virtual package should have a list of dependents`);

    for (const dependentLocatorHash of dependents) {
      const dependentPackage = allPackages.get(dependentLocatorHash);
      if (!dependentPackage) continue;
      dependentPackage.dependencies.set(primaryDescriptor.identHash, primaryDescriptor);
    }

    const duplicateVirtualPackage = getPackageFromDescriptor(duplicateDescriptor);
    allDescriptors.delete(duplicateDescriptor.descriptorHash);
    allPackages.delete(duplicateVirtualPackage.locatorHash);
    allResolutions.delete(duplicateDescriptor.descriptorHash);
    virtualDescriptors.delete(duplicateDescriptor);
  }
}
georeith picture georeith on 26 Nov 2019

Here are all the primaryDescriptor which don't have a dependentPackage:
https://gist.github.com/georeith/4f67bf1895e8b4aef1424e097043cb91

There is a lot of [email protected] however only 3 of my packages depend on it. A few sub-packages depend on it also, here's how many occurrences it has in yarn.lock:

image

georeith picture georeith on 26 Nov 2019

Commenting out all of these lines allows me to get passed the above error albeit it is trying to build many copies of the same versions of packages now:

// allDescriptors.delete(duplicateDescriptor.descriptorHash);
// allPackages.delete(duplicateVirtualPackage.locatorHash);
// allResolutions.delete(duplicateDescriptor.descriptorHash);

It also never encounters a descriptor without a dependent with the above:

// this doesn't fire when the above are commented out
if (!dependentPackage) throw new Error(`Assertion failed: The dependent package could not be found`);

image

Edit: it completes like that

image

georeith picture georeith on 26 Nov 2019

Can you try removing your workspaces (you can just remove the glob pattern from the workspaces property and add the exact relative paths) and adding them back one by one until it crashes, then doing the same for dependencies? 🤔

arcanis picture arcanis on 26 Nov 2019

~Sadly every workspace package is cross dependant and isn't published to a registry as individual packages (yet) so I can't isolate them yet 😞~

Ignore me, I managed to isolate the main cross dependencies and they work in isolation so I can add the rest in one by one.

georeith picture georeith on 26 Nov 2019

OK so I've managed to find a package that produces the error and the bare minimum required to fix it. It isn't the only one though, there are multiple packages that when imported trigger this error.

It appears to happen when any given package in the repository imports two devDependencies @marvelapp/marvel-3-test-application and react. These do not seem to like co-existing as devDependencies.

georeith picture georeith on 26 Nov 2019

What's the package.json for marvel-3-test-application?

arcanis picture arcanis on 26 Nov 2019

@arcanis I have created a reproduction out of it: https://github.com/georeith/berry-609

I am working on removing the extraneous fluff from that but you can see if you remove @apollo/react-hooks from dependencies in packages/marvel-upsells OR remove @marvelapp/marvel-3-test-application and react from its devDependencies then the error no longer occurs.

georeith picture georeith on 26 Nov 2019

Thanks a lot! I've been able to reproduce the problem, I'll work to reduce it tomorrow. It seems to be linked to react-apollo too 🤔

arcanis picture arcanis on 27 Nov 2019
😄1

I reduced the repro case down a lot more. There's quite a few ways to remove the error now. If you remove marvel-ui-internal package it disappears too along with the ways I mentioned above 🤔

georeith picture georeith on 27 Nov 2019

I've reduced it down to the following, and after checking the logic it seems coherent with my free-befoe-use theory. Now to write it into code 😄

image

arcanis picture arcanis on 27 Nov 2019
🚀1

@arcanis amazing! I won't pretend to fully understand it, think I'm lacking a bit of domain knowledge but I see you made a PR for it.

If that's ready I can try it out against my repo, what's the easiest way to do that? Just modify the yarnPath of my repo to point at the bundle path output after running yarn build:cli?

georeith picture georeith on 27 Nov 2019

You can test PRs by running yarn set version from sources --branch <PR number>. I just merged it in master though so it should work without --branch 👍

Closing this issue to keep the history clean, but feel free to open another if something else looks weird! (like maybe the assertion in https://github.com/yarnpkg/berry/issues/609#issuecomment-558694328 - I'm not sure if it was connected to the problem I fixed or something different)

arcanis picture arcanis on 27 Nov 2019
Was this page helpful?
0 / 5 - 0 ratings

Related issues

milichev picture milichev  ·  3Comments
IanVS picture IanVS  ·  4Comments
larixer picture larixer  ·  4Comments
chrisands picture chrisands  ·  3Comments
benwainwright picture benwainwright  ·  3Comments