Berry: Does pnp require adding lots more dependencies to package.json?

[swrobel]

I'm attempting to migrate a project to the 2.0rc & use pnp but am running into tons of issues like so:

Module build failed (from ./.yarn/virtual/babel-loader-virtual-ec6f956c5b/0/cache/babel-loader-npm-8.0.6-ccc68d8d38.zip/node_modules/babel-loader/lib/index.js):
Error: A package is trying to access another package without the second one being listed as a dependency of the first one

Required package: babel-plugin-macros (via "babel-plugin-macros/package.json")
Required by: MyApp@workspace:.

I'm able to fix these by, in this example, running yarn add --dev babel-plugin-macros but then another one pops up and so on & on this whac-a-mole goes with seemingly every babel plugin that's in babel-present-env & required in my .babel.config.js. Isn't it enough that they're bundled by virtue of being a second-order dependency, or do they really have to be a first-order dependency?

I apologize if this seems like a support request, but I genuinely just want to migrate a Rails/webpacker project to yarn 2 & document the process! Feel free to close if there's no time to deal w/ this.

[arcanis]

Hey @swrobel!

Isn't it enough that they're bundled by virtue of being a second-order dependency, or do they really have to be a first-order dependency?

Nope! What you describe a behaviour that used to kinda work, but it was an unintended side effect of the way the packages were layout on the disk. The problem was that since Yarn had no idea about those dependencies, it couldn't really lock them, causing very subtle issues where the version of a package suddenly changed (or disappeared altogether) depending on the selected hoisting - which could changed based on the other packages in your dependencies, or potentially the Yarn version.

The v2 features a new system called Plug'n'Play (more details here) that's able to surface the problem before it causes serious issues in production, hence the message you're seeing. I probably should add a link to the error message to explain it in more details, though 🤔

[arcanis]

There's one bug, though. The following:

A package is trying to access another package without the second one being listed as a dependency of the first one

Should have been reported as:

Something that got detected as your top-level application (because it doesn't seem to belong to any package) tried to access a package that is not declared in your dependencies

This following check doesn't take workspaces into account:
https://github.com/yarnpkg/berry/blob/master/packages/yarnpkg-pnp/sources/loader/makeApi.ts#L532

[eps1lon]

on this whac-a-mole goes with seemingly every babel plugin that's in babel-present-env

This seems odd. If you don't use plugins from preset-env then yarn shouldn't complain. Or are you in fact manually listing those plugins in which case the error is expected. The point of preset-env is however that you don't need to list those manually.

[arcanis]

Right, I missed the preset-env part - can you share your package.json and babelrc?

[swrobel]

Right, I missed the preset-env part - can you share your package.json and babelrc?

Sure thing: https://gist.github.com/swrobel/75a48fd84331dbcb20e4506c8ba182b3

The babel.config.js is a slightly modified version of the webpacker default

[swrobel]

Also, hate to ask this question, but is there some way to just disable PnP in 2.0 for now with some sort of config setting? I tried this but it didn't work:

pnpIgnorePattern: ".*"

[arcanis]

Not at the moment. @vlasenko is working on adding a basic support for n_m installs, but that won't be for some time (and of course I don't recommend to rely on it on a permanent basis 😊).

I'll take a look at your files later this week 👍

[swrobel]

@arcanis did you ever get a chance to look at my files?

[arcanis]

Hey @swrobel! I just gave a look, and i think the problem is that the dependencies you're referencing aren't dependencies of @babel/preset-env. Since you're the one authoring the babel.config.js file, your application is responsible for listing the packages you use in your own dependencies. By contrast, the plugins listed in preset-env the files are loaded relative to preset-env and thus work properly.

In particular, here are the packages I've added to your manifest:

11a12,15
>     "@babel/cli": "^7.7.0",
>     "@babel/core": "^7.7.2",
>     "@babel/plugin-proposal-class-properties": "^7.7.0",
>     "@babel/plugin-proposal-object-rest-spread": "^7.6.2",
12a17,21
>     "@babel/plugin-syntax-dynamic-import": "^7.2.0",
>     "@babel/plugin-transform-destructuring": "^7.6.0",
>     "@babel/plugin-transform-regenerator": "^7.7.0",
>     "@babel/plugin-transform-runtime": "^7.6.2",
>     "@babel/preset-env": "^7.0.0",
18a28
>     "babel-plugin-macros": "^2.6.1",

After that, running yarn babel <path/to/js> worked (some more might be required in test environment since you selectively enable part of them).

[bertho-zero]

I had the same problem and I was able to fix it by replacing my .babelrc:

{
  "presets": [
    "@openagenda"
  ],
  "sourceType": "unambiguous"
}

with a .babelrc.js:

'use strict';

module.exports = {
  presets: [
    [
      require('@openagenda/babel-preset'),
      {
        modules: 'commonjs'
      }
    ]
  ],
  sourceType: 'unambiguous'
};

Otherwise, in my case, it's @babel-core which loads the presets and plugins from the wrong place. Maybe this should be mentioned on the migration doc? and maybe it can be improved on the babel side.

The problem may be the same for eslint (https://github.com/yarnpkg/berry/issues/1120, https://github.com/yarnpkg/berry/issues/1057, ...).

Like https://github.com/yarnpkg/berry/issues/843 ?

[bbugh]

It would be helpful if the Required package warning gave the expected version as well. When upgrading to PnP, we had a lot of packages that were not explicitly in package.json, but were intentionally included by another package. It's taking quite a bit of time to reference the yarn.lock file for the expected version, install it, then run webpack to see what fails next. If it said Require Package: package@npm:1.1.1 instead, that would make it a lot easier.

[merceyz]

It would be helpful if the Required package warning gave the expected version as well

We have no way of knowing what the "expected" version would be, even if we performed hoisting in memory to check what it would have gotten that isn't the same thing.