Berry: Installing yarn's dependencies into a specific folder

[thisiscab]

Hello,

We're trying to use Yarn V2 (Berry) workspaces feature as a way to build our dependencies for our Lambda Functions. We have multiple packages in our monorepo and each package represents a Lambda function with its own dependencies.

For context, Lambda Functions using NodeJS require to have node_modules to run. For that reason, we've opted to use the nodeLinker: node-modules flag but we're running into a problem.

With Yarn V1, we would run the following: yarn install --production --modules-folder lib/deployments. This would ensure that it would only install the production dependencies and put them into a specific folder that we could then use to deploy our Lambda function.

Unfortunately, there are no counter-parts to --modules-folder in Yarn V2. If we were to run yarn workspaces focus --production then it would modify our project's build-state.yml and install-state.gz files and the dev dependencies would become unusable until we would re-run yarn at the project level.

In our ideal world, there would be a way for us to export a workspace's dependencies into a node_modules folder given a specific path. Unless I'm wrong, I don't think this is achievable currently and we can't seem to find a workaround, thus making Yarn V2 not something we can use.

Am I missing something?

Thanks!

[Larry1123]

Hi I have a plugin that will do just that. plugin-production-install.
It does all it's work before the linker runs so it works the same for pnp and n_m. Hope it works for you. If you run into issues with it feel free to reach out.

[jakepearson]

@Larry1123, thanks this is really helpful. The last step I need to deploy to aws lambda would be to startup the lambda with pnp enabled. Can anyone point me in the direction of how to do that?

[merceyz]

You can require the .pnp.js file and call its setup function

require('../.pnp.js').setup()

[jakepearson]

@merceyz Thanks a bunch. I think I nearly have this working. My last problem appears to be related to how we have setup layers in our lambda. We put all of the dependencies into a layer and then the runtime code goes into the main layer. That leads to my lambda not being able to find the dependency zips ...

Error: Qualified path resolution failed - none of the candidates can be found on the disk.
Source path: /var/task/lib/.yarn/cache/source-map-support-npm-0.5.19-65b33ae61e-59d4efaae9.zip/node_modules/source-map-support/register
Rejected candidate: /var/task/lib/.yarn/cache/source-map-support-npm-0.5.19-65b33ae61e-59d4efaae9.zip/node_modules/source-map-support/register
Rejected candidate: /var/task/lib/.yarn/cache/source-map-support-npm-0.5.19-65b33ae61e-59d4efaae9.zip/node_modules/source-map-support/register.js
Rejected candidate: /var/task/lib/.yarn/cache/source-map-support-npm-0.5.19-65b33ae61e-59d4efaae9.zip/node_modules/source-map-support/register.json
Rejected candidate: /var/task/lib/.yarn/cache/source-map-support-npm-0.5.19-65b33ae61e-59d4efaae9.zip/node_modules/source-map-support/register.node

Is there a way to tell require('../.pnp.js').setup() the location of the dependencies. Currently, /opt/nodejs/.yarn/cache. Maybe it looks relative to where the .pnp.js lives. I'll try that in the morning.

[Larry1123]

The .pnp.js file is a part of the install. It will need to be in the same relative location as it was to the dependencies when it was created. You should treat the .pnp.js file as a dependency, and not a source file.

I have since added a --pack argument for my plugin that will run any prepack script and pack your built source into the out dir.

[jakepearson]

Thanks for the help. I was able to get everything working with our dependency layer. We now run this code at the beginning of each lambda:

const pnpPath = '/opt/nodejs/.pnp.js';
if (fs.existsSync(pnpPath)) {
  const dirnameSave = __dirname;
  __dirname = '/opt/nodejs';
  const pnp = require('/opt/nodejs/.pnp.js');
  pnp.setup();
  __dirname = dirnameSave;
}

[merceyz]

Closing as this seems to boil down to "production deployments" which can be solved by using either of these Yarn plugins https://gitlab.com/Larry1123/yarn-contrib/-/tree/master/packages/plugin-production-install or https://yarn.build/