Serverless-webpack: Bug - yarn packager is causing sls deploy to fail
This is a Bug Report
Description
Using serverless pro to deploy using serverless-webpack "serverless-webpack": "^5.3.5". Successfully deployed if I change packager to npm. not sure what is going on. If I deploy locally with yarn as the packager it works.
Additional Data
Serverless: Invoke webpack:package
Error --------------------------------------------------
Error: yarn list --depth=1 --json --production failed with code 1
at ChildProcess.
at ChildProcess.emit (events.js:314:20)
at ChildProcess.EventEmitter.emit (domain.js:483:12)
at maybeClose (internal/child_process.js:1021:16)
at Process.ChildProcess._handle.onexit (internal/child_process.js:286:5)
Get Support --------------------------------------------
Docs: docs.serverless.com
Bugs: github.com/serverless/serverless/issues
Issues: forum.serverless.com
Your Environment Information ---------------------------
Operating System: linux
Node Version: 12.19.0
Framework Version: 2.23.0 (local)
Plugin Version: 4.4.3
SDK Version: 2.3.2
Components Version: 3.6.2
Build step failed: serverless deploy --stage feature-sales-profiles-lambdas --region us-east-1 --force --org myOrg --app myApp --debug
failure
Deploy failed!
Error: Build step failed: serverless deploy --stage feature-sales-profiles-lambdas --region us-east-1 --force --org myOrg --app myApp --debug
Kevchan85
All 20 comments
Unfortunately I have no space to investigate, but I can tell that this happens when running npm as well - both locally and on pipeline.
Serverless: Fetch dependency graph from [redacted]/package.json
Error --------------------------------------------------
Error: npm ls -prod -json -depth=1 failed with code 1
at ChildProcess.<anonymous> ([redacted]/node_modules/serverless-webpack/lib/utils.js:91:16)
at ChildProcess.emit (events.js:314:20)
at ChildProcess.EventEmitter.emit (domain.js:483:12)
at maybeClose (internal/child_process.js:1022:16)
at Socket.<anonymous> (internal/child_process.js:444:11)
at Socket.emit (events.js:314:20)
at Socket.EventEmitter.emit (domain.js:483:12)
at Pipe.<anonymous> (net.js:675:12)
For debugging logs, run again after setting the "SLS_DEBUG=*" environment variable.
Get Support --------------------------------------------
Docs: docs.serverless.com
Bugs: github.com/serverless/serverless/issues
Issues: forum.serverless.com
Your Environment Information ---------------------------
Operating System: darwin
Node Version: 12.20.2
Framework Version: 1.83.2 (local)
Plugin Version: 3.8.4
SDK Version: 2.3.2
Components Version: 2.34.9
Workaround:
In order to allow my services to deploy successfully I had to downgrade the package via eg.
npm install --save-dev [email protected]
itskemo
on 12 Feb 2021
Just set this to [email protected] same issue. :/ just using npm for now...
Kevchan85
on 12 Feb 2021
I'm getting the same issues with [email protected] both npm and yarn.
ariel-frischer
on 23 Feb 2021
Same issue with 5.3.4 and 5.3.5.
ciceropablo
on 26 Feb 2021
Actually my yarn error is slightly different:
Error: yarn.cmd install --frozen-lockfile --non-interactive failed with code 1
at ChildProcess.<anonymous> (C:\Users\Ariel\initiate-video\backend\node_modules\serverless-webpack\lib\utils.js:91:16)
at ChildProcess.emit (events.js:314:20)
at ChildProcess.EventEmitter.emit (domain.js:483:12)
at maybeClose (internal/child_process.js:1021:16)
at Process.ChildProcess._handle.onexit (internal/child_process.js:286:5)
- Which version of yarn are you all using?
- Could you trying using
"serverless-webpack": "serverless-heaven/serverless-webpack#master"just to check if the issue is already fixed in the upcoming 5.4.0?
j0k3r
on 26 Feb 2021
@j0k3r Im using yarn: 1.22.4 and npm 6.14.8. I'll test it now.
I've also already tried setting noFrozenLockFile: true which doesn't seem to do anything.
ariel-frischer
on 26 Feb 2021
@j0k3r Looks like the serverless-heaven/serverless-webpack#master is working!
ariel-frischer
on 26 Feb 2021
As the master seems to fix the problem, I'm closing that issue.
Let me know if you have the problem when using the master and not a version lower than 5.4.0
j0k3r
on 4 Mar 2021
Unfortunately I'm seeing this again with Windows 10 specifically. Even with the master branch...
ariel-frischer
on 16 Mar 2021
But nothing really changed since your last comment 馃
j0k3r
on 16 Mar 2021
But nothing really changed since your last comment 馃
Yea, it's strange it happens on my laptop but not my desktop, could be different versions of yarn I'll look into it... I also think this may have something to do with using a linked package.
ariel-frischer
on 16 Mar 2021
Hello,
@j0k3r
I've been trying to get it to work for the past couple of hours but with no luck. I'm using the latest version and tried the master commit as you suggested but did not work.
Here are some verbose logs:
webpack compiled successfully in 64989 ms
Serverless: Invoke webpack:package
(node:16957) [DEP_WEBPACK_CHUNK_MODULES_ITERABLE] DeprecationWarning: Chunk.modulesIterable: Use new ChunkGraph API
(Use `node --trace-deprecation ...` to show where the warning was created)
(node:16957) [DEP_WEBPACK_MODULE_ISSUER] DeprecationWarning: Module.issuer: Use new ModuleGraph API
Serverless: Package lock found - Using locked versions
Serverless: Packing external modules: @prisma/[email protected], apollo-errors@^1.9.0, apollo-server-lambda@^2.21.1, graphql@^15.3.0, aws-sdk@^2.866.0, axios@^0.21.1, bcrypt@^5.0.1, class-validator@^0.13.1, graphql-depth-limit@^1.1.0, graphql-fields@^2.0.3, graphql-middleware@^6.0.4, graphql-shield@^7.5.0, joi@^17.4.0, [email protected], md5@^2.3.0, moment@^2.29.1, object-path@^0.11.5, p-map@^4.0.0, random@^3.0.3, reflect-metadata@^0.1.13, type-graphql@^1.1.1
..................] \ : timing npm:load Completed in 19ms
Error ---------------------------------------------------
Error: yarn install --non-interactive failed with code 1
at ChildProcess.<anonymous> (/home/omar/Desktop/graphqla-api/node_modules/serverless-webpack/lib/utils.js:91:16)
at ChildProcess.emit (events.js:315:20)
at ChildProcess.EventEmitter.emit (domain.js:486:12)
at maybeClose (internal/child_process.js:1048:16)
at Process.ChildProcess._handle.onexit (internal/child_process.js:288:5)
Get Support --------------------------------------------
Docs: docs.serverless.com
Bugs: github.com/serverless/serverless/issues
Issues: forum.serverless.com
Your Environment Information ---------------------------
Operating System: linux
Node Version: 14.15.1
Framework Version: 2.29.0
Plugin Version: 4.5.0
SDK Version: n/a
Components Version: 3.7.3
npm timing command:run-script Completed in 72520ms
npm verb exit 1
npm timing npm Completed in 72628ms
npm verb code 1
yarn v2.4.1
serverless-webpack v5.4.0
omar-dulaimi
on 18 Mar 2021
@omar-dulaimi Same here, I turned to
packagerOptions:
noFrozenLockfile: true
jp928
on 18 Mar 2021
@jp928 Unfortunately setting this flag did not solve the issue for me:
webpack: {
webpackConfig: './webpack.config.js',
includeModules: true,
packager: 'yarn',
packagerOptions: {
scripts: ['prisma generate'],
noFrozenLockfile: true,
},
},
Any update on this?
We tried to downgrade but that also did not help, and as a result, we can't deploy our app and are forced to keep on hold.
omar-dulaimi
on 20 Mar 2021
So after a little bit of digging around, here's what I found:
"./node_modules/serverless-webpack/lib/packagers/yarn.js"
static install(cwd, packagerOptions) {
const command = /^win/.test(process.platform) ? 'yarn.cmd' : 'yarn';
const args = [ 'install', '--non-interactive' ];
if (!packagerOptions.noFrozenLockfile) {
args.push('--frozen-lockfile');
}
if (packagerOptions.ignoreScripts) {
args.push('--ignore-scripts');
}
// return Utils.spawnProcess(command, args, { cwd }).return(); // replaced with below return statement
return BbPromise.fromCallback(cb => {
childProcess.exec(command, {
cwd: cwd,
maxBuffer: 10000000,
encoding: 'utf8'
}, cb);
})
.return();
}
static runScripts(cwd, scriptNames) {
const command = /^win/.test(process.platform) ? 'yarn.cmd' : 'yarn';
// return BbPromise.mapSeries(scriptNames, scriptName => { // replaced with below code
// const args = [ 'run', scriptName ];
// return Utils.spawnProcess(command, args, { cwd });
// }).return();
return BbPromise.mapSeries(scriptNames, scriptName => BbPromise.fromCallback(cb => {
childProcess.exec(`yarn run ${scriptName}`, {
cwd: cwd,
maxBuffer: 100000000,
encoding: 'utf8'
}, cb);
}))
.return();
}
Basically, I had to change how serverless-webpack install and runScripts methods work, by changing them back to their @5.1.0 forms.
In those forms, the child processes were called using a different node api, that sets the max buffer for stdio.
This maxBuffer value can be set from the serverless template like this(as mentioned in @5.1.0 README):
custom:
webpack:
packExternalModulesMaxBuffer: NUMBER_OF_YOUR_CHOICE # Size of stdio buffers for spawned child processes
Hopefully, some people here shed some light on the notes above so we could choose sensible values.
Happy to submit a PR for this @j0k3r
omar-dulaimi
on 20 Mar 2021
any1 managed to solve this? its really annoying
yehonadav
on 31 May 2021
so i noticed i was missing a dependency in my package.json that was installed inside my node_modules.
after adding the module to package.json everything worked out fine.
i was using jsonwebtoken and had @types/jsonwebtoken in my dev-dependencies but not jsonwebtoken in my dependencies.
was running with:
win10
yarn v1.22.10
serverless-webpack 5.5.0
hope this helps
yehonadav
on 31 May 2021
This is still occurring for us, but only randomly in some components which all have mostly identical package, webpack + sls configs.
BB pipelines using Atlassian Linux distro
yarn 2
serverless-webpack 5.3.0 - 5.5.1 (tried them all)
serverless 2.43.1
mrowles
on 10 Aug 2021
Related issues
bericp1
路
5Comments
tommedema
路
4Comments
hassankhan
路
3Comments
jmparsons
路
5Comments
agreea
路
3Comments
Most helpful comment
So after a little bit of digging around, here's what I found:
Basically, I had to change how serverless-webpack install and runScripts methods work, by changing them back to their @5.1.0 forms.
In those forms, the child processes were called using a different node api, that sets the max buffer for stdio.
This maxBuffer value can be set from the serverless template like this(as mentioned in @5.1.0 README):
Hopefully, some people here shed some light on the notes above so we could choose sensible values.
Happy to submit a PR for this @j0k3r