Aws-cdk: @aws-cdk/aws-lambda-nodejs doesn't give useful error messages when builds fail.

[thekevinbrown]

We're looking at switching over from Serverless to CDK for deploying our lambdas to minimise size. One of the things that Serverless does that I don't know how to do cleanly in CDK is excluding devDependencies from the final built package that gets zipped / uploaded.

How do I go about configuring this behaviour in CDK?

---

This is a 📕 documentation issue

[thekevinbrown]

I should note that I've also tried to use @aws-cdk/aws-lambda-nodejs but I get Error: spawnSync parcel ENOENT when I try that, so I was wondering if there was a way to do this without using aws-lambda-nodejs or if I should keep trying to work through the ENOENT error.

[jogold]

should keep trying to work through the ENOENT error.

See #6206

[thekevinbrown]

@jogold I went ahead and ran npm i -g parcel-bundler to work around that error.

Now I get:

[project directory]/backend/packages/infrastructure/node_modules/@aws-cdk/aws-lambda-nodejs/lib/build.ts:85
    throw new Error(`Failed to build file at ${options.entry}: ${err}`);
          ^
Error: Failed to build file at [project directory]/backend/packages/consumer-api/src/index.ts: Error
    at Object.build ([project directory]/backend/packages/infrastructure/node_modules/@aws-cdk/aws-lambda-nodejs/lib/build.ts:85:11)
    at new NodejsFunction ([project directory]/backend/packages/infrastructure/node_modules/@aws-cdk/aws-lambda-nodejs/lib/function.ts:89:5)
    at Object.exports.attach ([project directory]/backend/packages/infrastructure/src/consumer-api/v1/index.ts:31:22)
    at Object.exports.attach ([project directory]/backend/packages/infrastructure/src/consumer-api/index.ts:17:5)
    at new ApiService ([project directory]/backend/packages/infrastructure/src/index.ts:29:15)
    at Object.<anonymous> ([project directory]/backend/packages/infrastructure/src/index.ts:36:1)
    at Module._compile (internal/modules/cjs/loader.js:778:30)
    at Module.m._compile ([project directory]/backend/packages/infrastructure/node_modules/ts-node/src/index.ts:814:23)
    at Module._extensions..js (internal/modules/cjs/loader.js:789:10)
    at Object.require.extensions.(anonymous function) [as .ts] ([project directory]/backend/packages/infrastructure/node_modules/ts-node/src/index.ts:817:12)

Not sure what to do to figure out what the Error actually is.

[jogold]

Looks like parcel outputs its error to stdout and not stderr see #6277. If you apply this fix you should see the reason why your build is failing.

[mishabruml]

Hi, I'm still having issues related to this.

I've got a nodejs lambda configured as follows:

    const restartServiceLambda = new lambdaNodejs.NodejsFunction(this, 'restart-service-lambda', {
      runtime: lambda.Runtime.NODEJS_12_X,
      entry: 'src/restart-service/index.ts',
      handler: 'handler',
      environment: {
        SERVICE_NAME: xxx,
        CLUSTER_NAME: yyy
      },
      sourceMaps: true
    })

Locally I can run cdk synth or run my test suite (which invokes the parcel build by calling new Stack() on the lambda's parent, as part of some cdk assertions). Thankfully, and curiously, I can also deploy. All of this no issues.

My test suite is failing in CI though (circleci), which is odd, and I'm struggling to diagnose the issue because of the lack of a meaningful error message:

FAIL  test/cdk/restart-service.ts (8.106s)
  ● RestartService › encountered a declaration exception

    Failed to build file at src/restart-service/index.ts: Error



      at Object.build (node_modules/@aws-cdk/aws-lambda-nodejs/lib/build.ts:91:11)
      at new NodejsFunction (node_modules/@aws-cdk/aws-lambda-nodejs/lib/function.ts:89:5)
      at new RestartService (lib/cdk/restart-service.ts:642:30)
      at Suite.<anonymous> (test/cdk/restart-service.ts:32:3)
      at Object.<anonymous> (test/cdk/restart-service.ts:12:1)

At first I thought it was perhaps due to a path issue, like the src/xxx... path wasn't being resolved properly in circleci, but after changing the path to one that I knew would fail, I realised that this results in a different error message (Cannot find entry file at fake/path/restart-service/index.ts)

So, some other unknown factor is at play here and I can't debug very easily

[jogold]

@mishabruml which version are you using? From your stack trace it looks like you are using an older version... can you try with v1.27.0?

[mishabruml]

I'm on 1.25, but that's the version that got the fix re: error messages right? Will try 1.27 and report back

[mishabruml]

@jogold same error, and lack of parcel message in CI when running with 1.27

[jogold]

Could you try running parcel manually in your CI to debug this?
parcel build src/restart-service/index.ts --out-dir .build --out-file index.js --global handler --target node --bundle-node-modules --log-level 2 --no-minify

[jogold]

Also are you running tsc in this folder before your tests? I mean does src/restart-service/index.js exist? And are you running your app with ts-node?

[mishabruml]

I'm not calling tsc at all, because calling new lambdaNodejs.NodejsFunction actually calls the parcel build command to generate the /build/index.js. I've just confirmed this locally

I'm not using ts-node

[jogold]

Did you try running parcel (https://github.com/aws/aws-cdk/issues/6274#issuecomment-597114485)?

[mishabruml]

Yes. I ssh'd in to the circleci image and ran it directly. Some observations:

a) new lambdaNodejs.NodejsFunction by default builds the handler code next to the source code, not in ./.build as your command implies
b) running your command (ssh'd in), and then running npm test passes the suite
c) deleting any build files, then modifying your command's out-dir to parcel build src/restart-service/index.ts --out-dir src/restart-service/.build --out-file index.js --global handler --target node --bundle-node-modules --log-level 2 --no-minify, THEN runnning the tests fails
d) encountered a declaration exception appears to be a jest-related exception

[jogold]

a) new lambdaNodejs.NodejsFunction by default builds the handler code next to the source code, not in ./.build as your command implies

Yes I know, the point was to see if this is a parcel error and what kind of error. In this case the out-dir doesn't matter.

deleting any build files, then modifying your command's out-dir to parcel build src/restart-service/index.ts --out-dir src/restart-service/.build --out-file index.js --global handler --target node --bundle-node-modules --log-level 2 --no-minify, THEN runnning the tests fails

The lambdaNodejs.NodejsFunction builds in a subfolder of src/restart-service/.build. Again just a test here to see if it's parcel related. Looks like it's not?

[mishabruml]

Ok, cheers, yes it seems like parcel can build from the command line just fine. The problem appears to be when parcel is called programatically via https://github.com/jogold/aws-cdk/blob/master/packages/@aws-cdk/aws-lambda-nodejs/lib/builder.ts

[mishabruml]

I put a console.log() in https://github.com/jogold/aws-cdk/blob/a2a738cf5df369b92c4039fc7442adddc2350180/packages/%40aws-cdk/aws-lambda-nodejs/lib/builder.ts#L102-L104 and that's where the error is coming from. No parcel error, and nothing on stdout

[jogold]

can you console.log() stderr there ?

[mishabruml]

it logs out <Buffer> 😆

npx jest run-in-band fixed it 🙄

[jogold]

it logs out <Buffer>

stderr.toString()

[mishabruml]

it logs out <Buffer>

stderr.toString()

logs out nothing. the stderr buffer is empty, presumably because the problem is actually that the parcel process can't even spawn properly due to lack of memory