Aws-cdk: (aws-lambda-nodejs): integration tests failing

[robertd]

:question: General Issue

The Question

Is anyone else experiencing issues with running integration tests on aws-lambda-nodejs module?

Btw... I'm running clean builds using docker and partial builds outlined in contribution guide... same results.

@aws-cdk/aws-lambda-nodejs: Bundling asset cdk-integ-lambda-nodejs-dependencies/external/Code/Stage...
@aws-cdk/aws-lambda-nodejs: /asset-input/node_modules/.bin/esbuild: /asset-input/node_modules/.bin/esbuild: cannot execute binary file
@aws-cdk/aws-lambda-nodejs: /Users/rdjurasaj/code/robertd/aws-cdk/packages/@aws-cdk/core/lib/asset-staging.js:309
@aws-cdk/aws-lambda-nodejs:             throw new Error(`Failed to bundle asset ${this.node.path}, bundle output is located at ${bundleErrorDir}: ${err}`);
@aws-cdk/aws-lambda-nodejs:             ^
@aws-cdk/aws-lambda-nodejs: Error: Failed to bundle asset cdk-integ-lambda-nodejs-dependencies/external/Code/Stage, bundle output is located at /Users/rdjurasaj/code/robertd/aws-cdk/packages/@aws-cdk/aws-lambda-nodejs/test/cdk-integ.out/bundling-temp-20ef025878490e89e55410b3389cafecbc28e7782e3f1e15aab6c54679de3899-error: Error: docker exited with status 126
@aws-cdk/aws-lambda-nodejs:     at AssetStaging.bundle (/Users/rdjurasaj/code/robertd/aws-cdk/packages/@aws-cdk/core/lib/asset-staging.js:309:19)
@aws-cdk/aws-lambda-nodejs:     at AssetStaging.stageByBundling (/Users/rdjurasaj/code/robertd/aws-cdk/packages/@aws-cdk/core/lib/asset-staging.js:183:14)
@aws-cdk/aws-lambda-nodejs:     at stageThisAsset (/Users/rdjurasaj/code/robertd/aws-cdk/packages/@aws-cdk/core/lib/asset-staging.js:64:41)
@aws-cdk/aws-lambda-nodejs:     at Cache.obtain (/Users/rdjurasaj/code/robertd/aws-cdk/packages/@aws-cdk/core/lib/private/cache.js:28:17)
@aws-cdk/aws-lambda-nodejs:     at new AssetStaging (/Users/rdjurasaj/code/robertd/aws-cdk/packages/@aws-cdk/core/lib/asset-staging.js:88:48)
@aws-cdk/aws-lambda-nodejs:     at new Asset (/Users/rdjurasaj/code/robertd/aws-cdk/packages/@aws-cdk/aws-s3-assets/lib/asset.js:25:25)
@aws-cdk/aws-lambda-nodejs:     at AssetCode.bind (/Users/rdjurasaj/code/robertd/aws-cdk/packages/@aws-cdk/aws-lambda/lib/code.js:225:26)
@aws-cdk/aws-lambda-nodejs:     at new Function (/Users/rdjurasaj/code/robertd/aws-cdk/packages/@aws-cdk/aws-lambda/lib/function.js:95:33)
@aws-cdk/aws-lambda-nodejs:     at new NodejsFunction (/Users/rdjurasaj/code/robertd/aws-cdk/packages/@aws-cdk/aws-lambda-nodejs/lib/function.js:45:9)
@aws-cdk/aws-lambda-nodejs:     at new TestStack (/Users/rdjurasaj/code/robertd/aws-cdk/packages/@aws-cdk/aws-lambda-nodejs/test/integ.dependencies.js:12:9)
@aws-cdk/aws-lambda-nodejs: Error: Command exited with status 1
@aws-cdk/aws-lambda-nodejs:     at exec (/Users/rdjurasaj/code/robertd/aws-cdk/tools/cdk-integ-tools/lib/integ-helpers.js:336:15)
@aws-cdk/aws-lambda-nodejs:     at IntegrationTest.cdkSynthFast (/Users/rdjurasaj/code/robertd/aws-cdk/tools/cdk-integ-tools/lib/integ-helpers.js:88:19)
@aws-cdk/aws-lambda-nodejs:     at main (/Users/rdjurasaj/code/robertd/aws-cdk/tools/cdk-integ-tools/bin/cdk-integ-assert.js:19:33)
@aws-cdk/aws-lambda-nodejs: Error: cdk-integ-assert exited with error code 1
@aws-cdk/aws-lambda-nodejs: Tests failed. Total time (6m24.7s) | /Users/rdjurasaj/code/robertd/aws-cdk/node_modules/jest/bin/jest.js (6m7.6s) | cdk-integ-assert (17.1s)
@aws-cdk/aws-lambda-nodejs: !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
@aws-cdk/aws-lambda-nodejs: npm ERR! code 1
@aws-cdk/aws-lambda-nodejs: npm ERR! path /Users/rdjurasaj/code/robertd/aws-cdk/packages/@aws-cdk/aws-lambda-nodejs
@aws-cdk/aws-lambda-nodejs: npm ERR! command failed
@aws-cdk/aws-lambda-nodejs: npm ERR! command sh -c cdk-test
@aws-cdk/aws-lambda-nodejs: npm ERR! A complete log of this run can be found in:
@aws-cdk/aws-lambda-nodejs: npm ERR!     /Users/rdjurasaj/.npm/_logs/2020-12-10T22_29_55_269Z-debug.log
@aws-cdk/aws-lambda-nodejs: error Command failed with exit code 1.
@aws-cdk/aws-lambda-nodejs: Verifying integ.dependencies.js against integ.dependencies.expected.json ... info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
lerna ERR! yarn run build+test exited 1 in '@aws-cdk/aws-lambda-nodejs'
lerna WARN complete Waiting for 8 child processes to exit. CTRL-C to exit immediately.

real    29m5.385s
user    81m32.599s
sys 7m24.827s
❌  Last command failed. Scroll up to see errors in log (search for '!!!!!!!!')

Environment

• CDK CLI Version: 1.77.0
• Node.js Version: 15.4.0
• OS: macOS Mojave

Other information

[jogold]

Looks like an issue with your Docker installation/setup on your Mac.

This specific integ test runs in Docker (forceDockerBundling is set to true).

[robertd]

@jogold I've installed docker through brew cask. I didn't do anything special really. My current docker for Mac version is 3.0.0.

[hassanazharkhan]

@jogold I am also facing the same issue on my Mac

[jogold]

What do you get when you run the following?

cd packages/@aws-cdk/aws-lambda-nodejs/lib/
docker build -t lambda-nodejs-docker .
docker run lambda-nodejs-docker esbuild --version
docker run lambda-nodejs-docker /bin/bash -c "esbuild --version"

[robertd]

@jogold

➜  lib git:(master) docker build -t lambda-nodejs-docker .
Sending build context to Docker daemon  108.5kB
Step 1/10 : ARG IMAGE=amazon/aws-sam-cli-build-image-nodejs12.x
Step 2/10 : FROM $IMAGE
 ---> 2eada5d9f2c4
Step 3/10 : RUN npm install --global [email protected]
 ---> Using cache
 ---> ecb36039b46f
Step 4/10 : ARG ESBUILD_VERSION=0
 ---> Using cache
 ---> 3abfa4e837fa
Step 5/10 : RUN npm install --global --unsafe-perm=true esbuild@$ESBUILD_VERSION
 ---> Using cache
 ---> 2708c6768759
Step 6/10 : RUN mkdir /tmp/npm-cache &&     chmod -R 777 /tmp/npm-cache &&     npm config --global set cache /tmp/npm-cache
 ---> Using cache
 ---> 790411e7cdc4
Step 7/10 : RUN mkdir /tmp/yarn-cache &&     chmod -R 777 /tmp/yarn-cache &&     yarn config set cache-folder /tmp/yarn-cache
 ---> Using cache
 ---> ed56042e40f1
Step 8/10 : RUN npm config --global set update-notifier false
 ---> Using cache
 ---> ee8315d30d95
Step 9/10 : RUN /sbin/useradd -u 1000 user && chmod 711 /
 ---> Using cache
 ---> d96bd80ef741
Step 10/10 : CMD [ "esbuild" ]
 ---> Using cache
 ---> 3ff3824d39ec
Successfully built 3ff3824d39ec
Successfully tagged lambda-nodejs-docker:latest
➜  lib git:(master) docker run lambda-nodejs-docker esbuild --version
0.8.21
➜  lib git:(master) docker run lambda-nodejs-docker /bin/bash -c "esbuild --version"
0.8.21

[jogold]

OK I think I got it.

Can you try updating bundling.ts like this? Then rebuild and rerun the integ test.

[jogold]

It doesn't fail on Linux because esbuild is symlinked even in the mounted volume.

[robertd]

@jogold That seems to work on targeted aws-lambda-nodejs build... Trying out the clean full docker build now.

edit: full docker build worked too. Thanks @jogold 👍

[github-actions[bot]]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.