Aws-cdk: [@aws-cdk/aws-lambda-nodejs] Unable to run lambda locally with `sam local` after updating to [email protected]

[priceld]

I'm using @aws-cdk/aws-lambda-nodejs to bundle JS lambda code. However, after updating aws-cdk to v1.61.0 (or higher) I can no longer run sam local with the template that cdk synth --no-staging generates. I believe this is because f5c9124 changed asset-staging.ts and now the template uses a different aws:asset:path for nodejs lambdas.

This is the Metadata of a working template:

Metadata:
  aws:cdk:path: TestStack/func/Resource
  aws:asset:path: /Users/lprice/work/playground/demo-aws-cdk-asset-path-bug/.cdk.staging/asset-bundle-IF4IYG
  aws:asset:property: Code

Versus that of a failing template:

Metadata:
  aws:cdk:path: TestStack/func/Resource
  aws:asset:path: asset.eaa6b4d3a12be11c844440afe5ee48ce3999505afaa441be0292009b5f7f2766
  aws:asset:property: Code

As you can see the path is different. When SAM tries to mount the lambda code to the docker image, this new relative path causes it to use a folder that does not contain the lambda code. The result is that when the lambda invoked, the code cannot be found and I get this error:

2020-09-15T14:52:59.899Z    undefined   ERROR   Uncaught Exception  {"errorType":"Runtime.HandlerNotFound","errorMessage":"index.handler is undefined or not exported","stack":["Runtime.HandlerNotFound: index.handler is undefined or not exported","    at Object.module.exports.load (/var/runtime/UserFunction.js:144:11)","    at Object.<anonymous> (/var/runtime/index.js:43:30)","    at Module._compile (internal/modules/cjs/loader.js:1138:30)","    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1158:10)","    at Module.load (internal/modules/cjs/loader.js:986:32)","    at Function.Module._load (internal/modules/cjs/loader.js:879:14)","    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)","    at internal/main/run_main_module.js:17:47"]}
START RequestId: f0477efb-b72d-158b-0cdc-cc3d07557371 Version: $LATEST
END RequestId: f0477efb-b72d-158b-0cdc-cc3d07557371
REPORT RequestId: f0477efb-b72d-158b-0cdc-cc3d07557371  Init Duration: 131.03 ms    Duration: 1.06 ms   Billed Duration: 100 ms Memory Size: 128 MB Max Memory Used: 40 MB  

{"errorType":"Runtime.HandlerNotFound","errorMessage":"index.handler is undefined or not exported"}

Reproduction Steps

I created this repo to demonstrate this bug and provide steps to reproduce.

What did you expect to happen?

I expected the lambda to run successfully.

What actually happened?

I see an error when trying to invoke the lambda:

{"errorType":"Runtime.HandlerNotFound","errorMessage":"index.handler is undefined or not exported"}

Environment

• CLI Version : 1.61.0 (build 72e6727)
• Framework Version: 1.61.0 (build 72e6727)
• Node.js Version: v12.18.3
• OS : Mac OS X 10.14.6 (Build: 18G6020)
• Language (Version): Javascript

Edit: Remove --no-staging flag from cdk synth
Edit: Undo previous edit.

---

This is :bug: Bug Report

[dave-graham]

Think this maybe linked to an issue I have at https://github.com/aws/aws-cdk/issues/9189 too. Local dev through "cdk synth --no-staging > template.yaml" then "echo '{}' | sam local invoke LambdaName --env-vars environment.json" isn't possible at the moment.

[priceld]

Good point @dave-graham. I think I am using --no-staging incorrectly in this scenario, but this is still not working even when not using --no-staging. I would like to see both of these issues resolved: it would be great to be able to run the dev version for debugging, but also to be able to run the bundled code locally.

[priceld]

I stand corrected, it looks like sam local only works prior to v1.61.0 when using --no-staging because it creates a template where the lambda's code path is an absolute, staging path (that actually exists). But according to #9189, this behavior is wrong to begin with.

[eladb]

@jogold thoughts?

[ryan-mars]

@priceld Do you have a workaround at the moment?

[priceld]

@ryan-mars, no workaround at the moment. We are just pinned to v1.60.0 for now.

[jogold]

@priceld can you summarize exactly what you expect for aws:asset:path both with and without --no-staging?

[ryan-mars]

Is the issue that aws:asset:path is a relative path, or that sam local invoke can't be told where to begin a search for assets (i.e. cwd or something)?

Making aws:asset:path an absolute path makes the cloud artifacts less portable. Is portability of generated artifacts a goal of CDK?

[priceld]

@jogold I guess it depends on the the purpose of the --no-staging flag. Based on the help documentation, it seems like output shouldn't be copied at all when using --no-staging but this doesn't seem to be happening (per #9189).

But @ryan-mars is getting at my point - what I would really like to do (and what I've lost the ability to do after v1.61.0) is be able to run sam local with my CDK template and test lambdas locally. I'm not sure if the answer is doing something with aws:asset:path or figuring out how to tell sam local where to search for assets.

[ryan-mars]

@priceld If for any reason in the future you need to patch relative paths to assets in a yaml file generated by CDK I'm using this.

I know it's a little late and the issue is already closed.

const yaml = require('js-yaml')
const fs = require('fs')
const path = require('path')
const { CLOUDFORMATION_SCHEMA } = require('cloudformation-js-yaml-schema')

const yamlPath = path.resolve('template.yaml')
if (!fs.existsSync(yamlPath)) throw `Your YAML isn't there: ${yamlPath}`

const template = yaml.safeLoad(fs.readFileSync(yamlPath, 'utf8'), {
  schema: CLOUDFORMATION_SCHEMA
})

const countPatched = Object.keys(template.Resources)
  .filter((resource) => template.Resources[resource].Type === 'AWS::Lambda::Function')
  .reduce((count, resource) => {
    const oldAssetPath = template.Resources[resource].Metadata['aws:asset:path']
    if (fs.existsSync(path.resolve(oldAssetPath))) {
      console.log(`Skipping ${resource} already a valid path\n${oldAssetPath}`)
      return count
    }
    const newAssetPath = path.resolve('cdk.out', oldAssetPath)
    if (!fs.existsSync(newAssetPath)) throw `There's no there, there: ${newAssetPath}`
    console.log(`Fixing path for ${resource}`)
    console.log(`${oldAssetPath} --> ${newAssetPath}`)
    template.Resources[resource].Metadata['aws:asset:path'] = newAssetPath
    return (count += 1)
  }, 0)

console.log(`Fixed ${countPatched} resources.`)

if (countPatched > 0) {
  console.log(`Writing ${yamlPath}`)
  fs.writeFileSync(yamlPath, yaml.dump(template, { schema: CLOUDFORMATION_SCHEMA }))
} else {
  console.log('Nothing to do, not touching the file.')
}