Amplify-cli: Add support for deploying/managing lambda layers

[mrcoles]

Is your feature request related to a problem? Please describe.

I was running into a problem where I wanted to share code between two different lambda functions.

Describe the solution you'd like

I would like to easily manage/deploy/use/test with lambda layers in my AWS Amplify project to share code between two different lambda functions.
https://aws.amazon.com/blogs/aws/new-for-aws-lambda-use-any-programming-language-and-share-common-components/

Describe alternatives you've considered

I’ve thought about symlinking some shared files, but this feels really jenky and brittle.

Additional context

By making this part of the AWS Amplify CLI, I can easily have all my code managed in one clean way, and be sure that when I deploy things, everything has the latest up-to-date code running and is deployed in a clean manner.

[willkara]

Any movement on this? @kaustavghosh06 it's out of my skill-set but I can help out.

[hisham]

+1. Looks like I need this too.

[wellrightamplify]

+1

[artista7]

@kaustavghosh06, I am using Lerna to link shareable packages in different packages. But I kinda bumped on this one because apparently amplify push don't support symlinks. I don't know if lambda layer is what is the solution. My question is would it be possible to bundle local dependencies with amplify push?

[hisham]

@artista7 for now you can resolve symlinks manually through a command line script before amplify push. See my solution at #1696.

@kaustavghosh06 not sure what the timeline for supporting lambda layers is, but perhaps as a stop gap, the amplify lambda packager should be made to support symlinks in node_modules. This would resolve these interim issues people like me and @artista7 are facing when we use lerna / npm link / yarn workspaces.

[ramsbaron]

+1

[psyrenpark]

I want found nice solution to this problem.
aws sam support lmabda layer.
similar solution use /opt dictectory.

[jieming]

I would be super nice to have this implemented! +1
Sharing code between lambdas is essential on backend architecture in my current projects.

[mrcoles]

@kaustavghosh06 can I use the new lambda build options feature as a workaround for this to effectively share code between lambdas? For example, I can have a shared code directory somewhere in my codebase and various lambda functions could import that safely as long as they’re getting es6 or Typescript transpiled (since the final code will end up in their lambda directory)?

[visusnet]

@mrcoles I was the one who implemented build hooks initially and I did it for two purposes: Getting Babel to run before push and copying shared files (in our case: the graphql/ folder). We commit the copied files, i.e. they are always there, even if we have a fresh clone of the repository. That way, IDEs will find the files and won‘t complain about missing files. However, build hooks are only executed if there are code changes in the respective lambda itself. There is no way to update the function automatically if there are only changes in the shared code.

[mrcoles]

@visusnet OK, wishful thinking 😜 Btw, how does the is-lambda-changed logic work (is it mtime based or content hashing based)?

[visusnet]

@mrcoles It's mtime-based. Actually, it's pretty straight forward. You can find it here: https://github.com/aws-amplify/amplify-cli/blob/master/packages/amplify-provider-awscloudformation/lib/build-resources.js#L31

[diego-palmeira]

Hi, I need the Amplify function layers too. Have you any roadmap to include it?
Could you suggest how to share code between lambda functions inside a same Amplify project?
e.g.

[my-project]
----|- amplify
--------|-backend
------------|- function
------------------|- [my-function A]
------------------|- [my-function B]
------------------|- [shared-code?] <-- layer code?

Any recommendation?

As a possible workaround... is it possible to manage layers with aws cli and rest of the project with amplify? for instance by publishing the layer using aws cli and then add the layer arn reference to the lambda cloud formation template which is inside amplify folder?

Thank you so much for your help

[houmark]

I'd also like to hear of workarounds until Lambda Layers are supported.

For now I have redundant code some places, and been considering to create a private git repository/npm package and then install that in each function. But this would still require npm update of all functions using the package when I do changes to this shared library. So once my shared code lib is "stable" and with mininmal ongoing changes/improvements needed, this can be a great option, but if code is changed daily, then it's a fairly large tasks to npm update on all functions using the package.

[hisham]

I was able to workaround this using yarn workspaces with a private npm package hosted on npmjs that can still be locally modified. It's not perfect, for example symlinks need to be resolved before an amplify push so I have a wrapper script that does that for me, and I have to use the 'nohoist' option for lambda function packages. But after working out the various kinks it is working fine now. Probably similar workarounds can be done using npm link. I'm hoping for official support for lambda layers soon!

[thedgbrt]

I’ve been rocking a similar setup. Basically I’ve created a monorepo with Lerna + yarn workspaces, with different functions for reuse between functions of my amplify project. It’s useful but not ideal. Especially since some of them are close to 50M when packaged (using pupeteer), it will require a full upload each time. Lambda layers would eliminate that problem.

[xoadrian]

+1

[tabularaza27]

+1

[emanuelherrmann]

:+1:

[yi-hoffnung]

Use of Custom Cloudformation Stacks. That's my way
https://aws-amplify.github.io/docs/cli-toolchain/quickstart#custom-cloudformation-stacks

[shanmukhm]

Hey guys! When can we expect support for lambda layers in amplify-cli?

[cle5eland]

+1. about to do some really sketchy rimraf-ing that I'm not stoked about

[renebrandel]

Hi everyone! We're actively looking into this now. We don't have an ETA just yet but we'll keep this thread updated!

[renebrandel]

🚀 We've recently made some good progress on this and would love to get some early user feedback!

➡️ The Amplify CLI team would love to give you hands-on walkthrough about this feature. If you're interested, please DM me on Twitter or on Discord "@renebrandel"

👀 Watch out for branch: "lambdaLayers". We still have a few quirks to work out around amplify push but would love to hear your feedback!

[renebrandel]

Hi all - we've gotten quite a bit of customer feedback and are in the final steps to enabling this functionality. If you're curious here's a sneak peak for the experience. https://github.com/aws-amplify/docs/pull/2000

[renebrandel]

🚀 Launch announcement!

Lambda layers in Amplify CLI has officially been shipped with Amplify CLI v4.23.0
Check out our blog post: https://aws.amazon.com/blogs/mobile/how-to-use-lambda-layers-with-the-amplify-cli/
Check out our docs: https://docs.amplify.aws/cli/function/layers

Thank you all for providing feedback throughout the development and special thanks to all the early testers.
Shout out to devs that actively worked on this: @jhockett @akshbhu @edwardfoyle @UnleashedMind @attilah @kaustavghosh06 @undefobj