Aws-sdk-js: Build a public Lambda Layer on every 5th (10th?) release

[GeekStocks]

A similar issue was raised in 2017 and I have read that answer. I have also posted in the AWS team forums as recommended. _However, this issue is a perennial problem that just never seems to get addressed._ We need a change to the status quo, not more referrals to a forum that, _I think we all know_, is quite lacking.

So you've just released version 2.400.0 in the last few days (kudos!). Running this code in Lambda reveals the problem:

console.log(AWS.VERSION) // 2.290.0

As of now, Lambda is 110 versions behind!

_All that kewl stuff introduced in November at AWS re:Invent 2018 has yet to arrive in the Lambda environment (that would be version 2.367.0)_. We peasants will never know why the Lambda team is either unable or unwilling to provide an up-to-date experience, but with Lambda Layers now a reality, I am wondering if they can be bypassed.

Would it be possible for this team to release the SDK as a public Lambda Layer on every 5th (or 10th?) release? Doing so would fix this problem without resorting to draconian measures like having to include the current SDK in our builds. Thoughts?

[klaytaybai]

Hi @GeekStocks, thanks for being involved in this discussion. The Lambda team is aware of the issue, and they are hoping to find a good long term solution. In the meantime, they do recommend your draconian measure of including the current SDK in your builds. We'll have some more discussion about whether the JavaScript SDK team can help out with Lambda Layers releases and other solutions.

[GeekStocks]

Probably everyone reading this has cut a software release they wish they could have back; its the business. I certainly am not advocating for every release to make it to an officially supported Lambda Layer. I am making these two points:

1. 110 versions behind is not acceptable;
2. lagging 3+ months behind the release of major new enhancements is not acceptable.

So for this to work, the JavaScript SDK team would have to decide upon some measure of a release's "maturity" to be a Lambda Layer candidate. _And maturity isn't a driving factor when re:Invent rolls around with a shiny new toy_. semver isn't going to help you much here — and, thank you for that btw ;-)

Unless there are considerations I am not aware of, this would seem to be a win-win solution, would it not? Adding a few MB's to every build is wasteful.

[AllanZhengYP]

Hi @GeekStocks,

Thank you for the advice. We are working with Lambda team to get the SDK updated and figure out a long term solution to solve this. Publishing public Lambda layer can be a good solution. We also want to make customers avoid extra configurations.

I will response here if we have any updates on this issue.

[GeekStocks]

We also want to make customers avoid extra configurations.

_That's a noble goal, and one I share with you._ The way that goal gets achieved best is for the Lambda team to become serious about delivering an up-to-date experience. There is zero extra configuration when the SDK is kept current-ish in Lambda. We'll call this "Plan A", and I am heartened to hear that your team is speaking with the Lambda team.

_This issue was opened to propose a "Plan B",_ because "Plan A" hasn't been working since 2017 (and prior, too). In any proposed "Plan B", there is _going_ to be extra configuration involved somewhere. Having a current-ish SDK as a public Lambda Layer _at least keeps the "extra configuration" from adding megabytes to EVERY build._

I'm _super stoked_ that you are talking to the Lambda team about "Plan A". An SDK refresh in Lambda timed to support newly announced capabilities would be a much more customer friendly modus operandi.

[adjourn]

This is a cause I can get behind.

Sad to see my bundle sizes to drastically go up and it adds a lot of confusion to bundler configuration, i.e which APIs should be included into bundle and which shouldn't. At the very least I would expect an update after Re:Invent which usually introduces many new services and features.

But I recognize that updating SDK is probably not as easy as it looks, why else would it lag behind..

[GeekStocks]

Can anyone ( @klaytaybai or @AllanFly120 or @srchase ) provide an update on how the discussions with the Lambda team have faired? This idea has now been open for two months and a status update would be appreciated. _Thanks!_

[zoellner]

I appreciate you're looking for a longterm solution. In the meanwhile (since the official nodejs runtime is now over 5 months behind), how about a short/midterm solution as suggested by OP and publish a publicly accessible lambda layer with the latest aws-sdk?
It'd be a day or two of work to automate that and safe your customers a lot of time.

As an example:
Cloudfront recommends using cache policies to be able to have consistent policies across an organization but the implementation is rather cumbersome and requires a cache policy id (which is different in every account) rather than allowing to refer to it by cache policy name.
So I have to create custom lambda to allow me to lookup those cache policy ids by name, that's rather quick and I can deploy the custom resource lambda through control tower custom configurations. Only problem: the aws-sdk is so far behind that cloudfront.listCachePolicies is not defined. So wasting an hour or two here to get a rather simple lambda workaround up just to hit another roadblock. A lambda layer with the latest aws-sdk would solve this quickly. Other solutions here aren't feasible since the lambda is embedded as Zipfile in the CloudFormation template. So the only option is to publish that layer myself - and that's just a waste of time for every single customer that you could eliminate by implementing a short/midterm solution as proposed while someone somewhere is working on a longterm solution that we get to enjoy in a few years when nodejs v28 is around.

[GeekStocks]

_Why this isn't viewed as a win-win option I'll probably never know._ Is AWS foot-dragging because they like the cash from selling all that wasted storage space from these SDK copies?

22 months have now gone by since this idea was proposed; look at all the waste that has occurred since then. And right now we are in that spot I mentioned in the OP where a _re:Invent_ has just taken place, there are shiny new toys to play with, and you can't use them in Lambda without bundling the latest SDK in your build.

There is no better use case for a public layer than a current(ish) SDK.