Terraform-provider-aws: Lambda Layers - Terraform deploys new version and removes the old one

[solveretur]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

Terraform v0.11.11

Affected Resource(s)

• aws_lambda_layer_version

Terraform Configuration Files

provider "aws" {
  region = "eu-central-1"
}

# If archive file is changed, updates zip to S3 repository
resource "aws_s3_bucket_object" "object_lambda_common_layer" {
  bucket = "dev-dwh-listings-dw-stst"
  key = "lambda/layers/lambda_common_layer/lambda_common_layer.zip"
  source = "${data.archive_file.layer_zip_lambda_common_layer.output_path}"
  depends_on = [
    "data.archive_file.layer_zip_lambda_common_layer"]
}

data "archive_file" "layer_zip_lambda_common_layer" {
  type = "zip"
  source_dir = "../../src"
  output_path = "../../lambda_common_layer.zip"
}

resource "aws_lambda_layer_version" "lambda_common_layer" {
  layer_name = "lambda_common_layer"
  s3_bucket = "${aws_s3_bucket_object.object_lambda_common_layer.bucket}"
  s3_key = "${aws_s3_bucket_object.object_lambda_common_layer.key}"
  s3_object_version = "${aws_s3_bucket_object.object_lambda_common_layer.version_id}"
  description = "Common layer providing logging"
  compatible_runtimes = ["python3.6"]
}

terraform {
  backend "s3" {
    bucket         = "tfstate-dev"
    region         = "eu-central-1"
    key            = "service/lambda/layers/lambda_common_layer.tfenv"
    dynamodb_table = "terraform_locks"
  }
}

Debug Output

Panic Output

Expected Behavior

If something changes in the code It should just add lambda layer new version and keep the old one

Actual Behavior

If something changes in the code, It adds lambda layer new version but everytime it also deletes the old one.

Steps to Reproduce

1. terraform apply

Important Factoids

• S3 bucket where I store layers zip has versioning enabled
• If nothing has changed in the code, it does not do anything which is correct

References

• https://stackoverflow.com/questions/54648704/terraform-deletes-old-aws-lambda-layer-version-instead-of-creating-new/

[amancevice]

+1 for this. It creates a really undesirable situation where Lambdas with old versions of layers will break when the layer is updated.

[asimb123]

+1

[gchamon]

Just arrived at a similar issue.

In my case I have the code deployment stage separated from the infrastructure deployment stage, so when creating the lambda infrastructure I don't have access to the actual code.

In my case, deploying a version and then updating it with ansible results in a layer version that terraform is unaware of.

My solution, which maybe good for your usecase too, is to manage layer versions _outside_ terraform, and import them using Data Sources, like this:

• Compile, pack and deploy lambda code

aws s3 cp {{ lambda_function_release_file }} s3://{{ lambda_s3_bucket }}/{{ s3_key }}

• Create new lambda layer version

aws lambda publish-layer-version \
          --layer-name 'my-layer' \
          --content S3Bucket={{ lambda_s3_bucket }},S3Key={{ s3_key }} \
          --compatible-runtimes {{ lambda_function.compatible_runtimes }}

• Access the version in terraform:

data "aws_lambda_layer_version" "existing" {
  layer_name = "my-layer"
}

There is a tradeoff in it though. You won't be managing a part of your infrastructure with it though, at least not directly, but at least the older layer versions won't be in jeopardy when updating lambda layer code.

A workaround for this would be to trigger code when deleting a function, so that its layers are also deleted, like so:

resource "aws_lambda_function" "example" {
  # ... other configuration ...
  layers = ["${aws_lambda_layer_version.example.arn}"]
}

resource "null_resource" "delete-layers" {
  depends_on = [aws_lambda_function.example]

  provisioner "local-exec" {
    when = "destroy"
    command = "aws lambda delete-layer-version --layer-name my-layer --version-number 1"
  }
}

From the documentation, however, it is stated that "when you delete a layer version, you can no longer configure functions to use it. However, any function that already uses the version continues to have access to it."[1] So it should not interfere with functions already configured with older layer versions when terraform deletes layer versions in order to update code.

[1] https://docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html#configuration-layers-manage

[vfilimonov]

From the documentation, however, it is stated that "when you delete a layer version, you can no longer configure functions to use it. However, any function that already uses the version continues to have access to it."[1] So it should not interfere with functions already configured with older layer versions when terraform deletes layer versions in order to update code.

That is indeed true. However it will prevent you to redeploy existing lambda. E.g. if you have some CI/CD routines that deploy some lambdas automatically, you would actually have to go and modify the layer requirements for all of them, otherwise all deploys will crash.

Furthermore, if the new version of layer brings some incompatibility issues (e.g. the lambdas working with layer-v1 would be broken after being updated to layer-v2) there would be no easy way to resolve the problem.

@CGreenburg I saw you've pushed a PR 👍👍
Are there any news on it?

[abhilashnarayanan]

Hi Team, when this fix will be released?
@CGreenburg

[CGreenburg]

It's waiting on integration tests I think. I haven't added them in that merge request which is why it can't move forward. If anyone is willing to add them, feel free to fork and submit a new pull request. Thanks!

[abhilashnarayanan]

Thanks for the information! the fix is much needed for our Organization. Will you be able to work on this or can you help me where/what the impacted component is and the necessary changes (on a high level) -as i haven't worked on this earlier.