Terraform-provider-aws: Breaking change on data source aws_lambda_function introduced in version 2.0.0

[pracucci]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

Terraform: 0.10.8
Terraform AWS provider: 2.11.0

Affected Resource(s)

• Data source aws_lambda_function

Terraform Configuration Files

data "aws_lambda_function" "lambda" {
  function_name = "myFunction"
  qualifier     = ""
}

resource "aws_cloudfront_distribution" "distribution" {
  default_cache_behavior {
    lambda_function_association {
      event_type   = "origin-response"
      lambda_arn   = "${data.aws_lambda_function.lambda.qualified_arn}"
      include_body = false
    }
  }
}

Expected Behavior

• The Lambda function qualified_arn should be qualified by the latest version number instead of $LATEST

Actual Behavior

• The Lambda function qualified_arn is qualified with $LATEST

References

The breaking change has been introduced in the PR https://github.com/terraform-providers/terraform-provider-aws/pull/7663 which has been merged into the provider version 2.0.0.

The dataSourceAwsLambdaFunctionRead() implementation switched from using resourceAwsLambdaFunctionRead() (defined in resource_aws_lambda_function.go) to a new (de-coupled) implementation.

The resourceAwsLambdaFunctionRead() implementation has support to fetch the latest version number in case qualifierExistance is falsy (see code here), while the new one doesn't have it.

Questions, please:

• Is there any interest to fix the breaking change?
• If not, shouldn't we document it in the upgrade guide?
• If not, is there any known workaround to get the latest version number of a Lambda function defined as a data source?

[pracucci]

Is there any interest to fix the breaking change?

@bflad If there's interest in fixing this and keep the old behaviour, I may try to work on a PR.

[ghost]

In order to associate a lambda functions with a CloudFront distribution as Lambda@Edge, it is required to provide a version number.

I would be interested to retrieve the latest fixed version number of a given lambda.
Is this possible currently by using aws_lambda_function data source?

data "aws_lambda_function" "example_lambda_edge" {
  function_name = "example"
  qualifier     = ""
}

resource "aws_cloudfront_distribution" "example_distribution" {
  # ...

  default_cache_behavior {
    # ...

    lambda_function_association {
      event_type   = "viewer-request"
      # Problem: the qualifier here will be "$LATEST" which will not work in that case.
      lambda_arn   = data.aws_lambda_function.example_lambda_edge.qualified_arn
    }
  }
}

[grahamlyus]

I ran into this today, and I found a workaround for my setup. It works as I reapply an alias when I deploy new versions. You can then lookup the version number using aws lambda get-alias:

> aws lambda get-alias --function-name example --name example-alias
{
    "AliasArn": "arn:aws:lambda:us-east-1:XXXXXXX:function:example:example-alias",
    "Name": "example-alias",
    "FunctionVersion": "102",
    "Description": "",
    "RevisionId": "e464023a-e7f6-4715-88fa-e7854488878e"
}

Therefore you can use an external data source to run the command:

data "external" "example_lambda_version" {
 program = ["aws", "lambda" ,"get-alias", "--function-name",  "example", "--name", "example-alias"]
}

data "aws_lambda_function" "example_lambda_edge" {
  function_name = "example"
  qualifier = data.external.example_lambda_version.result.FunctionVersion
}

resource "aws_cloudfront_distribution" "example_distribution" {
  # ...

  default_cache_behavior {
    # ...

    lambda_function_association {
      event_type   = "viewer-request"
      lambda_arn   = data.aws_lambda_function.example_lambda_edge.qualified_arn
    }
  }
}

[ghost]

Interesting thoughts @grahamlyus , by doing that it is needed to update the alias manually after each deploy then?

It sort of replicates the $LATEST qualifier, which cannot be used directly as an alias if I understand correctly (I tried, it didn't pass the validation regexp)?

[grahamlyus]

@flosch-hb Yes, it requires updating the alias after each deploy.

In my case I deploy the lambda in question separate from my terraform setup via claudiajs which has a --version argument to do just that. I think it's common to use it as a tag for the stage, e.g. staging, production etc. https://claudiajs.com/tutorials/versions.html

[rprieto]

If you need to get the latest version number, you can now fetch the version behind an alias (e.g. an alias called latest that you update on every publish). This replaces the need for the AWS CLI call via data "external":

data "aws_lambda_alias" "latest" {
  function_name = "my-lambda-at-edge"
  name = "latest"
}

You can then use data.aws_lambda_alias.latest.function_version directly, or fetch the actual Lambda using:

data "aws_lambda_function" "lambda" {
  function_name = "my-lambda-at-edge"
  qualifier = data.aws_lambda_alias.latest.function_version
}

[brucedvgw]

Thanks @rprieto for this workaround. Unfortunately, I tried your approach but I still get the ...:$LATEST as a version instead of a number when using data.aws_lambda_alias.latest.function_version

[rprieto]

Hi, apologies using the alias might not work. Fetching the Lambda as above works for me, using data.aws_lambda_function.lambda.qualified_arn to get the full ARN including the actual version number.

[brucedvgw]

Thanks @rprieto, I still get $LATEST using data.aws_lambda_function.lambda.qualified_arn. I've worked around it for now :)

[brucedvgw]

So the workaround I did was to ensure that aws_lambda_function had the publish = true set. This then enabled me to use aws_lambda_function.this.qualified_arn to get the arn with the version number.

Hope this helps someone!

[eretica]

Lambda created from terraform with publish=true. recieved arn with number use this aws_lambda_function.this.qualified_arn

Did you know where to change the parameter corresponding to publish=true of terraform in Lambda created from AWS GUI?

[brucedvgw]

If you are using the AWS Console @eretica you will need to create the lambda in us-east-1 N. Virgina. Then you will need to publish a version that will give you a version number.

You can check out the doco here https://docs.aws.amazon.com/lambda/latest/dg/configuration-versions.html

In Terraform you need to make sure that your region is set to us-east-1 and have publish = true so that you can get the version (other than $LATEST) with aws_lambda_function.this.qualified_arn

[nkoterba]

@rprieto I tried your suggested approach:

data "aws_lambda_alias" "latest" {
  function_name = "my-lambda-at-edge"
  name = "latest"
}

data "aws_lambda_function" "latest-lambda"{
  function_name = "my-lambda-at-edge"
  qualifier = data.aws_lambda_alias.latest.function_version
}

My use in my CloudFront distribution definition:

    lambda_function_association {
      event_type = "viewer-request"
      lambda_arn = data.aws_lambda_function.latest-lambda.qualified_arn
    }

When I do terraform apply:

Error: Error getting Lambda alias: ResourceNotFoundException: Cannot find alias arn: my-lambda-at-edge:latest
{
  RespMetadata: {
    StatusCode: 404,
    RequestID: "679426ed-37eb-470c-aba8-57eb5c0cbe5f"
  },
  Message_: "Cannot find alias arn: my-lambda-at-edge:latest",
  Type: "User"
}

Do you actually publish the lambda as it sounds like @brucedvgw does? I don't control or access this lambda so I'm still trying to find a way to get the latest lambda version and apply that to my cloudfront distribution.

[rprieto]

Yes I had to publish the function (publish=true) and maintain a manual alias called latest. It could be any other name which you fetch in data "aws_lambda_alias".