Terraform-provider-aws: Multiple Lambda Triggers from one S3 Bucket

[hashibot]

_This issue was originally opened by @Timboo89 as hashicorp/terraform#16131. It was migrated here as a result of the provider split. The original body of the issue is below._

---

Terraform Version

0.10.2

Terraform Configuration Files

resource "aws_s3_bucket_notification" "bucket_trigger_lambda" {
  count  = "${length(var.s3_file_prefixs)}"
  bucket = "${var.s3_bucket_id}"

  lambda_function {
    lambda_function_arn = "${aws_lambda_function.lambda.arn}" # ARN of a Lambda function
    events              = ["s3:ObjectCreated:*"]
    filter_prefix       = "${var.s3_file_prefixs[count.index]}"
  }
}

variable "s3_file_prefixs" {
  default = ["path/1/static/", "path/2/static/"]
}

variable "s3_bucket_id" {
  default = "TestBucket"
}

Debug Output

Crash Output

Expected Behavior

Multiple trigger should have been created.

Actual Behavior

Each Trigger tries to override the last one.

Steps to Reproduce

1. Create a lambda function that's ARN you replace in the code snippet
2. Create a S3 bucket that's name you need to replace in the code snippet
3. terraform init
4. terraform apply

[apparentlymart]

Hi @Timboo89!

This is unfortunately a limitation of the underlying API: each S3 bucket can have only one "notification configuration", which is what this resource is creating. Like many things in the S3 API, no distinction is made between "create" and "update" and so Terraform can't tell that it's replacing an existing configuration when it tries to create a new one.

For this resource I think unfortunately the best we can do here is try to be clearer about this limitation in the documentation. :confounded:

---

For your use-case in particular it looks like what you really wanted here was something like described in hashicorp/terraform#7034. Per the current sketch, this might one day be done with a config like this:

# DRAFT: Not yet implemented, and details may change before final release

resource "aws_s3_bucket_notification" "bucket_trigger_lambda" {
  bucket = "${var.s3_bucket_id}"

  dynamic "lambda_function" {
    foreach = "${var.s3_file_prefixs}"
    content {
      lambda_function_arn = "${aws_lambda_function.lambda.arn}" # ARN of a Lambda function
      events              = ["s3:ObjectCreated:*"]
      filter_prefix       = "${dynamic.foreach.value}"
    }
  }
}

Unfortunately we have lots of work to do before we can support the above, since the underlying implementation of the configuration language can't support this currently. We're working on the necessary changes right now, but we have some other steps to get through before we can support this specific feature.

In the mean time, the only viable option is to write the multiple lambda_function blocks out longhand in the configuration; Terraform's current language can't support the dynamic behavior you wanted here.

---

Since we already have that other issue covering the use-case of generating child config blocks, I'm going to tag this as a documentation issue to clarify the fact that each bucket can only have one resource of this type.

[bflad]

Related: #271 and #765

[007]

@apparentlymart is there any ETA on the dynamic fix you mentioned above? https://github.com/hashicorp/terraform/issues/7034 seems to show it's still TBD ~2 years after reporting, with maybe some progress in January but nothing since.

This is the second or third time I've come across this kind of limit / weirdness for TF re: iterators not iterating, and this is the first one that I'm not going to be able to work around easily. Having to hard-code a list isn't going to work for my use-case.

[bflad]

Hi folks, this was closed with a documentation update to include a note at the top of the aws_s3_bucket_notification resource documentation that multiple declarations of this resource is not supported against the same S3 Bucket (which will get deployed when version 2.3.0 is released later this week). We will keep #501 open with the feature request to support multiple resources with the same S3 Bucket, however as noted there, S3 API limitations may prevent that implementation.

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!