Terraform-provider-aws: Better error message for s3_bucket_notification if the prefixes are overlapping for the same event type

[Techbrunch]

Terraform Version

Terraform v0.10.3

Affected Resource(s)

• s3_bucket_notification

Terraform Configuration Files

resource "aws_s3_bucket_notification" "bucket_notification" {
  bucket = "${aws_s3_bucket.access_logs.id}"

  queue {
    queue_arn     = "${aws_sqs_queue.queue.arn}"
    events        = ["s3:ObjectCreated:*"]
    filter_suffix = ".log.gz"
  }
}

Debug Output

Error applying plan:

1 error(s) occurred:

* module.s3.aws_s3_bucket_notification.bucket_notification: 1 error(s) occurred:

* aws_s3_bucket_notification.bucket_notification: Error putting S3 notification configuration: InvalidArgument: Unable to validate the following destination configurations
    status code: 400, request id: XXX, host id: XXX

Expected Behavior

The error should be more explicit as in the console:

Configuration is ambiguously defined. Cannot have overlapping suffixes in two rules if the prefixes are overlapping for the same event type.

Steps to Reproduce

The bucket should already have an overlapping suffix.

[Techbrunch]

Same thing happens with this other error: Unable to validate the following destination configurations. Permissions on the destination queue do not allow S3 to publish notifications from this bucket. (arn:aws:sqs:eu-west-1:xxx)

[sjauld]

We should be able to validate the overlap without hitting the API. I'll put together a PR.

[sjauld]

Wait my mistake, I was thinking of topics, not queues.

[sumedhsakdeo]

I am hitting this issue. Appreciate some help.

aws_s3_bucket_notification.bucket_notification: Error putting S3 notification configuration: InvalidArgument: Unable to validate the following destination configurations

What are some of the possible reasons why this may happen? And some clues to root-cause this.
I don't think we have any overlapping prefixes or suffixes.

[radekg]

@sumedhsakdeo have you tried running with TF_LOG=debug + https://docs.aws.amazon.com/AmazonS3/latest/dev/troubleshooting.html?

[demisx]

Adding TF_LOG=debug definitely helps to understand what's really causing the problem. In my case, there was no permission for S3 bucket event to call lambda function.

[njlr]

Thanks @demisx, the bit that I was missing was indeed the permission for S3 to call the lambda.

This configuration is in the example in the Terraform docs:

resource "aws_lambda_permission" "allow_bucket" {
  statement_id  = "AllowExecutionFromS3Bucket"
  action        = "lambda:InvokeFunction"
  function_name = aws_lambda_function.func.arn
  principal     = "s3.amazonaws.com"
  source_arn    = aws_s3_bucket.bucket.arn
}