Terraform-provider-aws: aws_sns_topic_subscription doesn't support unconfirmed endpoints

[lxop]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

Terraform v0.11.11

• provider.aws v1.60.0

Affected Resource(s)

• aws_sns_topic_subscription

Expected Behavior

A subscription is created with an email endpoint (or non-auto-confirming HTTP endpoint). If the subscription is not confirmed, then the endpoint simply doesn't receive any messages.

Actual Behavior

The resource doesn't allow these subscriptions. The resource docs note that

These are unsupported because the endpoint needs to be authorized and does not generate an ARN until the target email address has been validated. This breaks the Terraform model and as a result are not currently supported.

However the SNS API includes a parameter ReturnSubscriptionArn which can be set to true to request that the ARN is returned even if the subscription isn't confirmed. As such, an ARN _is_ generated before the target email address has been validated, and this should then fit into Terraform's model.

[lorengordon]

Yes, please! 😁

[lorengordon]

FWIW, looks like this is a new-ish feature of the service, became available in the SDK in June 2018.

• https://github.com/aws/aws-sdk-go/blob/master/CHANGELOG.md#release-v11360-2018-06-01

[lxop]

Starting to look at this

[lxop]

It's not looking pretty - creation appears to work fine, but it all falls over in the destruction department. It is not possible (by any means, API or console) to delete a subscription before it is confirmed. This seems like a fundamentally pointless restriction and I have asked AWS why it is the case, but it seems to have been this way for a long time so I'm not expecting any changes any time soon.

It would be possible to simply accept this restriction and report the error to the user during destruction, but this would then mean that an infrastructure deployment would be impossible to destroy if it includes an unconfirmed subscription. Making the failure non-fatal would mean that destroying a subscription does not necessarily destroy that subscription (if it were unconfirmed when the destroy command was performed, then the recipient subsequently confirmed the subscription, then it would be active but untracked by terraform).

So I'm not entirely sure what to do about this now. The HTTP/S subscription presumably suffers from this same problem if I specify auto-accepts on a subscription that doesn't actually auto-accept. I might have to look into how that is handled.

[bryanlalexander]

@lxop any additional updates on this? It seems these are automatically deleted after 72 hours if not yet confirmed. This edge-case may be acceptable or perhaps a hard-fail option to hard-fail if cleanup is not possible.

resource "aws_sns_topic_subscription" "topic-subscription" {
  topic_arn       = "${aws_sns_topic.topic.arn}"
  protocol        = "email"
  endpoint        = "${var.email_address}"
  require_cleanup = true
}

[alex-aire]

@bryanlalexander no update yet - I haven't had much time to work on it. I was/am considering this approach, but I'm not sure about how good it is to introduce a failure-mode to the destroy operation. It would need to also have a way to change the requirement after removing the resource block, otherwise a user could find themself stuck unable to destroy their infrastructure because the resource block that is failing doesn't exist anymore.
In saying that, I suppose the user could simply re-add a resource block with the appropriate name, and update the requirement there.

[bryanlalexander]

@alex-aire I'm happy to invest the time, if we can come to an agreeable solution. @bflad do you have a preference for a path forward on this item?

[jonjohnston]

Any progress on this? I would love to have this included.

[crobo1337]

Not sure if I'm having the same issue, or a slightly different one with the same potential solution. I'm registering a subscription with the pagerduty HTTPS API, the subscription auto confirms, and I can confirm that in the console, but every subsequent build fails with "Error: Protocol http/https is only supported for endpoints which auto confirms!"

[sagagliardo]

I'm also having similar issues to @crobo1337 with similar messages, setting up pager duty subs.

[devonbleak]

@crobo1337 @sagagliardo Re the PagerDuty issues, there are 2 different URLs in the docs: https://events.pagerduty.com/x-ere/{INTEGRATION_KEY} in the text and https://events.pagerduty.com/integration/{INTEGRATION_KEY}/enqueue in the screenshots. Only the second one (/integration/{INTEGRATION_KEY}/enqueue) would work for me. Hope that helps.