Terraform-provider-aws: Terraform constantly updates resource policy on API Gateway

[geoffwa]

We're seeing an issue where Terraform constantly updates the resource policy of an API gateway:

module.apps.aws_api_gateway_rest_api.segmentation-etl-creation-api: Modifying... (ID: x023a0eez5)
  policy: "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Sid\":\"\",\"Effect\":\"Allow\",\"Principal\":\"*\",\"Action\":\"execute-api:Invoke\",\"Resource\":\"arn:aws:execute-api:us-west-1:999999999999:0123456789a/*\",\"Condition\":{\"StringEquals\":{\"aws:SourceVpc\":\"vpc-bcdef123\"}}}]}" => "{\n  \"Version\": \"2012-10-17\",\n  \"Statement\": [\n    {\n      \"Sid\": \"\",\n      \"Effect\": \"Allow\",\n      \"Action\": \"execute-api:Invoke\",\n      \"Resource\": \"execute-api:/*\",\n      \"Principal\": \"*\",\n      \"Condition\": {\n        \"StringEquals\": {\n          \"aws:SourceVpc\": \"vpc-bcdef123\"\n        }\n      }\n    }\n  ]\n}"

In our terraform module we want to apply a resource policy our API gateway, so we have the following:

data "aws_iam_policy_document" "resource-policy" {
  statement {
    principals {
      type        = "*"
      identifiers = ["*"]
    }
    actions = [
      "execute-api:Invoke",
    ]
    resources = [
      "execute-api:/*",
    ]
    condition {
      test     = "StringEquals"
      variable = "aws:SourceVpc"
      values   = ["${var.vpc_id}"]
    }
  }
}
resource "aws_api_gateway_rest_api" "rest-api" {
  name        = "rest-api"
  description = "API to manage things"
  endpoint_configuration {
    types = ["PRIVATE"]
  }
  policy = "${data.aws_iam_policy_document.resource-policy.json}"
}

According to the Amazon docs we can use this execute-api:/stage/method/part short hand and AWS will expand this to the full ARN of the aws_api_gateway_rest_api instance.

Ideally we'd like to be able to change the resources part of the resource policy to reference the ARN of the aws_api_gateway_rest_api instance directly, like so:

    resources = [
      "${aws_api_gateway_rest_api.rest-api.execution_arn}:/*",
    ]

except that this introduces a cycle between the resource policy and the REST API as both require each other to exist before they can be created:

Error: Cycle: module.apps.data.aws_iam_policy_document.resource-policy, module.apps.aws_api_gateway_rest_api.rest-api

This appears to be due to the unfortunate way that resource policies are stored in AWS. Reading the docs it looks like resource policies don't exist as entities themselves but only as things that hang off a REST API.

At the moment we're working around this by ignoring policy changes - it'd be great if there were a nicer way to do this.

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

• Terraform version 0.11.7
• AWS provider version 1.31.0

Affected Resource(s)

• aws_api_gateway_rest_api

[geoffwa]

Updating a resource policy should also trigger deployment of new stages/deployments of the REST API in order for the resource policy to take effect according to https://docs.aws.amazon.com/apigateway/latest/developerguide/apigateway-resource-policies-create-attach.html

If you update the resource policy after the API is created, you'll need to deploy the API to propagate the changes after you've attached the updated policy. Updating or saving the policy alone won't change the runtime behavior of the API. For more information about deploying your API, see Deploying an API in Amazon API Gateway.

Ugh 😱 😱 😱

[atali]

I have the same issue with the following template :

resource "aws_api_gateway_rest_api" "rest_api" {
  name        = "${var.api_name}"
  body = "${data.template_file.swagger_api.rendered}"

  endpoint_configuration {
    types = ["${var.type}"]
  }

  policy = "${data.template_file.policy_file.rendered}"
}

data "template_file" "policy_file" {
  template = "${file(var.policy_file)}"
  vars = "${var.variables}"
}

[bogeylnj]

I include my policy in my swagger api definition (as x-amazon-apigateway-policy). I have a similar issue - I don't set the terraform rest_api.policy argument, so terraform tries to update my policy => "" on next tf apply

In this case, perhaps the rest_api.policy argument can be ignored when the body argument is supplied according to the defined pattern here (i.e., it seems the intention was not for the two definition methods to work together):
https://github.com/terraform-providers/terraform-provider-aws/pull/1197

[sfdc-afraley]

Anyone found a workaround for this yet? For a workaround, I'd be fine with some method that creates aws_api_gateway_rest_api and then requires another terraform run to correct the policy. I haven't been able to come up with anything yet that doesn't create a circular dependency on aws_api_gateway_rest_api.myapi.root_resource_id

[SandyChapman]

@sfdc-afraley : There is a workaround.

You can set a lifecycle argument with "policy" in the ignore_changes list.

E.g.

resource "aws_api_gateway_rest_api" "rest_api" {
...
  policy = "${data.aws_iam_policy_document.api_gateway_resource_policy.json}"
  lifecycle {
    # Workaround this issue: 
    # https://github.com/terraform-providers/terraform-provider-aws/issues/5549
    ignore_changes = [
      "policy" 
    ]
  }
...
}

I believe you could then explicitly taint the aws_api_gateway_rest_api.rest_api to then have the policy updated.

[bevanbennett]

I just ran into the same cycle issue... but decided to just set the policy resources to "*" (it's attached only to this particular rest_api object anyway, so it can't effect anything else...)

[kcd83]

I have some workarounds to force the stage to be deployed and I thought I would try similar with the policy. I tested improvements to ignore_changes = ["policy"]

• Changing name or description will trigger an update BUT ignore_changes means the policy is _not_ updated so this is no use
• Tainting works BUT is effectively a destroy and recreate which redeploys all the resources that depends of the rest_api (which is everything!) and you get a new URL
• Commenting out the ignore_changes = ["policy"] (or changing "policy") does apply as expected

This looks like the same as issue #576 where for Elasticsearch a aws_elasticsearch_domain_policy was created. I.e. do we need a aws_api_gateway_rest_api_policy?

[bostrowski13]

any update on this?

[Constantin07]

I have a feeling that the resource reference is wrong in example given originally:

resources = [
       "execute-api:/*",
    ]

it should be:

resources = [
      "arn:aws:execute-api:*:*:*"
    ]

Judging from: https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-control-access-using-iam-policies-to-invoke-api.html

Any idea how to trigger trigger deployment in order for the resource policy to take effect ?

[sergei-ivanov]

@Constantin07 no, these two patterns are different. execute-api:/* refers to the _current_ API, whereas arn:aws:execute-api:*:*:* refers to _any_ API. Admittedly, it does not seem to be documented well.

[Constantin07]

@sergei-ivanov thanks for explanation, indeed terraform tries to change the resource ARN every time is applied which is annoying.

~ Statement = [
                  ~ {
                        Action    = "execute-api:Invoke"
                        Condition = {
                            IpAddress = {
                                aws:SourceIp = [
                                    "10.0.0.0/8",
                                ]
                            }
                        }
                        Effect    = "Allow"
                        Principal = "*"
                      ~ Resource  = "arn:aws:execute-api:eu-west-1:************:8i8p1y79gi/*/*/*" -> "execute-api:/*/*/*"
                        Sid       = "IPRangeWhitelist"
                    },

[sergei-ivanov]

...for which I already have a linked PR (#10986), which needs some attention from the project team.

[Constantin07]

nice work @sergei-ivanov, much appreciated!

[timothygarza-dazn]

I'm encountering an issue in v0.11.14 whereby the resource policy for API Gateway is not getting created at all during the first terraform apply. If I re-run terraform apply immediately afterwards the policy gets created and attached but now the API itself doesn't get updated and requires a manual deploy through the AWS console in order for the policy to take effect.
Is this related and has anyone else encountered this?

[timothygarza-dazn]

A Terraform equivalent of this command in AWS CLI is what we're looking for. That would allow to define the Resource Policy outside of the API and then 'attach' it and 'deploy' the API afterwards (all within Terraform)

aws apigateway update-rest-api \
--rest-api-id api-id \
--patch-operations op=replace,path=/policy,value='{"jsonEscapedPolicyDocument"}'

[nkoterba]

I just ran into the same cycle issue... but decided to just set the policy resources to "*" (it's attached only to this particular rest_api object anyway, so it can't effect anything else...)

Is there any security risk or concern using resources: "*"?

I implemented this approach as I do not want to ignore policy updates or have to remember to always taint something for changes to be updated/applied.

[AnkurJain10]

I have run into the same cycle issue. Is there no other solution than to use resources: "*"?

[bflad]

Hi folks 👋

Similar to other services that support resource policies, we have opted to provide support for a new aws_api_gateway_rest_api_policy resource, which will allow interpolating the resource ARN directly in the policy. Using the new resource will be the preferred method for configuring these policies, although we have no plans to deprecate the existing policy argument at this time. Existing policies can be imported via the REST API identifier and Terraform will correctly ignore differences when the existing policy argument of aws_api_gateway_rest_api resource is then not configured.

This functionality will release in version 3.16.0 of the Terraform AWS Provider later this week. Thank you to @sergei-ivanov and @DrFaust92 for their implementation efforts. 👍

[hashibot[bot]]

This has been released in version 3.16.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!