Terraform-provider-aws: Apply does not create AWS API Gateway correctly

[hashibot[bot]]

_This issue was originally opened by @keinproblem as hashicorp/terraform#18554. It was migrated here as a result of the provider split. The original body of the issue is below._

---

When constructing an AWS API Gateway using _terraform apply_ the state described in code will not be reached. Meaning that some information will not be applied, which leads to a difference between described state and current state when executing another subsequent _terraform apply_.
The API Gateway resource uses a OAS 2.0 file's content in the body parameter which is parsed using the _template_file_ data structure in order to inject dynamically generated information.
The same procedure applies for the Resource Policy - the dynamically generated VPC Endpoint'd ID will be injected there.

When executing a _terraform apply_ command the data for the resources to be created is presented corretly.
But when executing a _terraform apply_ command immediately again the _description_ and the _policy_ fields of the API Gateway were not applied correctly, since they are empty.

1st run:

description: "" => "This static descriptions will not be applied on initial trraform apply"
endpoint_configuration.#: "0" => "1"
endpoint_configuration.0.types.#: "0" => "1"
endpoint_configuration.0.types.0: "" => "PRIVATE"
execution_arn: "" => ""
minimum_compression_size: "" => "-1"
name: "" => "api-gateway-default"
policy: "" => "{\r\n \"Version\": \"2012-10-17\",\r\n \"Statement\": [\r\n {\r\n \"Effect\": \"Allow\",\r\n \"Principal\": \"\",\r\n \"Action\": \"execute-api:Invoke\",\r\n \"Resource\": \"\",\r\n \"Condition\": {\r\n \"StringEquals\": {\r\n \"aws:sourceVpce\": \"vpce-045978512aa67288f\"\r\n }\r\n }\r\n }\r\n ]\r\n}"

2nd run:

description: "" => "This static descriptions will not be applied on initial trraform apply"
policy: "" => "{\r\n \"Version\": \"2012-10-17\",\r\n \"Statement\": [\r\n {\r\n \"Effect\": \"Allow\",\r\n \"Principal\": \"\",\r\n \"Action\": \"execute-api:Invoke\",\r\n \"Resource\": \"\",\r\n \"Condition\": {\r\n \"StringEquals\": {\r\n \"aws:sourceVpce\": \"vpce-045978512aa67288f\"\r\n }\r\n }\r\n }\r\n ]\r\n}"

After the 2nd run the changes are applied correctly.

Terraform Version

Terraform v0.11.7
+ provider.aws v1.29.0
+ provider.template v1.0.0

Terraform Configuration Files

https://drive.google.com/file/d/1CoYRHW-tgmOLKiSe3g_NjMxuISgUNncQ/view?usp=sharing

Debug Output

Full logs of both _terraform apply_ executions are provided in the previously linked archive.

Crash Output

No crash occurs.

Expected Behavior

After running the initial _terraform apply_ once, there should be no new changes detected when executing _terraform apply_ or _terraform plan_, hence everything should be applied in the initial run.

Actual Behavior

A subsequent _terraform apply_ still shows new changes immediately after the initial _terraform apply_.
After the second execution of terraform apply the desired state is reached.

Steps to Reproduce

terraform init

terraform apply

terraform apply or terraform plan

Additional Context

An already existing VPC, Subnet and Security Group is required in order to reproduce this issue. The respective IDs of the resources need to be specified in in the variable blocks. Simply change the default values.

When creating the API Gateway without a body parameter, all changes are applied in the first run, hence no diff is detected in the second run.

References

none

[bostrowski13]

I'm facing this same issue still. I'm on TF 0.12.8, using AWS provider 2.29.0.

The first apply fails because it doesn't look like its creating the resource policy on the first run that is associated with the REST API, and the api deployment fails because of the lack of this policy.

I can confirm that policy is actually being created/rendered during the first apply, but its not being attached to the rest api via the "policy" arguement, or at least aws isn't taking it.

[skimonkey]

I am using terraform v0.12.15 and provider.aws v2.39.0

Applying the following code, results in the rest api being created and deployed from my swagger.yml but the policy is missing. Running Apply again for a second time and the policy is created but it is not deployed. So I have to apply the terraform twice for the api and policy to get created and deployed.
This seems consistent with the op

resource "aws_api_gateway_rest_api" "MyApi" {
  name = "API Terraform Swagger Test"
  endpoint_configuration {
    types = ["REGIONAL"]
  } 
  body = file("swagger.yml")
  policy = data.aws_iam_policy_document.api_endpoint_access.json
}

resource "aws_api_gateway_deployment" "MyApi" {
  rest_api_id = aws_api_gateway_rest_api.MyApi.id
  stage_name  = "int"
  variables = {
     deployed_at = timestamp()
  }    
}

Is there a fix/workaround for this..?

[keinproblem]

@bostrowski13 and @skimonkey Both of your cases seem to have the same effect as in my initial finding.
Surprisingly this is still possible in the terraform and provider.aws versions mentioned in your cases, expecting that this was not addressed yet.
It looks like there needs to be a way to forcefully set the policy, before performing the deployment.

It is still unclear what the reason for the empty description and policy field value is. Applying the API-Gateway resource entirely seems to happen in multiple sets (probably related to some AWS API mechanism or Terraform internal implementation). I assume it fails silently in the first step due to whatever reason.

[jamesrwarren]

The issue here is not that the policy is left blank on the first run. What seems to happen is that the policy is created by the CreateRestApi function and then the body is subsequently added with PutRestApi which requires an existing rest api to exist to apply the body against.

The same issue happens when we create an API without the body being set and then apply aws apigateway put-rest-api ... from the aws cli. We initially see the policy attached and then apon running the put-rest-api command the policy gets wiped.

As the issue doesn't lie with terraform I can't see a good way of fixing this. I have added a workaround for that resource that just recreates the policy afterwards as part of the initial body create. I've tried it locally and it works but it's not very elegant.. Maybe it will be of some use until this issue is fixed. I've created it as a [WIP] so maybe others could comment on the approach.

https://github.com/terraform-providers/terraform-provider-aws/pull/12432

[MartiUK]

An update in place to a private apigw's body will cause any deployments to fail with: Error creating API Gateway Deployment: BadRequestException: Private REST API doesn't have a resource policy attached to it due to this issue, it's annoying having to apply our changes twice.