Terraform-provider-aws: Terraform unable to create ecs_service if iam_role is set

[hashibot]

_This issue was originally opened by @egoldschmidt as hashicorp/terraform#5292. It was migrated here as part of the provider split. The original body of the issue is below._

---

I've come across an issue in which an ECS service apparently cannot be created if iam_role is set. As you can see from this example I've set it to the ARN of the default ECS service user, but I've tried a custom user to no avil. The only thing that appears to resolve the problem is removing the iam_role field altogether. Obviously this is a non-starter because we need ELBs, so any ideas would be appreciated.

module.vpn_service.aws_ecs_service.vpn: Creating...
  cluster:                                 "" => "arn:aws:ecs:us-west-2:xxx:cluster/vpn"
  desired_count:                           "" => "1"
  iam_role:                                "" => "arn:aws:iam::xxx:role/ecsServiceRole"
  load_balancer.#:                         "" => "2"
  load_balancer.3901468935.container_name: "" => "vpn"
  load_balancer.3901468935.container_port: "" => "943"
  load_balancer.3901468935.elb_name:       "" => "vpn-elb"
  load_balancer.450980314.container_name:  "" => "vpn"
  load_balancer.450980314.container_port:  "" => "443"
  load_balancer.450980314.elb_name:        "" => "vpn-elb"
  name:                                    "" => "vpn"
  task_definition:                         "" => "arn:aws:ecs:us-west-2:xxx:task-definition/vpn:4"
Error applying plan:

1 error(s) occurred:

* aws_ecs_service.vpn: timeout while waiting for state to become '[success]'

[chrisLeeTW]

I also have similar problem. here is my error message.

terraform version : 0.11.2

Error: Error applying plan:

1 error(s) occurred:

• aws_ecs_service.test_service_001: 1 error(s) occurred:

• aws_ecs_service.test_service_001: InvalidParameterException: You cannot specify an IAM role for services that require a service linked role.
  status code: 400, request id: 666f86b5-fb80-11e7-bd3a-cfd17371dfbc "test-service-001"

[jackric]

This is affecting me also.
Looking at the code, service creation should fail sooner:
https://github.com/terraform-providers/terraform-provider-aws/blob/master/aws/resource_aws_ecs_service.go#L294
Here we can see the code retrying when it recieves InvalidParameterException - however this kind of problem won't solve itself by waiting; the loop should exit (so I don't have to wait 2 minutes to see the error).

[bflad]

You are correct. Generally, we try to be very specific on the error _message_ we're retrying on to ensure its IAM eventual consistency issues for new services instead of just blanket retrying on an error code like InvalidParameterException.

We'll gladly accept a PR to improve the retry logic there depending on the error message given when an IAM role might still be creating. Looking in the debug logs for our acceptance test TestAccAWSEcsService_withIamRole I see at least one of the specific messages we would want to retry on:

2018/01/31 05:37:55 [DEBUG] [aws-sdk-go] {"__type":"InvalidParameterException","message":"Unable to assume role and validate the listeners configured on your load balancer. Please verify that the ECS service role being passed has the proper permissions."}

The code can presumably be adjusted there to something like isAWSErr(err, "InvalidParameterException", "Please verify that the ECS service role being passed has the proper permissions."). There might be error messages we'll need to add too (to keep the same behavior as today), but at least a PR will bring those up when it gets reviewed.

[bflad]

I'll be fixing this issue soon as part of merging another PR.

[bflad]

The fix for this is in master and will be released in v1.9.0 of the AWS provider, which we are hoping to release today.

[dpiechota]

I still have the problem Terraform v0.11.7
plugin.terraform-provider-aws_v1.20.0_x4
aws_ecs_service.prometheus-server: InvalidParameterException: You cannot specify an IAM role for services that require a service linked role.
resource "aws_ecs_service" "prometheus-server" { ... iam_role = "arn:aws:iam::${data.aws_caller_identity.current.account_id}:role/aws-service-role/ecs.amazonaws.com/AWSServiceRoleForECS"
Role ARN arn:aws:iam::XXX:role/aws-service-role/ecs.amazonaws.com/AWSServiceRoleForECS

[bflad]

Hi @mozowski 👋 Sorry you're having trouble. Does it work if you completely remove iam_role? Otherwise, can you please open a new bug report with all the relevant details? Thanks!

We have quite a few acceptance tests that use iam_role with custom roles just fine, but none that explicitly try to use a service role. It might not be a valid configuration for the service and might be worth mentioning in the resource documentation if that is actually the case.

[dpiechota]

Hi @bflad I have reported this at https://github.com/terraform-providers/terraform-provider-aws/issues/4657
Thank you!

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!