Terraform-provider-aws: aws_ecs_service fails to create CODE_DEPLOY deployment with alb

[sam-burrell]

Terraform Version

Terraform v0.11.7

Affected Resource(s)

aws_ecs_service
aws_codedeploy_deployment_group

Terraform Configuration Files

resource "aws_ecs_service" "service" {
  count                              = "${var.autoscaling ? 0 : 1}"
  name                               = "${local.name}"
  cluster                            = "${var.cluster}"
  task_definition                    = "${aws_ecs_task_definition.dummy.arn}"
  deployment_maximum_percent         = "${var.deployment_maximum}"
  deployment_minimum_healthy_percent = "${var.deployment_minimum}"

  load_balancer = {
    elb_name = "prod-edge-elb"
    container_name   = "application"
    container_port   = "${var.port}"
  }

  desired_count = "${var.desired_count}"
  launch_type   = "EC2"

  ordered_placement_strategy {
    type  = "spread"
    field = "attribute:ecs.availability-zone"
  }

  ordered_placement_strategy {
    type  = "spread"
    field = "instanceId"
  }

  deployment_controller {
    type = "CODE_DEPLOY"
  }

  placement_constraints {
    type       = "memberOf"
    expression = "${data.template_file.placement_constraints.rendered}"
  }

  lifecycle {
    ignore_changes = ["task_definition", "launch_type"]
  }
}

Debug Output

* aws_codedeploy_deployment_group.service: InvalidInputException: Could not load ECS service information for cluster: prod-signal, service: prod-test. Cause: Unable to load ECS service info for [cluster: prod-signal, service: prod-test]. arn:aws:ecs:eu-west-1:655100936961:service/prod-test failed with MISSING.
        status code: 400, request id: 7fb4e9ce-f7c9-11e8-b0a9-6759947bb180
* module.test.module.service.aws_ecs_service.service: 1 error(s) occurred:

* aws_ecs_service.service: InvalidParameterException: The load balancer prod-edge-elb does not exist.
        status code: 400, request id: 7fb733a7-f7c9-11e8-9652-0da581d1e724 "prod-test"

Expected Behavior

Use the specified alb (prod-edge-alb) (ALB not ELB) and either generate or use the 2 specified target groups for blue/green production and testing in the aws_codedeploy_deployment_group defined separately.

The prod-edge-able does exist. I feel there is something about elb_name not being an alb.

Actual Behavior

Errors and fails to create service.

[bflad]

Hi @sam-burrell 👋 Sorry you are running into trouble here.

There are two arguments available under load_balancer with the aws_ecs_service resource which control whether its an ELB or ALB/NLB, a distinction required by the ECS API:

• elb_name - (Required for ELB Classic) The name of the ELB (Classic) to associate with the service.
• target_group_arn - (Required for ALB/NLB) The ARN of the Load Balancer target group to associate with the service.

Does switching to target_group_arn (e.g. a reference like aws_lb_target_group.example.arn) help?

[sam-burrell]

I can give a taget_group_arn (this is normally how we specifiy ecs services)

However now through the console when making a CODE_DEPLOY ecs_service it is possible to create two target groups 'on the fly' while making an ecs_service after specifying an alb.

[bflad]

From the CodeDeploy side, we provide an example of how to wire up ECS Blue/Green deployment: https://www.terraform.io/docs/providers/aws/r/codedeploy_deployment_group.html#blue-green-deployments-with-ecs

And what's interesting is that the ECS API doesn't complain if you provide only a single target group in the ECS service with CodeDeploy, as acceptance tested here:

https://github.com/terraform-providers/terraform-provider-aws/blob/1709dcd0fee48db04eab928a26eb3feaa3663379/aws/resource_aws_codedeploy_deployment_group_test.go#L3342-L3577

However the ECS API documentation for the LoadBalancer type does seem to suggest that two load balancer configurations could (or maybe even _should_) be provided when the deployment controller is CODE_DEPLOY.

targetGroupArn
The full Amazon Resource Name (ARN) of the Elastic Load Balancing target group or groups associated with a service. For services using the ECS deployment controller, you are limited to one target group. For services using the CODE_DEPLOY deployment controller, you are required to define two target groups for the load balancer.

We do currently limit the Terraform configuration to a single load_balancer:

https://github.com/terraform-providers/terraform-provider-aws/blob/1709dcd0fee48db04eab928a26eb3feaa3663379/aws/resource_aws_ecs_service.go#L147-L151

It would be curious to find out the behavior of actually trying to deploy via CodeDeploy if the ECS service only has a single load_balancer defined (but CodeDeploy has its required target group pair). I'm guessing though that we'll need to lift the MaxItems: 1 restriction with load_balancer

[sam-burrell]

Thanks very much for your focused response on this.

Yeah the code_deploy part works well. However it does currently fail for us as its currently depending on this aws_ecs_service being created.

We are currently making the 2 target groups in terraform.
The console seems to make these on the fly... Not sure what the intention from AWS is for this exactly. I can try and contact them to see what their desired functionality is intended?

resource "aws_codedeploy_deployment_group" "service" {
  count                  = "${var.enable_codedeploy ? 1 : 0}"
  app_name               = "${var.name}"
  deployment_config_name = "CodeDeployDefault.ECSAllAtOnce"
  deployment_group_name  = "${var.name}-group"
  service_role_arn       = "${aws_iam_role.codedeploy.arn}"

  auto_rollback_configuration {
    enabled = true
    events  = ["DEPLOYMENT_FAILURE"]
  }

  blue_green_deployment_config {
    deployment_ready_option {
      action_on_timeout = "CONTINUE_DEPLOYMENT"
    }

    terminate_blue_instances_on_deployment_success {
      action                           = "TERMINATE"
      termination_wait_time_in_minutes = 5
    }
  }

  deployment_style {
    deployment_option = "WITH_TRAFFIC_CONTROL"
    deployment_type   = "BLUE_GREEN"
  }

  ecs_service {
    cluster_name = "${local.cluster}"
    service_name = "prod-${var.name}"
  }

  load_balancer_info {
    target_group_pair_info {
      prod_traffic_route {
        listener_arns = ["${var.alb_outputs["listener_arn"]}"]
      }

      target_group {
        name = "${module.alb-target-group-blue.target_group_name}"
      }

      target_group {
        name = "${module.alb-target-group-green.target_group_name}"
      }
    }
  }
}

[lielran]

This is classic use case. Ecs(fargate or not) with code deploy and ALB.

I think that update the ecs service(e.g. update thr desired count) failed because of this configuration. Is the example above (from the blue green docs) works for anyone?

[chenrui333]

We are running into very much the same issue, but the ece_service was created with CF, right now we are trying to add the blue green deployment_group resource on top of it.

$ tf version
Terraform v0.11.11
+ provider.aws v1.57.0

[antonbabenko]

It looks that the solution proposed by @bflad (https://github.com/terraform-providers/terraform-provider-aws/issues/6703#issuecomment-444220591) regarding the change from TypeSet to TypeList will help. It is already implemented in this PR - https://github.com/terraform-providers/terraform-provider-aws/pull/7316 if someone wants to try it out.

[krisread]

FWIW I am another user affected by this issue - seems like ECS service should support >1 load balancer target groups.

[bflad]

Hi folks 👋 Back in August of last year (version 2.22.0 of the Terraform AWS Provider), support for multiple load_balancer configuration blocks in the aws_ecs_service resource was released. If there are are still lingering features or issues with the setup, please create a new GitHub issue. Thanks.

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!