Terraform-provider-aws: ECS service resource does not recreate when change in security groups

[hashibot[bot]]

_This issue was originally opened by @rakesh-landge as hashicorp/terraform#20746. It was migrated here as a result of the provider split. The original body of the issue is below._

---

Terraform Version
0.11.*

Affected Resource(s)
aws_ecs_service

Issue:
When updating security groups associated with ecs service configured for green/blue deployment, plan shows in place update of resource. When we apply the changes, it fails with message to deploy with CODE_DEPLOY deployment controller. But when we redeploy the service using CODE_DEPLOY(blue/green CodeDeploy associated with service) it does not update security groups as there is no provision to specify same in it.

Error message:
aws_ecs_service.web: error updating ECS Service (arn:aws:ecs:us-east-1:XXXX:service/sample_ecs_service): InvalidParameterException: Unable to update network parameters on services with a CODE_DEPLOY deployment controller. Please use Code Deploy to trigger a new deployment.

Expected Behavior:
Security groups should be updated for ECS service

[nywilken]

Hi @rakesh-landge sorry that you are running into trouble here. In order for us to best help with this issue can you provide us with a redacted configuration that can be used for reproduction. Thanks!

[therakesh]

Hi @nywilken thanks for looking into this. Follows the requested details.

Configuration:

resource "aws_ecs_cluster" "example" {
  name = "example"
}

resource "aws_ecs_task_definition" "example" {
  family                = "service"
  container_definitions    = "${data.template_file.web_task.rendered}"
  requires_compatibilities = ["FARGATE"]
  network_mode             = "awsvpc"
  execution_role_arn       = "${data.aws_iam_role.task_role.arn}"
  task_role_arn            = "${data.aws_iam_role.task_role.arn}"
  memory = "512"
  cpu = "256"
}

resource "aws_ecs_service" "example" {
  name            = "example"
  cluster         = "${aws_ecs_cluster.example.id}"
  task_definition = "${aws_ecs_task_definition.example.arn}"
  desired_count   = 2
  launch_type     = "FARGATE"

  load_balancer {
    target_group_arn = "${aws_lb_target_group.foo.arn}"
    container_name   = "example"
    container_port   = 80
  }

  network_configuration {
    security_groups = ["${aws_security_group.connect_repo.id}", "${aws_security_group.intra_resorce_all.id}"]
    subnets         = ["${data.aws_subnet.private.*.id}"]
  }

  deployment_controller {
    type = "CODE_DEPLOY"
  }

  lifecycle {
    ignore_changes = ["desired_count", "task_definition", "load_balancer"]
  }
}

resource "aws_codedeploy_app" "example" {
  compute_platform = "ECS"
  name             = "example"
}

resource "aws_codedeploy_deployment_group" "example" {
  app_name               = "${aws_codedeploy_app.example.name}"
  deployment_config_name = "CodeDeployDefault.ECSAllAtOnce"
  deployment_group_name  = "example"
  service_role_arn       = "${data.aws_iam_role.code_deploy_role.arn}"

  auto_rollback_configuration {
    enabled = true
    events  = ["DEPLOYMENT_FAILURE"]
  }

  blue_green_deployment_config {
    deployment_ready_option {
      action_on_timeout = "CONTINUE_DEPLOYMENT"
    }

    terminate_blue_instances_on_deployment_success {
      action                           = "TERMINATE"
      termination_wait_time_in_minutes = 5
    }
  }

  deployment_style {
    deployment_option = "WITH_TRAFFIC_CONTROL"
    deployment_type   = "BLUE_GREEN"
  }

  ecs_service {
    cluster_name = "${aws_ecs_cluster.example.name}"
    service_name = "${aws_ecs_service.example.name}"
  }

  load_balancer_info {
    target_group_pair_info {
      prod_traffic_route {
        listener_arns = ["${data.aws_alb_listener.blue_listener.arn}"]
      }

      target_group {
        name = "${data.aws_alb_target_group.blue_target_group.name}"
      }

      target_group {
        name = "${aws_alb_target_group.alb_green_target_group.name}"
      }
    }
  }
}

Objective:

Need to add another security group to ecs service

resource "aws_ecs_service" "example" {
  :
 :
 :
  network_configuration {
    security_groups = ["${aws_security_group.connect_repo.id}", "${aws_security_group.intra_resorce_all.id}", ${aws_security_group.another_sg.id}"]
    subnets         = ["${data.aws_subnet.private.*.id}"]
  }
 :
 :
}

Error:

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
~ update in-place

Terraform will perform the following actions:

~ aws_ecs_service.example
network_configuration.0.security_groups.#: "2" => "3"
network_configuration.0.security_groups.2115051421: "" => "sg-xxx"
network_configuration.0.security_groups.3249404650: "sg-yyy" => "sg-yyy"
network_configuration.0.security_groups.3888678103: "sg-zzz" => "sg-zzz"

Plan: 0 to add, 1 to change, 0 to destroy.

Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.

Enter a value: yes

aws_ecs_service.example: Modifying... (ID: arn:aws:ecs:us-east-1:aaaaa:service/example/example)
network_configuration.0.security_groups.#: "2" => "3"
network_configuration.0.security_groups.2115051421: "" => "sg-xxx"
network_configuration.0.security_groups.3249404650: "sg-yyy" => "sg-yyy"
network_configuration.0.security_groups.3888678103: "sg-zzz" => "sg-zzz"

Error: Error applying plan:

1 error(s) occurred:

• aws_ecs_service.example: 1 error(s) occurred:

• aws_ecs_service.example: error updating ECS Service (arn:aws:ecs:us-east-1:aaaaa:service/example/example): InvalidParameterException: Unable to update network parameters on services with a CODE_DEPLOY deployment controller. Please use Code Deploy to trigge
  r a new deployment.
  status code: 400, request id: wwwwwwww

Additional Info:

When we re-deploy the service using blue/green CodeDeploy associated with service, it does not update security groups as there is no provision to specify same in it.

[therakesh]

Any update on this issue?

[chenrui333]

I am receiving similar error. (tf 0.11.13, provider 2.17)

2020-01-17T17:30:49.490-0500 [DEBUG] plugin.terraform-provider-aws_v2.17.0_x4: 2020/01/17 17:30:49 [DEBUG] [aws-sdk-go] {"__type":"InvalidParameterException","message":"Unable to update network parameters on services with a CODE_DEPLOY deployment controller. Use AWS CodeDeploy to trigger a new deployment."}
2020-01-17T17:30:49.490-0500 [DEBUG] plugin.terraform-provider-aws_v2.17.0_x4: 2020/01/17 17:30:49 [DEBUG] [aws-sdk-go] DEBUG: Validate Response ecs/UpdateService failed, attempt 0/25, error InvalidParameterException: Unable to update network parameters on services with a CODE_DEPLOY deployment controller. Use AWS CodeDeploy to trigger a new deployment.
2020-01-17T17:30:49.490-0500 [DEBUG] plugin.terraform-provider-aws_v2.17.0_x4:  status code: 400, request id: d43aed14-e43a-475c-b13e-2b121f6f199d
2020/01/17 17:30:49 [ERROR] root: eval: *terraform.EvalApplyPost, err: 1 error(s) occurred:

* aws_ecs_service.mup_web_fargate_blue_green_service_prod: error updating ECS Service (arn:aws:ecs:us-east-1:541129006137:service/mwp-mup-web-prod/mup-web-codedeploy-blue-green): InvalidParameterException: Unable to update network parameters on services with a CODE_DEPLOY deployment controller. Use AWS CodeDeploy to trigger a new deployment.
    status code: 400, request id: d43aed14-e43a-475c-b13e-2b121f6f199d
2020/01/17 17:30:49 [ERROR] root: eval: *terraform.EvalSequence, err: 1 error(s) occurred:

* aws_ecs_service.mup_web_fargate_blue_green_service_prod: error updating ECS Service (arn:aws:ecs:us-east-1:541129006137:service/mwp-mup-web-prod/mup-web-codedeploy-blue-green): InvalidParameterException: Unable to update network parameters on services with a CODE_DEPLOY deployment controller. Use AWS CodeDeploy to trigger a new deployment.
    status code: 400, request id: d43aed14-e43a-475c-b13e-2b121f6f199d
2020/01/17 17:30:49 [TRACE] [walkApply] Exiting eval tree: aws_ecs_service.mup_web_fargate_blue_green_service_prod

[chenrui333]

error on latest version of plugin:

2020-01-17T17:39:12.291-0500 [DEBUG] plugin.terraform-provider-aws_v2.17.0_x4: 2020/01/17 17:39:12 [DEBUG] [aws-sdk-go] {"__type":"InvalidParameterException","message":"Unable to update network parameters on services with a CODE_DEPLOY deployment controller. Use AWS CodeDeploy to trigger a new deployment."}
2020-01-17T17:39:12.291-0500 [DEBUG] plugin.terraform-provider-aws_v2.17.0_x4: 2020/01/17 17:39:12 [DEBUG] [aws-sdk-go] DEBUG: Validate Response ecs/UpdateService failed, attempt 0/25, error InvalidParameterException: Unable to update network parameters on services with a CODE_DEPLOY deployment controller. Use AWS CodeDeploy to trigger a new deployment.
2020-01-17T17:39:12.291-0500 [DEBUG] plugin.terraform-provider-aws_v2.17.0_x4:  status code: 400, request id: d47dc35e-6040-4bb9-a290-d966d87596e3
2020/01/17 17:39:12 [ERROR] root: eval: *terraform.EvalApplyPost, err: 1 error(s) occurred:

* aws_ecs_service.mup_web_fargate_blue_green_service_prod: error updating ECS Service (arn:aws:ecs:us-east-1:541129006137:service/mwp-mup-web-prod/mup-web-codedeploy-blue-green): InvalidParameterException: Unable to update network parameters on services with a CODE_DEPLOY deployment controller. Use AWS CodeDeploy to trigger a new deployment.
    status code: 400, request id: d47dc35e-6040-4bb9-a290-d966d87596e3
2020/01/17 17:39:12 [ERROR] root: eval: *terraform.EvalSequence, err: 1 error(s) occurred:

* aws_ecs_service.mup_web_fargate_blue_green_service_prod: error updating ECS Service (arn:aws:ecs:us-east-1:541129006137:service/mwp-mup-web-prod/mup-web-codedeploy-blue-green): InvalidParameterException: Unable to update network parameters on services with a CODE_DEPLOY deployment controller. Use AWS CodeDeploy to trigger a new deployment.
    status code: 400, request id: d47dc35e-6040-4bb9-a290-d966d87596e3
2020/01/17 17:39:12 [TRACE] [walkApply] Exiting eval tree: aws_ecs_service.mup_web_fargate_blue_green_service_prod