Terraform-provider-aws: Blue Green Deployments with ECS

[lielran]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

terraform -v
Terraform v0.11.8

Your version of Terraform is out of date! The latest version
is 0.11.10. You can update by downloading from www.terraform.io/downloads.html

Affected Resource(s)

• aws_1.51.0

Terraform Configuration Files

https://www.terraform.io/docs/providers/aws/r/codedeploy_deployment_group.html#blue-green-deployments-with-ecs

Debug Output

https://gist.github.com/lielran/9b9926fe644a89c3e51cab75e5458820

Panic Output

Expected Behavior

Create a deployment group to connected to FARGATE service with blue_green deployment option

Actual Behavior

failed to run the example as is.
first error:

* aws_codedeploy_deployment_group.deployment_gruop: InvalidECSServiceException: Should not specify ECS services parameter for Server deployment group
    status code: 400, request id: 71958563-fd77-11e8-9dc7-1fb37a7f1f52

if I remove the ecs_service inside aws_codedeploy_deployment_group I'll get this error:

* aws_codedeploy_deployment_group.deployment_gruop: InvalidLoadBalancerInfoException: The specification for load balancing in the deployment group is invalid. The deploymentOption value is set to WITH_TRAFFIC_CONTROL, but either no load balancer was specified in elbInfoList or no target group was specified in targetGroupInfoList.
    status code: 400, request id: 3ea70a03-fd77-11e8-8f15-ed70b7bae79

Steps to Reproduce

copy the example from here: 1.https://www.terraform.io/docs/providers/aws/r/codedeploy_deployment_group.html#blue-green-deployments-with-ecs

1. terraform apply

Important Factoids

using ECS FARGATE service with ALB

References

https://www.terraform.io/docs/providers/aws/r/codedeploy_deployment_group.html#blue-green-deployments-with-ecs

[bflad]

Hi @lielran 👋 Sorry for the trouble here. Is your aws_codedeploy_app resource that the aws_codedeploy_deployment_group resource is dependent on similar to the following?

resource "aws_codedeploy_app" "example" {
  compute_platform = "ECS"
  name             = "example"
}

If not, apologies for the omission of that important information with the aws_codedeploy_deployment_group documentation example there and I can get that fixed up.

[lielran]

Thanks @bflad !
this is the missing part 🎉

compute_platform = "ECS"

[lielran]

Also, I'm getting this error while trying to deploy(from ecs update service).
error in CodeDeploy:
A deployment group associated with an application that uses the ECS compute platform must contain a valid pair of target groups that identify a load balancer. The target ECS service must be configured using one of those two target groups.
where in the aws_codedeploy_deployment_group I do have target_group_pair_info with two target groups

[bflad]

The documentation update for the aws_codedeploy_deployment_group resource has been merged and will publish with version 1.52.0 of the AWS provider

For the second issue you mention, it may be related to #6703 where we limit aws_ecs_service to only a single load_balancer configuration at the moment. Let's track that separately in #6703 or if that doesn't make sense, create a new issue. 👍

[jfrolich]

When I try to update the task definition for a service, that should trigger a deploy. However I get this:

InvalidParameterException: Unable to update task definition on services with a CODE_DEPLOY deployment controller.

How to deploy if we can't update the task definition?

[miere]

Hi @jfrolich, I just faced the same problem today.

In my case I had to configure my ECS Service to have its deployment life cycle controlled by _AWS Code Deploy_.

resource "aws_ecs_service" "default" {
  name            = "my-simple-app"
  launch_type     = "FARGATE"

  // ... other configuration entries

  deployment_controller {
    type = "CODE_DEPLOY"
  }
}

[jfrolich]

Yes I have set the deployment_controller do you mean that you ignore changes using terraform lifecycles?

  lifecycle {
    ignore_changes = [
      "load_balancer",
      "task_definition",
    ]
  }

(This is what I am trying to do now)

Is there a way to let terraform change the service and push a deploy?

[jfrolich]

I even can't change service parameters:

* aws_ecs_service.cogo: 1 error(s) occurred:

* aws_ecs_service.cogo: error updating ECS Service (arn:aws:ecs:us-east-1:744536757714:service/cogo_prod): InvalidParameterException: Unable to update network parameters on services with a CODE_DEPLOY deployment controller. Please use Code Deploy to trigger a new deployment.
    status code: 400, request id: ea3a7721-2395-11e9-b8ce-c3c7fff85a80

[jfrolich]

Anyone has an idea how to resolve this?

[tsukakei]

Hi @jfrolich , I met the same problem as you.

This problem might be caused by limitations of AWS API.

For services using the blue/green (CODE_DEPLOY) deployment controller, only the desired count, deployment configuration, and health check grace period can be updated using this API. If the network configuration, platform version, or task definition need to be updated, a new AWS CodeDeploy deployment should be created. For more information, see CreateDeployment in the AWS CodeDeploy API Reference.
https://docs.aws.amazon.com/ja_jp/AmazonECS/latest/APIReference/API_UpdateService.html

[jfrolich]

Ok, but there isn't a terraform API to create a new deployment (if needed) right?

[therakesh]

@jfrolich I am also facing same issue. Workaround would be to taint the aws_ecs_service resource to recreate it with updated values.

[jfrolich]

Do you have this in code, or you taint the service manually?

[therakesh]

I did it manually

[alekbarszczewski]

I don't get it.
In my task definition I have some env vars.
At some point I want to add new env var to task definition.
However I get this error:

InvalidParameterException: Unable to update network parameters on services with a CODE_DEPLOY deployment controller. 
Please use Code Deploy to trigger a new deployment

So what else I have to change in my terraform config to make it work?
Expected behaviour is that after changing env var in task definition CodeDeploy will normally perform blue/green deployment and new tasks will have this env var set.

If this is impossible then CodeDeploy/Blue-Green is completely useless.

[mvoropaiev]

Not sure if the best approach, haven't tested it properly, but in order to trigger a deployment when updating task definition via terraform I have added the following

• aws_alb_listener_rule

  lifecycle {
  ignore_changes = [
    "action.0.target_group_arn",
  ]
  }
  

• ecs_service

  lifecycle {
  ignore_changes = [
    "task_definition",
    "load_balancer",
  ]
  }
  

• add null_resource

  resource "null_resource" "redeploy" {
  triggers {
    task_definition_arn = "${aws_ecs_task_definition.example.arn}"
  }
  
  provisioner "local-exec" {
    command = "./redeploy.sh app_name deploy_group_name nginx 8080 ${aws_ecs_task_definition.example.arn}"
  }
  }
  

• codedeploy script (redeploy.sh):

  #!/usr/bin/env bash
  set -e
  
  application_name=$1
  deployment_group_name=$2
  container_name=$3
  container_port=$4
  task_definition_arn=$5
  
  app_spec_content_string=$(jq -nc \
  --arg container_name "$container_name" \
  --arg container_port "$container_port" \
  --arg task_definition_arn "$task_definition_arn" \
  '{version: 1, Resources: [{TargetService: {Type: "AWS::ECS::Service", Properties: {TaskDefinition: $task_definition_arn, LoadBalancerInfo: {ContainerName: $container_name, ContainerPort: $container_port}}}}]}')
  app_spec_content_sha256=$(echo -n "$app_spec_content_string" | shasum -a 256 | sed 's/ .*$//')
  revision="revisionType=AppSpecContent,appSpecContent={content='$app_spec_content_string',sha256=$app_spec_content_sha256}"
  
  aws --profile example deploy create-deployment \
  --application-name="$application_name" \
  --deployment-group-name="$deployment_group_name" \
  --revision="$revision"
  

[jamie3]

Why is this closed? There is a work around but the root problem isnt solved.

[finferflu]

Yeah, I agree, what we have now works, but it’s a hack.

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!