Terraform: aws_ecs_service and aws_alb_listener precedenct

[h4rdL1nk]

Hi there,

Thank you for opening an issue. Please note that we try to keep the Terraform issue tracker reserved for bug reports and feature requests. For general usage questions, please see: https://www.terraform.io/community.html.

If your issue relates to a specific Terraform provider, please open it in the provider's own repository. The index of providers is at https://github.com/terraform-providers .

Terraform Version

Run terraform -v to show the version. If you are not running the latest version of Terraform, please try upgrading because your issue may have already been fixed.

Terraform Configuration Files

# Copy-paste your Terraform configurations here.
#
# For large Terraform configs, please use a service like Dropbox and
# share a link to the ZIP file. For security, you can also encrypt the
# files using our GPG public key.

Debug Output

Full debug output can be obtained by running Terraform with the environment variable TF_LOG=trace. Please create a GitHub Gist containing the debug output. Please do _not_ paste the debug output in the issue, since debug output is long.

Debug output may contain sensitive information. Please review it before posting publicly, and if you are concerned feel free to encrypt the files using the HashiCorp security public key.

Crash Output

If the console output indicates that Terraform crashed, please share a link to a GitHub Gist containing the output of the crash.log file.

Expected Behavior

What should have happened?

Actual Behavior

What actually happened?

Steps to Reproduce

Please list the full steps required to reproduce the issue, for example:

1. terraform init
2. terraform apply

Important Factoids

Are there anything atypical about your situation that we should know? For example: is Terraform running in a wrapper script or in a CI system? Are you passing any unusual command line options or environment variables to opt-in to non-default behavior?

References

Are there any other GitHub issues (open or closed) or Pull Requests that should be linked here? For example:

• #6017

[h4rdL1nk]

Hi,

Terraform version

Terraform v0.10.0

Crash Output

Error applying plan:

1 error(s) occurred:

• module.ecs-services.aws_ecs_service.default: 1 error(s) occurred:

• aws_ecs_service.default: InvalidParameterException: The target group with targetGroupArn arn:aws:elasticloadbalancing:eu-west-1:176423637738:targetgroup/TG-test-pre/d6ee8854549ffbc3 does not have an associated load balancer.
  status code: 400, request id: ece4ac87-ab67-11e7-a327-3514dd6e526d "SVC-test"

Terraform does not automatically rollback in the face of errors.
Instead, your Terraform state file has been partially updated with
any resources that successfully completed. Please address the error
above and apply again to incrementally change your infrastructure.

Actual Behavior

This is due to the 'aws_ecs_service' and 'aws_alb_listener' resouces provisioning precedence, I think. The new service and the load-balancer are waiting for creation at the same time, and the ecs service is waiting for the target-group configured for being assigned to the listener, but it never ends, as the corresponding load-balancer is still waiting for provision, as we can see in this 'terraform apply' output:

module.ecs-services.aws_ecs_service.default: Still creating... (1m40s elapsed)
module.cluster-instances.aws_alb.main: Still creating... (1m40s elapsed)
module.ecs-services.aws_ecs_service.default: Still creating... (1m50s elapsed)
module.cluster-instances.aws_alb.main: Still creating... (1m50s elapsed)
module.cluster-instances.aws_alb.main: Still creating... (2m0s elapsed)
module.cluster-instances.aws_alb.main: Still creating... (2m10s elapsed)
module.cluster-instances.aws_alb.main: Creation complete (ID: arn:aws:elasticloadbalancing:eu-west-1:...dbalancer/app/ALB-pre/4a5afef57932873f)
module.ecs-services.aws_alb_listener.detault: Creating...

I don't know if this problem as some workarround, but executing the 'apply' command again finish OK, as the load-balancer listener is assigned to the corresponding target group.

[bitbrain]

@h4rdL1nk Could you please provide your Terraform syntax which creates the resources (alb, alb listener, ecs service etc.)? If you specify a an ALB listener but do not attach it to an ALB it will lead to this error.

[tombuildsstuff]

@h4rdL1nk so that we can investigate this further - would it be possible to see the Terraform Configuration you're using here?

Thanks!

[rafaljanicki]

Hey @bitbrain ,

I have the same issue, so let me provide you with more details:

resource "aws_alb" "application_load_balancer" {
  subnets = ["${var.public_subnets}"]
  security_groups = ["${aws_security_group.public_security_group.id}"]

  tags {
    Environment = "${terraform.workspace}"
    Terraform = true
  }
}

resource "aws_alb_target_group" "default_target_group" {
  vpc_id = "${var.vpc_id}"
  port = 80
  protocol = "HTTP"

  health_check {
    path = "${var.ecs_healthcheck_path}"
  }

  tags {
    Environment = "${terraform.workspace}"
    Terraform = true
  }
}

resource "aws_alb_listener" "default_alb_listener" {
  "default_action" {
    target_group_arn = "${aws_alb_target_group.default_target_group.arn}"
    type = "forward"
  }

  load_balancer_arn = "${aws_alb.application_load_balancer.arn}"
  port = 443
  protocol = "HTTPS"
  certificate_arn = "${var.certificate_arn}"
}

resource "aws_ecs_service" "health_check_service" {
  name = "healthcheck"
  cluster = "${aws_ecs_cluster.cluster.id}"
  desired_count = 1
  task_definition = "${aws_ecs_task_definition.health_check_task_definition.arn}"
  iam_role = "${aws_iam_role.health_check_service_role.arn}"

  load_balancer {
    container_name = "healthcheck"
    container_port = 80
    target_group_arn = "${module.load_balancer_with_asg.default_target_group_arn}"
  }
}

The Service and ALBs are defined in separate modules hence the differences in references, but you should get the overall view on the configuration.
Let me know if you need any more information

[bitbrain]

@rafaljanicki a few things:

• In order to avoid timing issues please add something like depends_on = [ "module. load_balancer_with_asg" ] to your aws_ecs_service resource (untested)
• Your aws_alb_listener is missing the ssl_policy attribute which is required since you're using HTTPS
• Do not use "default_action" but default_action
• Giving your service a static name which is not based on interpolation is generally a bad idea, since you can not have two services with the same name, once you try to scale-out. Use name_prefix where possible.

Apart from that I don't see anything suspicious in the configuration. Maybe I'll notice something later.

[rafaljanicki]

Thanks @bitbrain ,

Ad. 1) That may fix the issue, I'll test that tomorrow and let you know if that works
Ad. 2) That's weird as it works correctly (fetches the default policy). I believe that the documentation is invalid here
Ad. 3 & 4) Thanks!

[rafaljanicki]

Hey @bitbrain ,

The depends_on change fixed the issue, thanks for the help!

[bitbrain]

@h4rdL1nk Does this fix it for you as well?

[h4rdL1nk]

The depends_on avoid this problem. I applied it at the first time and solved the issue, but I oppened this issue because I thought it was an internal resource dependency bug or anything.

Thanks for your help!

[h4rdL1nk]

The issue was solved by using the depends_on to mark explicit resources deoendencies

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.