Terraform-provider-aws: ASGs with Mixed Instance Policies

[kemra102]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

• 0.10.8

Affected Resource(s)

• aws_autoscaling_group
• aws_launch_template

Terraform Configuration Files

resource "aws_launch_template" "lt" {
  name_prefix = "danny_test-"

  vpc_security_group_ids = [
    "${module.sec_groups.security_group_id_common}",
    "${module.sec_groups.security_group_id_ssh}",
    "${module.sec_groups.security_group_id_web}",
  ]

  image_id = "${module.core_ami.core_ami_id}"

  instance_type = "t2.small"

  key_name = "${var.ssh_privkey_name}"

  user_data = "${base64encode(data.template_file.common_user_data.rendered)}"

  iam_instance_profile {
    name = "${module.infradev_remote_state.iam_profile_updater_id}"
  }

  lifecycle {
    create_before_destroy = true
  }
}

resource "aws_autoscaling_group" "asg" {
  name = "danny_test"

  vpc_zone_identifier = ["${split(",", module.environment.private_subnet_ids)}"]

  #launch_configuration      = "${aws_launch_configuration.lc.name}"

  min_size = "0"
  max_size = "1"
  desired_capacity = "1"
  health_check_grace_period = "600"
  health_check_type = "EC2"

  mixed_instances_policy {
    launch_template {
      launch_template_specification {
        launch_template_id = "${aws_launch_template.lt.id}"
      }
    }

    instances_distribution {
      on_demand_base_capacity = "1"
    }
  }
}

Expected Behavior

An ASG should be created with a Mixed Instances policy that ensures one of the EC2 instances is on-demand and the others are spots.

Actual Behavior

We get an error:

aws_autoscaling_group.asg: Error creating AutoScaling Group: ValidationError: The number of LaunchTemplateOverrides must be between 2 and 20.

Steps to Reproduce

1. terraform apply

References

• #7364
• https://stackoverflow.com/q/54400333/2291321
• https://groups.google.com/forum/#!topic/terraform-tool/9jY8J4JczNA

[shanemeyers]

I'm trying to do the same: use AutoScaling to launch a mixed set of spot and on-demand instances that are the same type within a single AutoScaling group.

This appears to be an AWS API response. Looking at the AWS AS API docs (https://docs.aws.amazon.com/autoscaling/ec2/APIReference/API_LaunchTemplate.html) I see that AWS enforces overrides to be between 2 and 20. But also Required: No, so that's a bit confusing.

As a result I tried creating the ASG using the console following the announcement blog post https://aws.amazon.com/blogs/aws/new-ec2-auto-scaling-groups-with-multiple-instance-types-purchase-options/. When I got to the first page of the console wizard, I see Add a minimum of two instance types under Instance Types. The only way to move on in the wizard is by specifying a second instance type.

So this will likely turn into a feature request for AWS to support mixed purchase options with a single instance type.

[zhaopengdirk]

I believe it's a doc miss to describe the overrides as Required:No, because I've tried this before using cloudformation, and in cloudformation doc or the AWS api doc, this parameter is describe as Required:Yes, that means you have to specify 2 ~ 20 overrides within the mixed policy, by the way, only instance type is supported for overrides currently, perhaps more parameters can be overridden in the future.

[kemra102]

Update: I _did_ raise a feature request with the Account Manager with the client I am working at to add the feature I thought already existed, and it was released a few weeks back (though I can't find the announcement now).

With a minor tweak to the above code (setting the on_demand_percentage_above_base_capacity as needed) you can get a balance of on-demand & spot on a single instance type:

resource "aws_launch_template" "lt" {
  name_prefix = "danny_test-"

  vpc_security_group_ids = [
    "${module.sec_groups.security_group_id_common}",
    "${module.sec_groups.security_group_id_ssh}",
    "${module.sec_groups.security_group_id_web}",
  ]

  image_id = "${module.core_ami.core_ami_id}"

  instance_type = "t3.small"

  key_name = "${var.ssh_privkey_name}"

  user_data = "${base64encode(data.template_file.common_user_data.rendered)}"

  iam_instance_profile {
    name = "${module.infradev_remote_state.iam_profile_updater_id}"
  }

  lifecycle {
    create_before_destroy = true
  }
}

resource "aws_autoscaling_group" "asg" {
  name = "danny_test"

  vpc_zone_identifier = ["${split(",", module.environment.private_subnet_ids)}"]

  #launch_configuration      = "${aws_launch_configuration.lc.name}"

  min_size = "0"
  max_size = "2"
  desired_capacity = "2"
  health_check_grace_period = "600"
  health_check_type = "EC2"

  mixed_instances_policy {
    launch_template {
      launch_template_specification {
        launch_template_id = "${aws_launch_template.lt.id}"
      }
    }

    instances_distribution {
      on_demand_base_capacity = "1"
      on_demand_percentage_above_base_capacity = "0"
    }
  }
}

This plans & applies without any error at all and works as I originally expected. As this is now working I'm going to close this issue. Thanks everyone for your input on this issue!

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!