Terraform-provider-aws: aws_spot_fleet_request - launch_specification - subnet_id: multiple values

[hashibot[bot]]

_This issue was originally opened by @Bryksin as hashicorp/terraform#23573. It was migrated here as a result of the provider split. The original body of the issue is below._

---

Hi

I have problems with aws_spot_fleet_request -> launch_specification -> subnet_id

According to the official AWS documentation it says:

To specify multiple subnets, separate them using commas; 
for example, "subnet-1234abcdeexample1, subnet-0987cdef6example2".

Here is my terraform code snippet:

resource "aws_spot_fleet_request" "ecs-spot-fleet" {
    iam_fleet_role          = var.SPOTFLEET_ROLE_ARN
    target_capacity         = var.NUMER_OF_SPOTFLEET_INSTANCES[var.ENV]

    dynamic "launch_specification" {
        for_each = toset(var.SPOTFLEET_INSTANCE_TYPES[var.ENV])
        content {
            instance_type               = launch_specification.value
            ami                         = var.ECS_AMI_ID
            iam_instance_profile_arn    = var.INSTANCE_PROFILE_ARN
            key_name                    = var.INSTANCE_KEY_PAIR
            user_data                   = data.template_file.ecs_user_data_tamplate.rendered
        subnet_id           = join(",", var.PRIV_SUBNET_VPC_IDs)
        vpc_security_group_ids  = concat(var.EC2_SECURITY_GROUPS_TO_ATTACH[var.ENV], [aws_security_group.spot-security-group.id])

            tags = {
                Name = "${var.SERVICE_NAME}-ECS-SpotFleet-Instance"
            }
        }
    }

    depends_on = [
        aws_security_group.spot-security-group
    ]
}

here is how it looks during planning phase:

*** omitted ***
 + launch_specification {
          + ami                         = "ami-0bf45a5f4ab05b949"
          + associate_public_ip_address = false
          + availability_zone           = (known after apply)
          + ebs_optimized               = false
          + iam_instance_profile_arn    = (known after apply)
          + instance_type               = "t3a.medium"
          + key_name                    = "international-ecs-test-112019"
          + monitoring                  = false
          + placement_group             = (known after apply)
          + subnet_id                   = "subnet-052d62913f554f574,subnet-0cd63408273ed571e,subnet-0cd63408273ed571e"
          + tags                        = {
              + "Name" = "TF-AuthService-ECS-SpotFleet-Instance"
            }
          + user_data                   = "66494d6cda4681a1f74e85445e62c413fceb3f5e"
          + vpc_security_group_ids      = (known after apply)

          + ebs_block_device {
              + delete_on_termination = (known after apply)
              + device_name           = (known after apply)
              + encrypted             = (known after apply)
              + iops                  = (known after apply)
              + kms_key_id            = (known after apply)
              + snapshot_id           = (known after apply)
              + volume_size           = (known after apply)
              + volume_type           = (known after apply)
            }

          + ephemeral_block_device {
              + device_name  = (known after apply)
              + virtual_name = (known after apply)
            }

          + root_block_device {
              + delete_on_termination = (known after apply)
              + encrypted             = (known after apply)
              + iops                  = (known after apply)
              + kms_key_id            = (known after apply)
              + volume_size           = (known after apply)
              + volume_type           = (known after apply)
            }
        }
*** omitted ***

however after I apply it, it fails midway through with error:

Error: Error requesting spot fleet: Error creating Spot fleet request, retrying: InvalidSpotFleetRequestConfig: Duplicate: Parameter combination in LaunchSpecification: t2.medium, ami-0bf45a5f4ab05b949, Linux/UNIX, c5.large, ami-0bf45a5f4ab05b949, Linux/UNIX, t3.medium, ami-0bf45a5f4ab05b949, Linux/UNIX, t3.small, ami-0bf45a5f4ab05b949, Linux/UNIX, c5d.large, ami-0bf45a5f4ab05b949, Linux/UNIX, t3a.medium, ami-0bf45a5f4ab05b949, Linux/UNIX, a1.large, ami-0bf45a5f4ab05b949, Linux/UNIX, t3a.small, ami-0bf45a5f4ab05b949, Linux/UNIX are invalid.
        status code: 400, request id: 1a9aff81-ec6f-49d8-b5b3-3bf20a4559d0

  on modules/auth_service/spot_fleet.tf line 12, in resource "aws_spot_fleet_request" "ecs-spot-fleet":
  12: resource "aws_spot_fleet_request" "ecs-spot-fleet" {


[terragrunt] 2019/12/05 16:02:44 Hit multiple errors:

if I would place only single subnet element(var.PRIV_SUBNET_VPC_IDs, 0) instead of join(",", var.PRIV_SUBNET_VPC_IDs)
everything works, but I'm bound to only one subnet and not able to spread instances properly

The same relates to availability_zone
in official docs says:

[Spot Fleet only] To specify multiple Availability Zones, separate them using commas; for example, "us-west-2a, us-west-2b".

but terraform takes only single value string

additionally, I would suggest allowing type list for subnet_id and availability_zone
so I would not need to join(", ", var.MyList)

[blckct]

Well, the documentation states clearly it is not supported: https://www.terraform.io/docs/providers/aws/r/spot_fleet_request.html
However I think dynamic blocks may be solution for it:
https://www.terraform.io/docs/configuration/expressions.html#dynamic-blocks

[Bryksin]

Oh my bad, missed Note: bock completely.
In that case, can we mark this ticket as improvement or feature request ? as it feels like the right feature which will allow reflecting AWS behaviour 1 to 1

But here is a question, I'm already using a dynamic block for launch_specification if I want to have also different subnets, it will have to be nested loop

like:

//Pseudo-code
for_each type in var.myListOfInstanceTypes {
    for_each subnet in var.myListOfsubnets {
        launch_specification {
             instance_type = type.value
             subnet_id = subnet.value
        }
    }
}

How to make it working with terraform dynamic block with nested loop?

resource "aws_spot_fleet_request" "my_spot_fleet" {
    some_props = omitted

    dynamic "launch_specification" {
        for_each = toset(var.myListOfInstanceTypes)
        content {
            instance_type = launch_specification.value
            subnet_id = ???
            other_props = omitted
        }
    }
}

[blckct]

@Bryksin How about https://www.terraform.io/docs/configuration/functions/setproduct.html ?

As for marking the ticket we'll have to wait for actual maintainers. I'd say that docs could probably be upgraded to point out the dynamic block functionality.

[Bryksin]

@blckct Thank you for the hint, it really helped:

resource "aws_spot_fleet_request" "my_spot_fleet" {
    some_props = omitted

    dynamic "launch_specification" {
        for_each = setproduct(var.myListOfInstanceTypes, var.listOfVPCzones)
        content {
            instance_type = launch_specification.value[0]
            subnet_id = launch_specification.value[1]
            other_props = omitted
        }
    }
}