Terraform-aws-eks: Better options for spot instances: Support aws_ec2_fleet or aws_spot_fleet_request

[max-rocket-internet]

I have issues

I'm submitting a feature request.

What is the current behavior?

No option of having a worker group run on either aws_ec2_fleet or aws_spot_fleet_request.

What's the expected behavior?

We should support this!

Current implementation using ASG/LC or ASG/LT are not as good and lack some of the advanced scheduling options that are available in aws_ec2_fleet and aws_spot_fleet_request. For example, specifying instance by number of CPUs instead of instance type.

It's super useful to run spot instances because:

• You can save a load of money
• AWS is pushing this too, e.g. 1, 2
• Using k8s-spot-termination-handler will drain your nodes gracefully when prices change
• The nature of k8s itself is well positioned for dealing with instances coming and going anyway
• You can save a load of money!

Are you able to fix this problem and submit a PR? Link here if you have already.

Not sure just yet, still investigating

[hato221]

The definition of spot instances is visible here https://github.com/terraform-aws-modules/terraform-aws-eks/blob/master/examples/eks_test_fixture/main.tf#L73 and here https://github.com/terraform-aws-modules/terraform-aws-eks/blob/master/workers_launch_template.tf#L17
Is it helpful for you?

[max-rocket-internet]

Is it helpful for you?

Not really 😃

That example will launch spot instances of only 2 instance types. That's great but I don't think 2 is enough to provide diversity for when AWS terminates the instances. Sometimes they will claim back instances or the price will suddenly rise for a single instance type or in a single AZ.

It would be ideal if this module could support between 2-6 instance type overrides per ASG. But I don't think it's possible to have them optional due to this issue: https://github.com/terraform-providers/terraform-provider-aws/issues/8475

And then supporting aws_ec2_fleet or aws_spot_fleet_request would allow for us to not even specify the instance type as you can just specify something like "I want instances with 4 CPUs", which is even more flexible.

[max-rocket-internet]

I will close this now as with TF 0.12 we can support any number of instance overrides: https://github.com/terraform-aws-modules/terraform-aws-eks/blob/4369f0271b12cd3f98f705df036662f0660efa4b/workers_launch_template.tf#L124-L134