Kops: Add the ability to deploy T2 Unlimited instances on AWS

[adammillerio]

1. AWS recently announced T2 Unlimited instances, which is essentially just a setting on a t2 class instance that will make it "purchase" extra credits instead of returning to the baseline performance level once credits are exhausted. This would be ideal for a master in a low-utilization cluster, as it would allow for a cheaper instance type, while also not running the risk of baseline performance causing problems with system components. Kops should allow the user to deploy a T2 unlimited by specifying it in an instance group configuration.

2. This might be annoying to implement, since it is an additional configuration option specific to a certain class of instances.

Essentially, you either call RunInstances with CreditSpecification=unlimited if they are new, or run ModifyInstanceCreditSpecification if it is being updated. See docs here.

As for configuration, two possibilities that come to mind would be either including specific machineTypes for unlimited instances (e.g. t2.medium.unlimited) or adding an additional configuration option such as machineCreditSpecification.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[mfrister]

/remove-lifecycle stale

[chrislovecnm]

/lifecycle frozen
/remove-lifecycle stale

[lawrencedudley]

This would be amazing

[lawrencedudley]

We use t2 unlimited in other areas - annoyingly it's not supported on launch configurations for autoscaling groups, just launch templates (which I think signifies that AWS is suggesting that's where the future lies).

So a prerequisite for this is to move the configuration to launch templates first. I've not dug into the kops code properly but will try and do so over the next few weeks to see what the implications might be if nobody has had a go yet by then.

[thedarkfalcon]

Just voicing my opinion as this thread is stale.

I still think this would be very useful. I have experienced pods/masters dying due to running out of CPU credits when running T instances. The T instances are much more cost effective for us, however sometimes if there is a sudden burst of CPU resource required, we can bottom out on CPU credit which can cause the cluster to stall. I don't want to increase the instance sizes, simply purchasing an additional hours worth of compute when required is much better (what T2 unlimited does).

However, the new T3 instances have this functionality built into them. So possibly if T3 instances become available via KOPS, this ticket would be redundant. Either way, I think KOPS urgently needs to support either T2 Unlimited, or T3 instances.

[Globegitter]

Even though T3 instances have this on per default it would still be nice to be able to turn this off, as sometimes it is preferable to have performance degradation rather than higher costs.

[thedarkfalcon]

I suspect the response for this would be simply to use T2 instances instead, which yes I know, is 30% less efficient per $.

[rifelpet]

LaunchConfigurations do not support credit specifications, and Kops currently uses LaunchConfigurations with its ASGs. #6277 has been opened to support LaunchTemplates which do support credit specification. Once merged it should be pretty simple to expose a credit specification field in the API, allowing the control of standard/unlimited T2s and T3s.

[bigdrum]

Hello, is there any plan on this now that #6277 is merged?

[david-in-perth]

@gambol99 : I see reference to "CreditSpecification" in the #6277 diff.
Does that mean this ticket is now implemented and can be closed (along with #6284)?

I couldn't find any reference to it in the docs.

[ahharu]

Any updates?

[erdii]

Ping

[lapcchan]

any update?

[constructdian]

Any updates to this?

[jkieley]

any update?

[olemarkus]

The plan is to enable LaunchTemplates by default in kops 1.18/1.19. But we Still need to make this configurable in the instance group spec.

[olemarkus]

/kind feature