Eksctl: Placement Group support when creating EKS cluster

[Jeffwan]

Before creating a feature request, please search existing feature requests to see if you find a similar one. If there is a similar feature request please up-vote it and/or add your comments to it instead
I didn't find it's there, please correct me if you already have it.

Why do you want this feature?
Lots of machine learning workloads requires low latency network support and users like to leverage placement group feature to achieve that. (I think single AZ is supported and we'd like to move further)

What feature/behavior/change do you want?
A clear and concise description of the change you would like to see in eksctl.

1. Add placement group option
2. support all placements
3. Add into cloudFormation template
4. do basic validation, for example, cluster placement group has to be in single AZ. has to get at least 2 AZ to support spread.

eksctl create cluster --region=us-west-2 --placement_group=cluster --xxxxx

Do not hesitate, when appropriate, to share the exact commands or API you would like, and/or to share a diagram (e.g.: asciiflow.com): "a picture is worth a thousand words".

Let me know this this is reasonable and I can help on the implementation

[whereisaaron]

I suggested in #198 that when creating node group we could provide an AWS Launch Template ID as the basis for the node group instances. Then many advanced options like Place Groups could be passed through to the ASG, with having to code every single AWS instance option into eksctl.

@errordeveloper can we leverage Launch Templates to provide options like spot instances? That could cover a lot of other requests I see in issues. E.g. I would like to be able to create node pools of nodes that have extra volumes attached. I could provide eksctl with a Launch Template.

Anything specified in the eksctl config or command line options would override the template, but otherwise everything from extra tags, extra volumes, tenancy, spot instance choices, T2/T3-unlimited option, placement groups etc. could come from the Launch Template, rather than duplicating all these options within eksctl?

[Jeffwan]

@whereisaaron Thanks. Expose Launch Templates makes sense what if I just want to have a clean cluster without making any other changes? That may not be easy in this way.

[whereisaaron]

Hi @Jeffwan, specifying a Launch Template would be entirely optional. For standard clusters eksctl would create the launch template for you using the command line or config file options you specify.

The option to provide your own custom Launch Template would allow you to customize a great many advanced setting, including placement groups.

If eksctl already had Launch Template support, you wouldn't need this feature request, you would just create a Launch Template with a Placement Group and pass it to eksctl.

[mbagliojr]

Does anyone have an example or documentation on how to do this?

[errordeveloper]

We have just merged Launch Template changes (#743), which are expected to land in the next release (potentially due in before end of this week).

[dalbhanj]

Has #743 landed already? Any examples on how to use launch templates along with placement groups available would be really helpful

[martina-if]

Hi @dalbhanj , eksctl already uses launch templates but it doesn't support placement groups yet. We will update this issue when it lands :) (hopefully soon)

[mbagliojr]

Any update on a placement group example?

[nigesh-pugazhendhi]

@martina-if any update on placement-groups via eksctl? is there any workaround if not ready yet?

[martina-if]

Hi @mbagliojr @nigesh-pugazhendhi , unfortunately this work has been down-prioritized lately but pull requests are welcome :)

[michaelbeaumont]

@Jeffwan We're looking at supporting this by specifying a launch template in the nodegroup config, where options like Placement group can be defined.

For the case of placement groups, would it be sufficient to be able to supply a launch template ID (for a launch template created outside of eksctl, where the placement group would be referenced) or is it important to be able to define the launch template referencing the placement group _inline_ in the config?

I.e.:

nodeGroups:
  - name: ng-6
    instanceType: m5.large
    desiredCapacity: 1
    customLaunchTemplateName: launch-template

or

nodeGroups:
  - name: ng-6
    instanceType: m5.large
    desiredCapacity: 1
    customLaunchTemplateData:
      Placement:
        GroupName: placement-test

[Jeffwan]

@michaelbeaumont Thanks for coming back on this. customLaunchTemplate is a good idea to support lot of generic cases. I think from user's perspective, they'd like to put nodes in nodeGroup under one placement group.

Either pass an existing placement for that node group (ASG) to use or create a new placement group if it doesn't exists.

I think following is more aligned with users's requirement

nodeGroups:
  - name: ng-6
    instanceType: m5.large
    desiredCapacity: 1
    customLaunchTemplateData:
      Placement:
        GroupName: placement-test

[Jeffwan]

@michaelbeaumont Any updates on this feature request?

[michaelbeaumont]

@Jeffwan The current plan for the first iteration is the basic version, to add a new option to nodeGroups:

placement:
  groupName: "group-name"

where the placement group has to exist already.

[Jeffwan]

Talked with @michaelbeaumont offline. customLaunchTemplateData in unmanaged node group is not on the road map yet. This needs more discussion.