Eksctl: document use of custom VPC and IAM

[errordeveloper]

We have /usage/vpc-networking which has general information, and /examples/reusing-iam-and-vpc/ which doesn't attempt to explains anything.
We also have examples/04-existing-vpc.yaml, but that's not on the website. It maybe a good idea to include examples/ on the website somehow.

We should create a more focused pages about IAM and VPC, and perhaps one that brings it all together.

[errordeveloper]

Per /examples/reusing-iam-and-vpc/, we haven't documented securityGroups.{withLocal,withShared}.

Here is the context that we should incorporate in the docs:

• by default each nodegroup inherits a security group that is shared with all other nodegroups in the cluster, this is in order to allow communication on all port between pods on all of the nodes; to disable this security group you can set securityGroups: {withShared: false}, which implies you will isolate the nodegroup from the rest of the cluster
• also by default, each nodegroup has a security group that is local to it, this security group allows communitcation between Kubernetes control plane and all the nodes in this nodegroup and all the pods running on them; additionally this security group holds any ingress/egress rules local to the given nodegroup (e.g. if you enable SSH for a nodegroup, this is the security group where access to port 22 will be controlled); to disable this security group you can set securityGroups: {withLocal: false}, which implies you will isolate the nodegroup from the rest of the cluster
• all in all, it is not recommended setting securityGroups: {withShared: false, withLocal: false} unless security groups must be managed outside of eksctl, in either cases you must specify all externally-managed security groups you want to use with securityGroups.attachIDs: []; i.e. normally securityGroups.attachIDs: [] is used to add extra security groups, but when either withShared: false or withLocal: false set, it should include security groups that fulfil the functionality of local and shared SGs that is outlined above

TL;DR: withShared: false or withLocal: false can be used to opt-out from built-in security groups, it's not recommended to use these settings unless you are required to control all of security groups externally.

[morsok]

Ah thank you for that ! Right know the documentation is really sparse, I have to use the config file schema, github issues and retro engineering to understand what i'm doing.

Your explanation is awesome @errordeveloper, this is exactly what I needed to use eksctl with networking managed by Terraform :) Thank you, I feel you should but that immediately into the docs !