Terraform-aws-eks: Deploying alb-ingress controller

[syst0m]

I have issues

with deploying the alb-ingress-controller.

I'm submitting a...

• [ ] bug report
• [ ] feature request
• [/ ] support request
• [ ] kudos, thank you, warm fuzzy

What is the current behavior?

Following the official guide located here:
https://kubernetes-sigs.github.io/aws-alb-ingress-controller/guide/controller/setup/
Have added kubelet_extra_args = "--cloud-provider=aws" in my worker-groups definition.

The guide states that adequate role and policies must be configured for the nodes.
There's an example policy here:
https://kubernetes-sigs.github.io/aws-alb-ingress-controller/examples/iam-policy.json

I'm not sure where and how to attach that policy to my workers

Environment details

• Affected module version: 1.7.0
• OS: 10.14.1
• Terraform version: 0.11.10

Any other relevant info

[syst0m]

I tried installing kiam with a helm chart, but I ran into issues with that as well:
https://github.com/helm/charts/issues/9698

How could I append the necessary iam-policy.json to the existing one used in terraform-aws-eks?

[syst0m]

Also, I didn't find any override key in locals.tf to use for my worker group.

[laverya]

You can attach the policy in question to the default IAM role produced for your worker nodes, if you don't need worker groups with different permissions. That would look something like this:

resource "aws_iam_role_policy_attachment" "albIngressControllerEksPolicyAttachment" {
  policy_arn = "${aws_iam_policy.albIngressControllerEksPolicy.arn}"
  role       = "${module.eks.worker_iam_role_name}"
}

resource "aws_iam_policy" "albIngressControllerEksPolicy" {
  name_prefix = "albIngressControllerEksPolicy"
  description = "ALB ingress controller eks policy"
  policy      = "${data.aws_iam_policy_document.alb_ingress_controller_eks.json}"
}

data "aws_iam_policy_document" "alb_ingress_controller_eks" {
  statement {
YOUR STATEMENT HERE
  }
}

[syst0m]

Hi @laverya ,

where should I add that block? In my main.tf?

[syst0m]

And if I wanted to use kube2iam,
where can I find which role is associated with my k8s nodes?
As mentioned in kube2iam walkthrough:
https://kubernetes-sigs.github.io/aws-alb-ingress-controller/guide/walkthrough/echoserver/

I'm seeing several diferent iam_role objects getting created after terraform apply:
aws_iam_role_policy_attachment.workers_AmazonEKSWorkerNodePolicy: Refreshing state... (ID: primarybid-eks-dev-8h8xxBJJ20181128104159665200000009-2018112810420110090000000e) aws_iam_role_policy_attachment.workers_AmazonEKS_CNI_Policy: Refreshing state... (ID: primarybid-eks-dev-8h8xxBJJ20181128104159665200000009-2018112810420093090000000d) aws_iam_role_policy_attachment.workers_AmazonEC2ContainerRegistryReadOnly: Refreshing state... (ID: primarybid-eks-dev-8h8xxBJJ20181128104159665200000009-2018112810420092980000000c) data.template_file.userdata: Refreshing state... aws_iam_instance_profile.workers: Refreshing state... (ID: primarybid-eks-dev-8h8xxBJJ2018112810420057470000000b)

Also an IAM role for the cluster:
aws_iam_role.cluster: Refreshing state... (ID: primarybid-eks-dev-8h8xxBJJ20181128103302665600000001)

Not sure what's the difference here, which one to use.

[max-rocket-internet]

Do you still need help here @syst0m ?

[syst0m]

I was just planning to try this out again with the new tf eks module, will update here with status, thanks!

[bmacauley-reward]

@max-rocket-internet Does anyone have an ALB ingress controller or kube2iam example where the roles/policies are attached to the worker nodes?

[max-rocket-internet]

I was just planning to try this out again with the new tf eks module, will update here with status, thanks!

OK I'll close now as it's been a month.

Does anyone have an ALB ingress controller or kube2iam example where the roles/policies are attached to the worker nodes?

I don't know, sorry.

[arthurio]

For posterity, here is how I declared my policy:

resource "aws_iam_policy" "aws_alb_ingress_controller" {
  # https://github.com/kubernetes-sigs/aws-alb-ingress-controller/blob/master/docs/examples/iam-policy.json
  name        = "${local.environment}-${var.project_name}-aws-alb-ingress-controller"
  description = "Allow alb ingress controller to do all the things it needs to"

  policy = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["acm:DescribeCertificate", "acm:ListCertificates", "acm:GetCertificate"],
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "ec2:AuthorizeSecurityGroupIngress",
        "ec2:CreateSecurityGroup",
        "ec2:CreateTags",
        "ec2:DeleteSecurityGroup",
        "ec2:DescribeInstances",
        "ec2:DescribeInstanceStatus",
        "ec2:DescribeSecurityGroups",
        "ec2:DescribeSubnets",
        "ec2:DescribeTags",
        "ec2:DescribeVpcs",
        "ec2:ModifyInstanceAttribute",
        "ec2:RevokeSecurityGroupIngress"
      ],
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "elasticloadbalancing:AddTags",
        "elasticloadbalancing:CreateListener",
        "elasticloadbalancing:CreateLoadBalancer",
        "elasticloadbalancing:CreateRule",
        "elasticloadbalancing:CreateTargetGroup",
        "elasticloadbalancing:DeleteListener",
        "elasticloadbalancing:DeleteLoadBalancer",
        "elasticloadbalancing:DeleteRule",
        "elasticloadbalancing:DeleteTargetGroup",
        "elasticloadbalancing:DeregisterTargets",
        "elasticloadbalancing:DescribeListeners",
        "elasticloadbalancing:DescribeListenerCertificates",
        "elasticloadbalancing:DescribeLoadBalancers",
        "elasticloadbalancing:DescribeLoadBalancerAttributes",
        "elasticloadbalancing:DescribeRules",
        "elasticloadbalancing:DescribeSSLPolicies",
        "elasticloadbalancing:DescribeTags",
        "elasticloadbalancing:DescribeTargetGroups",
        "elasticloadbalancing:DescribeTargetGroupAttributes",
        "elasticloadbalancing:DescribeTargetHealth",
        "elasticloadbalancing:ModifyListener",
        "elasticloadbalancing:ModifyLoadBalancerAttributes",
        "elasticloadbalancing:ModifyRule",
        "elasticloadbalancing:ModifyTargetGroup",
        "elasticloadbalancing:ModifyTargetGroupAttributes",
        "elasticloadbalancing:RegisterTargets",
        "elasticloadbalancing:RemoveTags",
        "elasticloadbalancing:SetIpAddressType",
        "elasticloadbalancing:SetSecurityGroups",
        "elasticloadbalancing:SetSubnets",
        "elasticloadbalancing:SetWebACL"
      ],
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": ["iam:GetServerCertificate", "iam:ListServerCertificates"],
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": ["waf-regional:GetWebACLForResource"],
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": ["tag:GetResources"],
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "waf:GetWebACL",
        "waf:CreateWebACL",
        "waf:DeleteWebACL"
      ],
      "Resource": "*"
    }
  ]
}
EOF
}

And how to attach it:

module "cluster" {
  # ...
  workers_additional_policies = [
    "${aws_iam_policy.aws_alb_ingress_controller.arn}",
  ]
  workers_additional_policies_count = 1
  # ...
}

[rokrokss]

Is there a different solution so we can use kiam?

[rokrokss]

above comment can be solved with using https://github.com/terraform-aws-modules/terraform-aws-iam/tree/master/modules/iam-assumable-role-with-oidc module and having eks.amazonaws.com/role-arn: {ARN} annotation on the serviceaccount, just like the irsa example.