Terraform-aws-eks: Support for aws-auth in terraform enterprise

[wstewartii]

I'm submitting a...

• [ ] bug report
• [ ] feature request
• [ x] support request
• [ ] kudos, thank you, warm fuzzy

I'd like to have terraform enterprise automatically apply the aws-auth ConfigMap after provisioning an eks cluster. Currently it's required to add the aws-iam-authenticator to your PATH and have kubectl available. The enterprise version of terraform doesn't allow me the flexibility to install binaries or touch the folders in the $PATH environment variable. I'd like to know if there are any plans to add support for this or if there is a recommended workaround.

[Nikamura]

Can't you create folder bin/ and add it to $PATH ? With applications you need downloaded.

[max-rocket-internet]

You can also just change your KUBECONFIG to a path of your choosing, right?

e.g.

...
users:
- name: my-cluster
  user:
    exec:
      apiVersion: client.authentication.k8s.io/v1alpha1
      command: /my/path/to/aws-iam-authenticator
      args:
        - "token"
        - "-i"
        - "my-cluster"

[discordianfish]

Trying the same here..

@max-rocket-internet The problem is the kubectl invocation here: https://github.com/terraform-aws-modules/terraform-aws-eks/blob/master/aws_auth.tf#L17

I'm currently trying to use a local-exec to install kubectl to /usr/bin/. That seems to success (eg the provisionber succeeeds without error output) but for some reason it still can't find it. I'll update this issues if I find a solution.

[discordianfish]

Okay, I spent most of my day trying to somehow get a single terraform apply run to first install kubectl locally and then run the aws-eks module. I don't think that's possible..

I've tried various hacks like running the install in data.external, but that seems to get execute in a different environment. I also tried running with null_(data_)resource which again isn't working since I can only capture the null_resources 'id' which is available before it finishs... So I conclude we need support for that in here, e.g something that passes through a reference to the null resource executing kubectl. I'll look into that tomorrow.

[discordianfish]

Okay, I finally managed to solve that even without changing this module. Instead of trying to install kubectl etc prior to the module being executed*, I set manage_aws_auth = false and uses the rendered config map module output and applied that after cluster creation in a null_resource/local-exec.

*) At this point I say is impossible since there is no way to make a module depend on a null resource, even with hacks since the null resource has no outputs or attributes that would be only available after it finished

[eytanhanig]

Here's what I use:

resource "null_resource" "apply_configs" {
  triggers {
    config_map_aws_auth = "${sha256(module.eks.config_map_aws_auth)}"
    kubeconfig          = "${sha256(module.eks.kubeconfig)}"
  }

  provisioner "local-exec" {
    environment = {
      AWS_CONFIG_MAP_RENDERED = "${module.eks.config_map_aws_auth}"
      KUBE_CONFIG_MAP_RENDERED = "${module.eks.kubeconfig}"
    }
    command = <<EOH
mkdir bin
curl -s -o bin/aws-iam-authenticator https://amazon-eks.s3-us-west-2.amazonaws.com/1.12.7/2019-03-27/bin/linux/amd64/aws-iam-authenticator
curl -s -L -o bin/kubectl https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl
chmod 755 bin/*
export PATH=$(readlink -f bin):$PATH
echo "$AWS_CONFIG_MAP_RENDERED" > aws_auth_configmap.yaml
echo "$KUBE_CONFIG_MAP_RENDERED" > kube_config.yaml
for i in `seq 1 10`; do
  kubectl apply -f aws_auth_configmap.yaml --kubeconfig kube_config.yaml && break || sleep 10;
done;
EOH
  }
}

It has the advantage of only running when configs change.

[max-rocket-internet]

Closing with @eytanhanig idea as a work around. If someone has a nice and elegant solution then feel free to make a PR 🙂