Terraform-provider-helm: Error when destroying in RBAC enabled cluster

[sebastianvoss]

Terraform Version

Terraform v0.11.7

Affected Resource(s)

• helm_release
• kubernetes_service_account
• kubernetes_cluster_role_binding

Terraform Configuration Files

resource "kubernetes_service_account" "tiller_service_account" {
  metadata {
    name      = "tiller"
    namespace = "kube-system"
  }
}

resource "kubernetes_cluster_role_binding" "tiller_cluster_role_binding" {
  metadata {
    name = "tiller"
  }

  role_ref {
    name = "cluster-admin"
    kind = "ClusterRole"
  }

  subject {
    kind      = "ServiceAccount"
    name      = "${kubernetes_service_account.tiller_service_account.metadata.0.name}"
    namespace = "kube-system"
  }
}

resource "helm_release" "xyz" {
  name       = "xyz"
  repository = "${helm_repository.myrepo.metadata.0.name}"
  chart      = "xyz"
  namespace  = "xyz"
}

Expected Behavior

The helm_release should have been destroyed before the kubernetes_service_account and kubernetes_cluster_role_binding get destroyed.

Actual Behavior

The helm_release, kubernetes_service_account and kubernetes_cluster_role_binding get destroyed in parallel which leads to permission problems for tiller when RBAC is enabled:

helm_release.xyz: Destroying... (ID: xyz)
kubernetes_cluster_role_binding.tiller_cluster_role_binding: Destroying... (ID: /tiller)
...

Error: Error applying plan:

1 error(s) occurred:

* helm_release.xyz (destroy): 1 error(s) occurred:

* helm_release.xyz: rpc error: code = Unknown desc = configmaps is forbidden: User "system:serviceaccount:kube-system:tiller" cannot list configmaps in the namespace "kube-system"

Steps to Reproduce

1. terraform apply
2. terraform destroy

[CharlieC3]

A workaround I found was to have the helm_release depend on the kubernetes_cluster_role_binding, and have the kubernetes_cluster_role_binding depend on the kubernetes_service_account. With this setup the resources are able to completely terminate as expected.

[meyskens]

I don't think that this is a bug in the Helm provider but more of how Terraform handles handles dependencies, the Helm provider isn't aware of the Kubernetes provider and vice versa.

[james-emerton]

I ran into this issue and was able to establish the correct dependencies without having to add a manual dependency to a large number of helm_release resources:

provider "helm" {
  service_account = kubernetes_cluster_role_binding.tiller.subject.0.name
}

I found that I couldn't add depends_on directly to a provider, but by referencing the service_account via the kubernetes_cluster_role_binding (as opposed to the more natural kubernetes_service_account) a proper dependency tree would be built that would force all the helm_release resources to be destroyed before kubernetes_cluster_role_binding

Hopefully this will be helpful to the next person who stumbles into this problem.

[4x0v7]

@james-emerton Thank you! This worked for me