Terraform-provider-kubernetes: kubernetes_service_account generates new secret on every apply

Created on 21 Oct 2020  路  12Comments  路  Source: hashicorp/terraform-provider-kubernetes

Terraform Version and Provider Version

Terraform v0.13.4

  • provider registry.terraform.io/hashicorp/kubernetes v1.13.2

Affected Resource(s)

kubernetes_service_account

Terraform Configuration Files

resource "kubernetes_service_account" "automated_tests" {
  metadata {
    name = "automated-tests"
    namespace = "automated-tests"
  }
  automount_service_account_token = true
}

Debug Output

Elided for security. Can be provided upon request

  # module.tivo_cluster.module.namespace.kubernetes_service_account.automated_tests will be updated in-place
  ~ resource "kubernetes_service_account" "automated_tests" {
        automount_service_account_token = true
        default_secret_name             = "automated-tests-token-l6srp"
        id                              = "automated-tests/automated-tests"

        metadata {
            annotations      = {}
            generation       = 0
            labels           = {}
            name             = "automated-tests"
            namespace        = "automated-tests"
            resource_version = "38561124"
            self_link        = "/api/v1/namespaces/automated-tests/serviceaccounts/automated-tests"
            uid              = "d2da3888-4394-4f46-9bc6-819199a94299"
        }

      - secret {
          - name = "automated-tests-token-m7md8" -> null
        }
    }

Panic Output

n/a

Expected Behavior

Terraform should create service account and corresponding secret _ONCE_.

Actual Behavior

Every apply, the secret is changed and updated on the service account. ALso, the secret and the "default_secret_name" don't match. And there are two secrets attached to the service account.

apiVersion: v1
automountServiceAccountToken: true
kind: ServiceAccount
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"v1","automountServiceAccountToken":true,"kind":"ServiceAccount","metadata":{"annotations":{},"name":"automated-tests","namespace":"automated-tests"}}
  creationTimestamp: "2020-08-07T11:50:35Z"
  name: automated-tests
  namespace: automated-tests
  resourceVersion: "38561712"
  selfLink: /api/v1/namespaces/automated-tests/serviceaccounts/automated-tests
  uid: d2da3888-4394-4f46-9bc6-819199a94299
secrets:
- name: automated-tests-token-l6srp
- name: automated-tests-token-9fbs9

Steps to Reproduce

Terraform apply twice or more, note that the secret keeps changing.

Important Factoids

n/a

References

n/a

Community Note

  • Please vote on this issue by adding a 馃憤 reaction to the original issue to help the community and maintainers prioritize this request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment
bug
Source
picture llamahunter
👍3

All 12 comments

There are secrets accumulating for this service account

% kubectl -n automated-tests get secret
NAME                          TYPE                                  DATA   AGE
automated-tests-token-2jxtw   kubernetes.io/service-account-token   3      8d
automated-tests-token-5lr5k   kubernetes.io/service-account-token   3      7d14h
automated-tests-token-6w84n   kubernetes.io/service-account-token   3      27d
automated-tests-token-6x7f5   kubernetes.io/service-account-token   3      10m
automated-tests-token-74l4f   kubernetes.io/service-account-token   3      8d
automated-tests-token-767rw   kubernetes.io/service-account-token   3      13h
automated-tests-token-7fk64   kubernetes.io/service-account-token   3      15d
automated-tests-token-84v8s   kubernetes.io/service-account-token   3      34d
automated-tests-token-8gfvs   kubernetes.io/service-account-token   3      7d11h
automated-tests-token-8nc9d   kubernetes.io/service-account-token   3      50d
automated-tests-token-9fbs9   kubernetes.io/service-account-token   3      4m37s
automated-tests-token-bv6s4   kubernetes.io/service-account-token   3      32d
automated-tests-token-cggr5   kubernetes.io/service-account-token   3      62d
automated-tests-token-d7bz8   kubernetes.io/service-account-token   3      4d17h
automated-tests-token-dln7c   kubernetes.io/service-account-token   3      8d
automated-tests-token-fl2tr   kubernetes.io/service-account-token   3      8d
automated-tests-token-gfqm9   kubernetes.io/service-account-token   3      7d12h
automated-tests-token-gkb87   kubernetes.io/service-account-token   3      40h
automated-tests-token-gng8g   kubernetes.io/service-account-token   3      7d11h
automated-tests-token-grz6s   kubernetes.io/service-account-token   3      7d20h
automated-tests-token-h82bq   kubernetes.io/service-account-token   3      4d17h
automated-tests-token-h8bsh   kubernetes.io/service-account-token   3      8d
automated-tests-token-h9hqb   kubernetes.io/service-account-token   3      28d
automated-tests-token-hs8w5   kubernetes.io/service-account-token   3      14d
automated-tests-token-j924v   kubernetes.io/service-account-token   3      8d
automated-tests-token-l4tkf   kubernetes.io/service-account-token   3      75d
automated-tests-token-ldpqf   kubernetes.io/service-account-token   3      8d
automated-tests-token-lklsg   kubernetes.io/service-account-token   3      38h
automated-tests-token-lm854   kubernetes.io/service-account-token   3      41d
automated-tests-token-lmzqc   kubernetes.io/service-account-token   3      62d
automated-tests-token-lwxtn   kubernetes.io/service-account-token   3      7d11h
automated-tests-token-m7md8   kubernetes.io/service-account-token   3      7m12s
automated-tests-token-mpkr5   kubernetes.io/service-account-token   3      25h
automated-tests-token-mxxhz   kubernetes.io/service-account-token   3      49d
automated-tests-token-mz5fw   kubernetes.io/service-account-token   3      7d
automated-tests-token-n9r47   kubernetes.io/service-account-token   3      3d23h
automated-tests-token-nglzs   kubernetes.io/service-account-token   3      8d
automated-tests-token-nj2hn   kubernetes.io/service-account-token   3      8d
automated-tests-token-qbp88   kubernetes.io/service-account-token   3      49d
automated-tests-token-r9q2q   kubernetes.io/service-account-token   3      34d
automated-tests-token-s6bt8   kubernetes.io/service-account-token   3      50d
automated-tests-token-v4ggf   kubernetes.io/service-account-token   3      4d17h
automated-tests-token-vvprc   kubernetes.io/service-account-token   3      8d
automated-tests-token-w2nm5   kubernetes.io/service-account-token   3      8d
automated-tests-token-w79kx   kubernetes.io/service-account-token   3      49d
automated-tests-token-wm84g   kubernetes.io/service-account-token   3      8d
automated-tests-token-wtq5t   kubernetes.io/service-account-token   3      15d
automated-tests-token-x4rkz   kubernetes.io/service-account-token   3      32d
automated-tests-token-xxktv   kubernetes.io/service-account-token   3      9d
automated-tests-token-zlxlj   kubernetes.io/service-account-token   3      7d10h
default-token-hlnk5           kubernetes.io/service-account-token   3      113d
llamahunter picture llamahunter on 21 Oct 2020

Destroying and re-creating the service account seems to have stopped the madness. Something very weird with the terraform state.

llamahunter picture llamahunter on 21 Oct 2020

I tried reproducing this and I'm unable too. I think we need more information about the environment where you run Terraform.

Also, are you able to reproduce this consistently or was it just a one-time?

alexsomesan picture alexsomesan on 28 Oct 2020

Discussed internally: It might be worth for us to test the provider when accessing the API in a low-latency situation (in-cluster).

alexsomesan picture alexsomesan on 28 Oct 2020

As above, it was 100% reproducible until I terraform destroyed and re-created the service account object. Then it stopped. Note the mismatch in the terraform state between the default_secret_name and the actual secret name. I think the problem was something with that.

llamahunter picture llamahunter on 28 Oct 2020

I have the same issue as the OP with the secret and the image_pull_secret. I see the same with that the default secret name is different to the secret name being created.

Running OpenShift 4.4

JoshStead picture JoshStead on 30 Nov 2020

I tried reproducing this and I'm unable too. I think we need more information about the environment where you run Terraform.

Also, are you able to reproduce this consistently or was it just a one-time?

It was consistently reproducible until I deleted the remote state, and then it stopped happening. I think there was something weird in the remote state between the default_secret_name and the actual secret in the resource. When they were out of sync, it kept making new secrets.

llamahunter picture llamahunter on 30 Nov 2020

Is there any update on this, it's going to stop us using Terraform to manage namspaces in our OpenShift clusters. @alexsomesan

JoshStead picture JoshStead on 11 Dec 2020

https://github.com/hashicorp/terraform-provider-kubernetes/issues/292 related issue

endzyme picture endzyme on 11 Dec 2020

@JoshStead @endzyme do you have a config/setup that reproduces this issue?

aareet picture aareet on 11 Dec 2020

Create a service account with the provider, then delete the service account secret in k8s (a normal process for rotations) - then run a plan on the resource you were managing. Can be tested with kind.

There's also a problem trying to import any Service Account if the secrets have changed. A new function to detect default secret breaks imports (already known issue).

endzyme picture endzyme on 11 Dec 2020
👀1 👍1

Also observed the default_secret_name and the existing secret were out of sync, and kept making new secrets on each apply.
To fix I did the following steps (no terraform state manipulation required):

  1. manually deleted all orphaned secrets
  2. manually deleted the problematic kubernetes_service_account
  3. terraform apply
adamrushuk picture adamrushuk on 25 Mar 2021
Was this page helpful?
0 / 5 - 0 ratings

Related issues

rehevkor5 picture rehevkor5  路  3Comments
bdronneau picture bdronneau  路  4Comments
ragelo picture ragelo  路  3Comments
dmitry-mightydevops picture dmitry-mightydevops  路  3Comments
noahbirrer-8451 picture noahbirrer-8451  路  3Comments