Terraform-provider-kubernetes: Importing a ServiceAccount fails because Secret creation timestamp differs "too much"

Created on 19 May 2020  路  2Comments  路  Source: hashicorp/terraform-provider-kubernetes

Importing an existing ServiceAccount fails because the Secret creationDate differs too much.

kubernetes_service_account.projekt_serviceaccounts["eec"]: Importing from ID "eec/eec"...

Error: Failed to discover the default service account token: Unable to find any service accounts tokens which could have been the default one

Terraform Version and Provider Version

Terraform v0.12.25

  • provider.kubernetes v1.11.2

Affected Resource(s)

  • kubernetes_service_account

Debug Output

I cut some unnecessary parts out...

kubernetes_service_account.projekt_serviceaccounts["service"]: Importing from ID "service/service"...
2020-05-19T13:08:22.816+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4: 2020/05/19 13:08:22 [DEBUG] Kubernetes API Request Details:
2020-05-19T13:08:22.817+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4: ---[ REQUEST ]---------------------------------------
2020-05-19T13:08:22.817+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4: GET /api/v1/namespaces/service/serviceaccounts/service HTTP/1.1
2020-05-19T13:08:22.817+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4: -----------------------------------------------------
2020-05-19T13:08:22.954+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4: 2020/05/19 13:08:22 [DEBUG] Kubernetes API Response Details:
2020-05-19T13:08:22.954+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4: ---[ RESPONSE ]--------------------------------------
2020-05-19T13:08:22.954+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4: {
2020-05-19T13:08:22.954+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4:  "kind": "ServiceAccount",
2020-05-19T13:08:22.954+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4:  "apiVersion": "v1",
2020-05-19T13:08:22.954+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4:  "metadata": {
2020-05-19T13:08:22.954+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4:   "name": "service",
2020-05-19T13:08:22.954+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4:   "namespace": "service",
2020-05-19T13:08:22.954+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4:   "creationTimestamp": "2020-05-14T12:21:28Z",
2020-05-19T13:08:22.954+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4:  },
2020-05-19T13:08:22.954+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4:  "secrets": [
2020-05-19T13:08:22.954+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4:   {
2020-05-19T13:08:22.954+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4:    "name": "service-token-kjpmt"
2020-05-19T13:08:22.954+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4:   }
2020-05-19T13:08:22.954+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4:  ]
2020-05-19T13:08:22.954+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4: }
2020-05-19T13:08:22.954+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4:
2020-05-19T13:08:22.954+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4: -----------------------------------------------------
2020-05-19T13:08:22.955+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4: 2020/05/19 13:08:22 [DEBUG] Kubernetes API Request Details:
2020-05-19T13:08:22.955+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4: ---[ REQUEST ]---------------------------------------
2020-05-19T13:08:22.955+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4: GET /api/v1/namespaces/service/secrets/service-token-kjpmt HTTP/1.1
2020-05-19T13:08:22.955+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4: -----------------------------------------------------
2020-05-19T13:08:22.960+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4: 2020/05/19 13:08:22 [DEBUG] Kubernetes API Response Details:
2020-05-19T13:08:22.960+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4: ---[ RESPONSE ]--------------------------------------
2020-05-19T13:08:22.960+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4: {
2020-05-19T13:08:22.960+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4:  "kind": "Secret",
2020-05-19T13:08:22.960+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4:  "apiVersion": "v1",
2020-05-19T13:08:22.960+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4:  "metadata": {
2020-05-19T13:08:22.960+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4:   "name": "service-token-kjpmt",
2020-05-19T13:08:22.960+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4:   "namespace": "service",
2020-05-19T13:08:22.960+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4:   "selfLink": "/api/v1/namespaces/service/secrets/service-token-kjpmt",
2020-05-19T13:08:22.960+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4:   "uid": "58e7c0f0-95dd-11ea-be2c-005056a1182f",
2020-05-19T13:08:22.960+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4:   "resourceVersion": "85652125",
2020-05-19T13:08:22.960+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4:   "creationTimestamp": "2020-05-14T12:20:50Z",
...
2020-05-19T13:08:22.961+0200 [DEBUG] plugin.terraform-provider-kubernetes_v1.11.2_x4: 2020/05/19 13:08:22 [DEBUG] Skipping service-token-kjpmt as it existed before the service account
2020/05/19 13:08:22 [ERROR] <root>: eval: *terraform.EvalImportState, err: Failed to discover the default service account token: Unable to find any service accounts tokens which could have been the default one
2020/05/19 13:08:22 [ERROR] <root>: eval: *terraform.EvalSequence, err: Failed to discover the default service account token: Unable to find any service accounts tokens which could have been the default one

Expected Behavior

Resource should have been imported.
The ServiceAccount references its own token by name, so why not import it ?

Actual Behavior

creationTimestamps are compared and because the Secret somehow got a creationTimestamp before the ServiceAccount, the import fails.
If the timestamp has a later time it fails too... so if i delete the secret, which gets recreated automatically, it has a newer date and fails

Correct me if i'm wrong, but shouldnt it be enough to query the secret name from the service account and use that ?
And the Secret also has annotations referencing the serviceaccount it belongs to

metadata:
  annotations:
    kubernetes.io/service-account.name: service
    kubernetes.io/service-account.uid: 7016a8d2-95dd-11ea-8c96-005056a1fd37

If i add another token to the serviceaccount, there is only an outgoing reference to the serviceaccount...

Any info welcome ;)

bug needs investigation
Source
picture erSitzt
👍10

Most helpful comment

Workaround to import an existing ServiceAccount:
(note: YMMV, manual state surgery is not a good idea, use with care, always have a backup)

  1. Add configuration for a kubernetes_service_account in Terraform, but use a different name in the metadata (eg. your-secret-name-2)
  2. Apply the configuration, creating the dummy ServiceAccount
  3. Manually delete the dummy ServiceAccount, eg. using kubectl delete sa your-secret-name-2
  4. Get the default secret name from your _real_ ServiceAccount by running eg. kubectl describe sa your-secret-name, noting down the value for Tokens: something like your-secret-name-abc12
  5. Run terraform state pull > temp.state and open temp.state in your code editor
  6. Look for each instance of your-secret-name-2 and replace it with your-secret-name (there should be about 4 - in the default_secret_name, id, name and self_link)
  7. In the same resource, change the value of default_secret_name to match that you noted down above (eg. your-secret-name-abc12)
  8. Near the very top of your state file, increment the numerical value for serial
  9. Save the file, then run terraform state push temp.state
  10. If you haven't already, change the name of your kubernetes_service_account back to the name it's meant to be (i.e. your-secret-name rather than your-secret-name-2
  11. Run terraform plan - if everything's gone well Terraform should now see your pre-existing ServiceAccount and report no changes to it
picture tdmalone on 1 Aug 2020
👍2

All 2 comments

Workaround to import an existing ServiceAccount:
(note: YMMV, manual state surgery is not a good idea, use with care, always have a backup)

  1. Add configuration for a kubernetes_service_account in Terraform, but use a different name in the metadata (eg. your-secret-name-2)
  2. Apply the configuration, creating the dummy ServiceAccount
  3. Manually delete the dummy ServiceAccount, eg. using kubectl delete sa your-secret-name-2
  4. Get the default secret name from your _real_ ServiceAccount by running eg. kubectl describe sa your-secret-name, noting down the value for Tokens: something like your-secret-name-abc12
  5. Run terraform state pull > temp.state and open temp.state in your code editor
  6. Look for each instance of your-secret-name-2 and replace it with your-secret-name (there should be about 4 - in the default_secret_name, id, name and self_link)
  7. In the same resource, change the value of default_secret_name to match that you noted down above (eg. your-secret-name-abc12)
  8. Near the very top of your state file, increment the numerical value for serial
  9. Save the file, then run terraform state push temp.state
  10. If you haven't already, change the name of your kubernetes_service_account back to the name it's meant to be (i.e. your-secret-name rather than your-secret-name-2
  11. Run terraform plan - if everything's gone well Terraform should now see your pre-existing ServiceAccount and report no changes to it
tdmalone picture tdmalone on 1 Aug 2020
👍2

Adding more evidence that the current import logic seems too fragile.

In my case, the creationTimestamp of the Secret is 7 seconds after that of the Service Account, causing the import to fail with this error:

2021-01-10T07:09:54.893Z [DEBUG] plugin.terraform-provider-kubernetes_v1.13.3_x4: 2021/01/10 07:09:54 [DEBUG] Skipping aws-node-token-zsgw8 as it wasn't created at the same time as the service account
erks picture erks on 10 Jan 2021
Was this page helpful?
0 / 5 - 0 ratings

Related issues

OrangePieiep picture OrangePieiep  路  3Comments
synhershko picture synhershko  路  3Comments
jaceq picture jaceq  路  4Comments
burdiyan picture burdiyan  路  4Comments
yaman picture yaman  路  4Comments