Terraform-provider-kubernetes: kubernetes_secret doesn't support .dockercfg: base64xxxx values

[synhershko]

I'm trying to pass a base64 encoded secret to kubernetes_secret and failing with the following error message (seems like it accepts only a map?)

* kubernetes_secret.docker_pull_secret: Failed to update secret: Secret "gitlab-com" is invalid: data[.dockercfg]: Invalid value: "<secret contents redacted>": invalid character 'b' looking for beginning of value

Terraform bits:

resource "kubernetes_secret" "docker_pull_secret" {
  metadata {
    name = "gitlab-com"
  }

  data {
    # decoded value
    # {"registry.gitlab.com":{"auth":"your-token-found-in ~/.docker/config.json"}}
    ".dockercfg" = "base64xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx="
  }

  type = "kubernetes.io/dockercfg"
}

Reference: https://gitlab.com/gitlab-org/gitlab-ce/issues/40951

[stigok]

Same problem here, trying to configure docker repository credentials with

metadata {
  name = "azurecr"
  namespace = "mynamespace"
}
data {
  ".dockerconfigjson" = "ewog[redacted...]=="
}

Plan fails after timeout with error

* kubernetes_secret.container-repository: Secret "azurecr" is invalid: data[.dockerconfigjson]: Invalid value: "<secret contents redacted>": invalid character 'e' looking for beginning of value

The pod creation will block due to failed authentication, while feedback about the failed secret creation is not output until after the plan has timed out (5 minutes) and failed.

[stigok]

I went with base64 because that was what the Kubernetes documentation said, but a different issue got me onto thinking there shouldn't be any base64 encoded values there at all.

I was actually doing ${base64encode(file("${path.module}/.docker/config.json"))}, but now getting the unencoded contents of the file, it's successful:

resource "kubernetes_secret" "container-repository" {
  metadata {
    name = "azurecr"
    namespace = "${var.namespace}"
  }

  data {
    ".dockerconfigjson" = "${file("${path.module}/.docker/config.json")}"
  }

  type = "kubernetes.io/dockerconfigjson"
}

I don't have a lot of experience with neither Kubernetes nor Terraform, so it was confusing to me that the values there was expected to be a JSON string, and not a normal string. Most certainly connected to https://github.com/hashicorp/terraform/issues/16332

[appilon]

With some debugging I think I've proved this error message is actually coming from Kubernetes and not this provider or upstream (terraform).

2018-04-11T15:20:21.263-0400 [DEBUG] plugin.terraform-provider-kubernetes: 2018/04/11 15:20:21 [INFO] Creating new secret: v1.Secret{TypeMeta:v1.TypeMeta{Kind.....

The error returns from creating the secret (from Kubernetes). I'm currently digging through kubernetes source, but with type dockercfg and dockerconfigjson (especially the latter) I don't think you are supposed to be providing base64 encoded values.

@synhershko can you link me to where Kubernetes instructs you to base64 encode the docker config? I've only seen docs saying to base64 encode Opaque secrets

UPDATE:
Here are tests case from Kubernetes:
https://github.com/kubernetes/kubernetes/blob/master/pkg/apis/core/validation/validation_test.go#L11526

You can clearly see from both types dockercfg and dockerconfigjson the valid test data is just a json payload.

[stigok]

The Secrets documentation says to encode it in the JSON/YAML file when you create it: https://kubernetes.io/docs/concepts/containers/images/#bypassing-kubectl-create-secrets

[appilon]

Hmm those docs only exist for version 1.8+ I will double down on the source to see how it knows when its encoded or not, I ran this against my GKE cluster version 1.8.8 however if the error came from a validator (which comes from the k8s client) then maybe we just need to bump up the version of the k8s client in the provider codebase

[sl1pm4t]

Hey @stigok @appilon @synhershko you might find this info useful:
https://github.com/terraform-providers/terraform-provider-kubernetes/issues/81#issuecomment-346219210

[appilon]

UPDATE:
Here is the validation on the master branch (granted kubernetes source structure is super confusing so maybe this isn't the right spot?!)

https://github.com/kubernetes/kubernetes/blob/master/pkg/apis/core/validation/validation.go#L4387-L4408

I just don't see how docker config secrets can be anything other than just json.

[stigok]

I agree. I was unsuccessful myself in trying to find any base64 decode related to dockerconfig. May just be invalid docs, as you hinted.

[appilon]

So it appears its the docs that are misleading, you would base64 encode the contents of the dockercfg when embedding them into a yaml for kubernetes. I will be closing this issue, thanks to all the input throughout.