Terraform-provider-kubernetes: kubernetes provider 2.0.0 during terraform plan try to load kubeconfig file and fail as it doesn't exit yet

[nsvijay04b1]

Error: could not open kubeconfig "my-kubeconfig-file":stat my-kubeconfig-file no such file or directory

Terraform Version, Provider Version and Kubernetes Version

Terraform version: v0.13.5
Kubernetes provider version: 2.0.0
Kubernetes version: k1.18

Affected Resource(s)

• Any kubernetes resource and because terraform exits with error nothing will be created

Failing Terraform Configuration Files - using k8s provider 2.0.0

terraform {
  required_providers {
  kubernetes = "2.0.0"
 }
}

provider "kubernetes" {
  config_path = "~/.kube/my-kubeconfig-file"
}

Working Terraform Configuration Files - using k8s provider 1.13.3

terraform {
  required_providers {
  kubernetes = "1.13.3"
 }
}

provider "kubernetes" {
  load_config_file = "false"
  config_path = "~/.kube/my-kubeconfig-file"
}

Debug Output

Error: could not open kubeconfig "my-kubeconfig-file":stat my-kubeconfig-file no such file or directory

Steps to Reproduce

1. provider.tf as given above in "Failing Terraform Configuration Files - using k8s provider 2.0.0" section
2. terraform plan

Expected Behavior

Terraform plan should not verify if the file exists

Actual Behavior

Terraform failing in "plan" searching for the file which should only exists in apply stage and cannot progress with the deployment

Important Factoids

~/.kube/my-kubeconfig-file is created by dependent module before this problematic module and hence we would not have an issue during apply. During plan this file wont be available.

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

[alexsomesan]

Hi! This indeed seems odd, that you get that error at plan time.
Does this happen when you plan against existing state or when planning with no prior state? Can you confirm the behaviour in both cases?

[nsteinmetz]

Hi there,

Same here, starting from scratch.

I used to have:

terraform {
  required_version = "~> 0.14"
  required_providers {
    kubernetes = {
      source  = "hashicorp/kubernetes"
      version = "2.0.2"
    }
    google     = {
      source  = "hashicorp/google"
      version = "3.55.0"
    }
    [...]
}

[...]
GKE Cluster Creation
[...]

resource "local_file" "kubeconfig" {
  content  = templatefile("${path.module}/templates/kubeconfig-template.yml", {
    cluster_name           = google_container_cluster.default.name
    user_name              = data.google_client_openid_userinfo.provider_identity.email
    token                  = data.google_client_config.current.access_token
    endpoint               = google_container_cluster.default.endpoint
    cluster_ca_certificate = google_container_cluster.default.master_auth[0].cluster_ca_certificate
  })
  filename = "${path.root}/kubeconfig"
}

provider "kubernetes" {
  host                   = google_container_cluster.default.endpoint
  token                  = data.google_client_config.current.access_token
  cluster_ca_certificate = base64decode(google_container_cluster.default.master_auth[0].cluster_ca_certificate)
  config_path            = local_file.kubeconfig.filename
}

If I just do a touch kubeconfig, then plan will work as expected...

[nsvijay04b1]

Hi! This indeed seems odd, that you get that error at plan time.
Does this happen when you plan against existing state or when planning with no prior state? Can you confirm the behaviour in both cases?

I have tested both with existing state and by deleting terraform.tfstate also. same result.

working syntax

terraform {
  required_providers {
    kubernetes = "1.13.3"
  }
}

provider "kubernetes" {
     config_path = "./my-kubeconfig-file"
     load_config_file = "false"
}

failed syntax

terraform {
  required_providers {
    kubernetes = "2.0.0"
  }
}

provider "kubernetes" {
     config_path = "./my-kubeconfig-file"
}

Error: could not open kubeconfig "./my-kubeconfig-file": CreateFile ./my-kubeconfig-file: The system cannot find the file specified.

[stepan111]

hello I have similar error but versions slightly different. I am retrieving SA credentials from vault and then substitute them to k8s provider:

provider "kubernetes" {
  host                   = data.vault_generic_secret.k8s_auth.data.HOST
  cluster_ca_certificate = data.vault_generic_secret.k8s_auth.data.CLUSTER_CA_CERTIFICATE
  token                  = data.vault_generic_secret.k8s_auth.data.k8s_sa_token
}

I don't see any issue in version 2.0.1 but in version 2.0.2 I am getting .kube/config: no such file or directory

P.S. There are no .kube/config at all in this environment. My goal is to keep those creds in memory (and terraform state, but this is another issue)

[watsonjm]

I'm having the same issue as well. This does not happen when there is no state file, but any time afterwards it will complain about not having a file. The Terraform code is ran through a CI/CD pipeline, and so the kubeconfig file does not live on.

[dak1n1]

I tested this with Phil's Progressive Apply repo. We would expect a failure if the provider were configured using a hard-coded filename, however, I'm seeing this also fail with the resource that references the file. This is definitely a regression.

[nsvijay04b1]

Thanks for fixing the issue. Any idea in which version we can expect the change? thank you.

[dak1n1]

It's released in version 2.0.3: https://github.com/hashicorp/terraform-provider-kubernetes/releases/tag/v2.0.3

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 [email protected]. Thanks!