Terraform-provider-kubernetes: Cannot create volume claim against NFS volume due to storageClassName

[brockpalen]

Terraform Version

V0.11.1

Affected Resource(s)

Please list the resources as a list, for example:

• kubernetes_persistent_volume_claim

Terraform Configuration Files

## NFS Server Service
resource "kubernetes_service" "jupyter-nfs" {
  metadata {
    name = "jupyter-nfs"
  }

  spec {
    selector {
      app = "${kubernetes_pod.jupyter-nfs.metadata.0.labels.app}"
    }

    port {
      name = "nfs"
      port = 2049
    }

    port {
      name = "mountd"
      port = 20048
    }

    port {
      name = "rpcbind"
      port = 111
    }
  }
}

## NFS Server as persistent volume
resource "kubernetes_persistent_volume" "nfs-volume" {
  metadata {
    name = "nfs-volume"
  }
  spec {
    capacity {
      storage = "5Gi"
    }
    access_modes = ["ReadWriteMany"]
    persistent_volume_source {
      nfs {
        path   = "/exports"
        server = "${kubernetes_service.jupyter-nfs.metadata.0.name}"
      }
    }
  }
}

resource "kubernetes_persistent_volume_claim" "nfs-volume" {
  metadata {
    name = "nfs-volume"
  }
  spec {
    access_modes = ["ReadWriteMany"]
    storage_class_name = ""
    resources {
      requests {
        storage = "5Gi"
      }
    }
  }
}

Expected Behavior

Should be able to create a PVC the same as on
https://github.com/kubernetes/examples/blob/master/staging/volumes/nfs/nfs-pvc.yaml

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: nfs
spec:
  accessModes:
    - ReadWriteMany
  storageClassName: ""
  resources:
    requests:
      storage: 1Mi

Actual Behavior

Cannot pass empty string for storage_class_name as noted here: http://blog.kubernetes.io/2017/03/dynamic-provisioning-and-storage-classes-kubernetes.html

Instead Storage class is set to standard (GCE)

PVC is created an in Pending state forever, but if using kubectl with the above YAML is created just fine.

Steps to Reproduce

1. terraform apply

Important Factoids

Ran on GCE using GKE

[radeksimko]

Hi @brockpalen
thanks for opening the issue.

It looks like it's a duplicate of https://github.com/terraform-providers/terraform-provider-kubernetes/issues/95 - do you mind me closing this issue in favour of #95 ?

[brockpalen]

I think this might be rooted in the same problem, but that issue is for storage_class_name not being recognized, and that works fine for me on the current version of terraform.

So they feel like different things to verify/have test case for,

eg:

resource "kubernetes_storage_class" "example" {
  metadata {
    name = "terraform-example"
  }

  storage_provisioner = "kubernetes.io/gce-pd"

  parameters {
    type = "pd-standard"
  }
}

resource "kubernetes_persistent_volume_claim" "jupyterhub-storage-claim" {
  metadata {
    name = "jupyterhub-storage-claim"
  }

  spec {
    access_modes       = ["ReadWriteOnce"]
    storage_class_name = "${kubernetes_storage_class.example.metadata.0.name}"

    resources {
      requests {
        storage = "5Gi"
      }
    }
  }
}

Works just fine, which to me (I'm new to terraform and kubernetes) looks like issue #95 works fine now.

[jeffmhastings]

You can work around this limitation by using labels on the persistent volume and a selector on the persistent volume claim:

resource "kubernetes_persistent_volume" "nfs-volume" {
  metadata {
    name = "nfs-volume"

    labels {
        dedicated_volume = "jupyter_nfs"
    }
  }

  spec {
    capacity {
      storage = "5Gi"
    }

    access_modes = ["ReadWriteMany"]

    persistent_volume_source {
      nfs {
        path   = "/exports"
        server = "${kubernetes_service.jupyter-nfs.metadata.0.name}"
      }
    }
  }
}

resource "kubernetes_persistent_volume_claim" "nfs-volume" {
  metadata {
    name = "nfs-volume"
  }
  spec {
    access_modes = ["ReadWriteMany"]
    storage_class_name = ""

    selector {
      match_labels {
        dedicated_volume = "jupyter_nfs"
      }
    }

    resources {
      requests {
        storage = "5Gi"
      }
    }
  }
}

And I think they've add storage_class_name to persistent volume resources now so you could go that route as well.

[SwitchedToGitlab]

Bumping this... issue is well over a year old and still unresolved. This is a show-stopper.

[tdmalone]

@SwitchedToGitlab The Hashicorp teams tend to prioritise issues based on thumbs up reactions to the issue - it’s probably worth adding yours to give this issue a little higher priority than it would already have.

[davidq2q]

Bump, we should be able to do things the kubernetes-way without work-arounds. Besides for terraform v0.12 the labeling trick from @jeffmhastings is not working.

[ikbenben]

I think this can be closed. I was facing the same issue and was able to resolve it based on the comments above by specifying my own custom storage class name when creating the persistent volume and persistent volume claim

[symbiont-jon-bogaty]

Not only is this still an issue but as has been said as of latest 0.12 labeling does not resolve the problem. Additionally Google's documentation for filestore recommends a blank storage class name in the official documentation for mapping filestore shares to K8s so there's now a frustrating disconnect in the main way to provision external NFS for GKE.

[dak1n1]

Hi, everyone! Thanks for your contributions to this bug report. I've been testing this out today, with the intention of getting the related PR merged. But I found that it actually already works with a blank storageclass, unless I'm missing something. Please let me know if you're still unable to provision a PVC with a blank storageclass name. Here's what worked for me:

resource "kubernetes_persistent_volume_claim" "example" {
  metadata {
    name = "mariadb-data"
    namespace = "nginx"
  }
  spec {
    access_modes = ["ReadWriteOnce"]
    storage_class_name = ""
    resources {
      requests = {
        storage = "1Gi"
      }
    }
  }
}

I tried it two different ways: once by omitting the storage_class_name completely, and once by setting it blank explicitly, as seen above. Both ways seemed to provision the PVC successfully.

[dakini@dax issue_102]$ terraform apply --auto-approve
kubernetes_namespace.example: Creating...
kubernetes_config_map.example: Creating...
kubernetes_persistent_volume_claim.example: Creating...
kubernetes_namespace.example: Creation complete after 0s [id=nginx]
kubernetes_config_map.example: Creation complete after 0s [id=nginx/nginx-config]
kubernetes_persistent_volume_claim.example: Creation complete after 0s [id=nginx/mariadb-data]
kubernetes_deployment.mariadb: Creating...
kubernetes_deployment.nginx: Creating...
kubernetes_deployment.mariadb: Creation complete after 4s [id=nginx/mariadb-example]
kubernetes_deployment.nginx: Creation complete after 4s [id=nginx/nginx]
kubernetes_service.example: Creating...
kubernetes_service.example: Creation complete after 0s [id=nginx/nginx]

Apply complete! Resources: 6 added, 0 changed, 0 destroyed.

[dakini@dax issue_102]$ kubectl get pvc -n nginx
NAME           STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
mariadb-data   Bound    pvc-515ec8cb-d615-4dea-a997-7581caa61c21   1Gi        RWO            standard       11s

This is the version I'm on:

[dakini@dax issue_102]$ terraform version
Terraform v0.12.24
+ provider.kubernetes v1.11.1

Please let me know if I've made a mistake or misunderstood the issue. Thanks!

[dak1n1]

It looks like I probably did misunderstand the issue. I'm working on it in this PR, and hoping to get it merged soon, depending on the outcome of my testing. https://github.com/terraform-providers/terraform-provider-kubernetes/pull/590

[dak1n1]

Hi, we have a few issues relating to this that are being consolidated here: https://github.com/terraform-providers/terraform-provider-kubernetes/issues/872 See the new issue for a work-around. Thanks!

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 [email protected]. Thanks!