Terraform-provider-kubernetes: PersistentVolumeClaim remains in Pending state

Created on 6 Oct 2017 · 10Comments · Source: hashicorp/terraform-provider-kubernetes

When I try to create PV & PVC in Kubernetes with Terraform targeting a specific existing EBS volume the PVC never gets bound. It looks the PV created using Terraform does not create it in correct manner that it can be successfully bound.

Terraform Version

Terraform v0.10.7

Affected Resource(s)

persistent_volume

Terraform Configuration Files

variable "vol_id" {
  # default = "aws://us-east-1a/vol-1111111111111"
}

provider "kubernetes" {
}

resource "kubernetes_persistent_volume" "pv" {
  metadata {
    name            = "test-volume"
  }
  spec {
    capacity {
      storage       = "1Gi"
    }

    access_modes    = ["ReadWriteOnce"]

    persistent_volume_source {
      aws_elastic_block_store {
        volume_id   = "${var.vol_id}"
        fs_type     = "ext4"
      }
    }
  }
}

resource "kubernetes_persistent_volume_claim" "pvc" {
  metadata {
    name            = "test-volume"
  }
  spec {
    resources {
      requests {
        storage     = "1Gi"
      }
    }
    access_modes    = ["ReadWriteOnce"]
    volume_name     = "test-volume"
    storage_class_name = "gp2"
  }
  wait_until_bound = false
  depends_on = ["kubernetes_persistent_volume.pv"]
}

Debug Output

https://gist.github.com/puco/23487f4d80001f3d44a41a48b53504cf

Expected Behavior

I should have a PV & bound PVC

Actual Behavior

I have a PV in Available state and PVC in Pending state

Steps to Reproduce

terraform apply

Important Factoids

Run of the mill Kubernetes 1.8 cluster w/o RBAC on AWS provisioned with kops. I created the EBS volume manually (as that part works w/o any issues).

What works

When I create the PV/PVC with kubectl directly they work. When I create the PV with terraform and PVC with kubectl it works as well, so there appears to be something wrong with the PV. I compared the definition (yaml) of both PVs (tf and kubectl) and there appears not to be any issues. Originally I assumed that the issue would be creating EBS volume, PV and PVC in one go but the issue still remains with an existing EBS volume.

apiVersion: v1
kind: PersistentVolume
metadata:
  name: test-volume-yaml
spec:
  accessModes:
  - ReadWriteOnce
  awsElasticBlockStore:
    fsType: ext4
    volumeID: aws://us-east-1a/vol-1111111111111
  capacity:
    storage: 1Gi
  persistentVolumeReclaimPolicy: Retain
  storageClassName: gp2
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: test-volume-yaml
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi
  storageClassName: gp2
  volumeName: test-volume-yaml

bug

Source

puco

All 10 comments

I'm facing a similar issue on Azure, but I believe it's related to the storage class name on the persistent volume that terraform creates (the empty string). Because the storage class names don't match the PVC fails to bind to the PV. Could this be your problem (the storage class names don't match)?

There doesn't seem to be a way to specify the storage class name on the PV, while the PVC resource storage class name defaults to "default". When I manually edited the PV storage class name using kubectl from "" -> "default" then the PVC successfully bound to the PV.

sam-at-luther on 20 Oct 2017

👍1

I just hit the same problem on Google, and agree with @sam-at-luther's diagnosis.

ccope on 21 Nov 2017

Agree with @sam-at-luther
I'm trying to use minikube with persistent local volumes, and hitting the same issue, where the volume claim is stuck in a pending state.

There are workarounds and solutions in these issues, but they only work if you can make the changes in kubernetes config, but since terraform doesn't allow you to specify the storage class on the volume, they won't work.

https://github.com/kubernetes/minikube/issues/214
https://github.com/kubernetes/minikube/issues/1783
This also shows my observations:
https://github.com/kubernetes/minikube/issues/1239

dionjwa on 21 Nov 2017

👍1

I have the same issue. But I don't understand, why terraform doesn't allow to specify storage class name on the persistent volume. Is there any reason for that?

plispe on 22 Nov 2017

@plispe I looked at the code, it's just a matter of adding the entry (which I would do if my go coding was up to it). Terraform explicitly maps parameters to the underlying drivers, so it cannot simply pass parameters it doesn't explicitly handle.

dionjwa on 22 Nov 2017

@dionjwa Thank you, I will try to make a pull request with the solution. I wrote already the code, but now I am trying to find how I should test the updated provider.

plispe on 22 Nov 2017

plispe There is already a pull request with a fix: https://github.com/terraform-providers/terraform-provider-kubernetes/pull/72

ccope on 23 Nov 2017

@ccope ok I will wait for merge and release. Thank you.

plispe on 23 Nov 2017

Hi all

any updates on this issue?