External-storage: FAQ: For aws/efs

[shadycuz]

I'm new to kubernetes and was able to set up EFS on my pods very easily, how ever I do have some questions. I will be updating the aws/efs documentation in an upcoming PR and this will be a FAQ at the end. Some of the questions might stem from my lack of knowledge with kubernetes.

configmap.yaml

1. I have witnessed how handy a configmap can be with prometheus, but why here? Couldn't those values have been hard coded into the deployment file? Or does the abstraction serve another purpose I am missing?

deployment.yaml

1. I noticed the AWS EFS gets mounted to the efs-provisioner container directly. What happens when that pod goes down?

2. Why shouldn't we just mount the EFS directly to our apps? Why use the PVC's?

3. Can you run multiple pods for HA? Will that work?

4. What is best practice? Should I use multiple claims on 1 EFS to serve all my pods? Or multiple EFS drives with multiple efs-provisioner deployments/configmaps to server seperate pods?

5. Can I put my files in the root of the EFS?

6. Can I automate making the /persistentvolumes directory some how?

class.yaml

On some guides on the internet using PV's that come out of the box, I noticed some people would create a claim directly to the PV with out a StorageClass.

1. Can you create a claim directly to an EFS PV with out a class?

claim.yaml

I have a claim of 5Mi but I wrote 20Gb's to it.

1. Do claims actually limit the storage the pod can use? or is that not functional with EFS?

[wongma7]

1. No particular reason, tbh. But it's arguably easier to tell users to 'make this configMap' with a single kubectl command than it is to say 'edit these x lines of deployment.yaml'
2. PVs will not be provisioned nor deleted while the provisioner is down. Existing PVs will continue to work though since they're pointing straight at EFS. When the provisioner comes back, it will delete PVs that need to be deleted in Released state, and create PVs for PVCs that are Pending.

3. You can do that, it's just that using PVCs is the general "recommended" way of consuming storage. Typically only admins have permissions to specify volumes directly in pods whereas users should ideally be forced to go through the PVC->PV workflow to be made subject to certain quotas, StorageClasses, etc. So most of the work being done in Kubernetes storage (e.g. mount options which only work when using PVs) assumes people will be familiar with & use PVCs.

   That said, only admins should have permission to create an EFS provisioner instance so it really doesn't matter in this case.

4. Yes, the provisioners won't conflict, they'll race to lock every PVC and only the "leader" will Provision for it. So yes HA is supported in that you can run one provisioner on node A another on node B, both serving the same Class of PVCs, and nothing should break.
5. Really depends, it may help to think about what the provisioner is doing under the hood. All it's doing is occupying a directory in your EFS and making per-PV directories inside of it. So you could have 1 EFS used by multiple efs-provisioner deployments, e.g. /abcd could serve claims that ask for storageclass abcd, /qwer for qwer. You might do this to be able to quota things or logically separate apps or something like that. EFS is supposed to be shared and huge and scalable so I don't think making multiple per-cluster should be needed.
6. Yes
7. No
8. Yes. EFS is just NFS. efs-provisioner exists only to provide automatic creation & deletion of EFS PVs. Which is useful for e.g. StatefulSets. As opposed to you having to manually create x NFS PVs pointing to x different directories & delete them later.
9. No PVCs don't limit anything. It makes sense for an EBS volume to make the EBS PV capacity match the actual disk capacity but for EFS where it's supposed to be scalable there's no notion of capacity

[shadycuz]

Thanks @wongma7 I will have a PR this weekend for you =)

I followed the guide to set up EFS storage for one deployment (prometheus), so for my next 2 deployments (different apps) I can make new claims but with different names, and reference the same storage class? This will create new "PVC" directories on my EFS but all three apps will be on the same EFS system but not be able to see each others files?

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: app3
  annotations:
    volume.beta.kubernetes.io/storage-class: "aws-efs"
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 5Mi

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: app2
  annotations:
    volume.beta.kubernetes.io/storage-class: "aws-efs"
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 5Mi

[wongma7]

Yes, that's correct.

On Thu, Jul 13, 2017 at 5:32 PM, Levi Blaney [e-mail]
wrote:

Thanks @wongma7 https://github.com/wongma7 I will have a PR this
weekend for you =)

I followed the guide to set up EFS storage for one deployment
(prometheus), so for my next 2 deployments (different apps) I can make new
claims but with different names, and reference the same storage class? This
will create new "PVC" directories on my EFS but all three apps will be on
the same EFS system but not be able to see each others files?

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: app3
annotations:
volume.beta.kubernetes.io/storage-class: "aws-efs"
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 5Mi

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: app2
annotations:
volume.beta.kubernetes.io/storage-class: "aws-efs"
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 5Mi

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/kubernetes-incubator/external-storage/issues/241#issuecomment-315208197,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AMgP-CE9tniPDoRcMXSMxrFMnTw7lADlks5sNo0JgaJpZM4OXfis
.

[mjsilva]

re: 7

You can kinda "automate" the create of the folder. In fact you don't need to create the folder. you can just set the path on the pv-volume to / and in the mountPath use /persistentvolumes . Like this:

kind: Deployment
apiVersion: extensions/v1beta1
metadata:
  name: efs-provisioner
  namespace: kube-system
spec:
  replicas: 1
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: efs-provisioner
    spec:
      containers:
        - name: efs-provisioner
          image: quay.io/external_storage/efs-provisioner:v0.1.0
          env:
            - name: PROVISIONER_NAME
              value: "aws-efs"
            - name: FILE_SYSTEM_ID
              valueFrom:
                configMapKeyRef:
                  name: efs-provisioner
                  key: file.system.id
            - name: AWS_REGION
              valueFrom:
                configMapKeyRef:
                  name: efs-provisioner
                  key: aws.region
          volumeMounts:
            - name: pv-volume
              mountPath: /persistentvolumes
      volumes:
        - name: pv-volume
          nfs:
            server: {{ efs_file_system_id }}.efs.{{ aws_region }}.amazonaws.com
            path: /

[bhack]

Can we specify where we can pass mountOptions?

[wongma7]

@bhack try storageclass.mountoptions, may need to update your efs-provisioner image

[wongma7]

@bhack actually mount options is not implemented yet, creating new issue https://github.com/kubernetes-incubator/external-storage/issues/810

[tiendungitd]

Regarding question no.9. I'm using EKS cluster, so eks worker node being separated AZ, using EBS volume for EBS PV is not a good way. Is there any idea to limited storage with efs-provisioner?

[live4live4]

re 6, I've noticed efs-provisioner creates an directory (with pv id as the name) on the root directory of the efs volume. Could you give more details on how to put files on the root directory instead of the sub-directory?