External-storage: Add new standalone cinder provisioner

[aglitke]

The builtin kubernetes cinder support expects that nodes are deployed on nova instances. In order to use cinder as a standalone storage service I'd like to add an external provisioner. This provisioner creates volumes in cinder and retrieves connection information. It then translates this connection information into a native k8s PV (ie. iscsi or rbd are already implemented). Initial working wip patches can be found in my forked repo here: https://github.com/aglitke/external-storage/tree/cinder-provisioner

[aglitke]

@rootfs

[j-griffith]

Might be worth looking at openstack/k8s-cloud-provider that @dims has and maybe we could consolidate some of the efforts if it makes sense and nobody objects.

[aglitke]

@j-griffith sure! I'm open to putting the code wherever it makes the most sense. The external provisioner is a bit orthogonal to the cloud provider. Originally I was using the cloud provider to make my cinder calls but we're also experimenting with alternative configurations for cinder that treat it as a stand-alone storage service. In this case, it makes more sense for the provisioner to consume the gophercloud sdk directly.

[humblec]

@aglitke can you point any design arch doc for this provisioner ?

[aglitke]

@humblec I don't have one yet but it's on my list to create.

[aglitke]

@humblec Please see https://github.com/aglitke/external-storage/blob/cinder-provisioner/openstack/cinder/README.md

[aglitke]

Hi all. I was wondering if others have an opinion about including this provisioner into the the kubernetes-incubator project. @j-griffith do you have any further thoughts on my comments regarding the differences in scope between this provisioner and the existing openstack cloud provider interface?

Ideally, I'd like to merge this and work on it in a public place where others can contribute and improve it.

[rootfs]

@aglitke @j-griffith and @dims :
I'd love to see this work going into external-storage. It is independent of and complementary to other projects for similar purposes. Move importantly, this project enables Kubernetes cluster to consume backend storage directly without having to running on Nova. This is going to help expand the scope where Cinder can orchestrate storage.

[dims]

+1 agree with @rootfs

[rootfs]

@aglitke just to avoid confusion. There is already a Cinder provisioner in k8s tree. Would you rename your provisioner?

[j-griffith]

@aglitke I'd love to see this merged into external-storage and get folks behind it. Agree with comments by @rootfs regarding the nature of the work.

[aglitke]

@rootfs Sure. I will rename it. Any suggestions? Perhaps standalone-cinder would make it clearer.

[rootfs]

@aglitke +1

[wenjun93]

@aglitke hi, is there any conflict with pv_controller inside controller-manager? I mean when user created a pvc, how can I decide whether pv_controller or the standalone provisioner do the provisioning?

[aglitke]

@wenjun1 Great question. Please correct me if I am wrong, but I think this should not be an issue due to storage classes. The standalone provisioner should only handle PVCs that are assigned to a storage class that is annotated with this provisioner name.

[wenjun93]

@aglitke Thanks. I see after provisioning, volume is marked as reserved/attaching at cinder side, why not set the state to in-use? I think set cinder volume to attaching state all the time is weird IMHO.

[aglitke]

@wenjun1 Agreed that "attaching" is a weird state to leave the volume in. The problem is that a provisioned volume in this case is not yet attached to any host. We could call the cinder attach verb with fake values for hostname and mountpoint but that also seems weird. The way the provisioner creates PVs means that it will not be notified when the volume is attached because kubernetes handles that whole flow internally.

[wongma7]

done