External-storage: New Release for RBD

[NicolaiSchmid]

Hey,
could we build and push a new release for Ceph RBD?
I'm on Kubernetes 1.9.2 and I'd like to use it for provisioning my PVCs.
Also, are the images still pushed to quay.io/external_storage?
Cheers,
Nicolai

[rootfs]

@childsb @wongma7 can you make a new release image?

[AnatolyRugalev]

I really stuck because of this issue, so I decided to make temporary image in docker hub to use it in my cluster.

https://hub.docker.com/r/anatolyrugalev/rbd-provisioner/

Hope it helps.

[NicolaiSchmid]

Thank you!
I'm currently using your image, but I still get an error mounting my PV to a pod:

Unable to mount volumes for pod "...": 
timeout expired waiting for volumes to attach/mount for pod "...".
list of unattached/unmounted volumes=[volume]

Do you think this is related to the provisioner or some other problem? I have absolutely no idea since the PVC gets satisfied, an image is created and tried to mount. Inside the RBD container, it can connect to Ceph and provision the image without any challenges. There are no error logs inside the provisioner after the timeout and I can't find any other logs regarding this problem.
Any suggestions are really appreciated!

[timn]

I made the experience yesterday that the ClusterRole does not give all the permissions necessary. In particular, the rbd-provisioner needs to resolve the kube-dns service to pass on the IP of the ceph monitor. If that fails it seems to not do some important steps afterwards (check the output of rbd-provisioner, look for a permission denial). You could see this since the volume does not create in the rbd pool at all.

I fixed this by adding the following at the end of https://github.com/kubernetes-incubator/external-storage/blob/master/ceph/rbd/deploy/rbac/clusterrole.yaml (didn't get to file a PR, yet):

  - apiGroups: [""]
    resources: ["services"]
    verbs: ["list", "get"]

Maybe that helps.

(the problem mention before this edit was none, when I looked this morning the pods came up, it was then related to some messing with secrets due to the new userSecretNamespace and using a kube-specific client account)

[timn]

Pushed #620 with more explicit restrictions now. @NicolaiSchmid, you could close your PR.

[klynch]

Any way we could get a release cut?

[timn]

@rootfs, anything you can do to get a release pushed?
@AnatolyRugalev, would you mind to give some advice on building the image. The release Dockerfile doesn't work for me and "make" errors out.

[klynch]

@childsb @wongma7 @rootfs any progress on this?

[rootfs]

ideally i'd like to CI push the images after successful deploy

@childsb @wongma7 have CI access

[Wylan]

@timn If it helps at all I also had trouble building the image I finally succeeded by doing the following:

docker run \
    -it \
    --rm  \
    -e "CGO_ENABLED=0" \
    -v $(pwd)/.go:/go \
    -v $(pwd):/go/src/github.com/kubernetes-incubator/external-storage/ \
    -v $(pwd):/go/bin \
    -v $(pwd)/.go/stdlib:/usr/local/go/pkg/linux_amd64_asdf \
    -w /go/src/github.com/kubernetes-incubator/external-storage/ceph/rbd \
    golang:1.8.3-alpine \
    ash

Inside that container I ran make

Then back on my machine I built the docker image:

docker build -t repo/image . and pushed it

Kind of convoluted! But I wasn't able to get it to build using the Docker.release file for whatever reason.

CI pushing the images would be awesome. 😄

[jamstar]

i would love to see an updated image.

[jamstar]

Poke any update on an offical image?

[jamstar]

For those of you who want to make your own image, i found https://github.com/kubernetes-incubator/external-storage/issues/745#issuecomment-385030482 which has the steps for building your own image. The only thing i changed in the Makefile is the REGISTRY and VERSION to ones of my own choosing.

[wongma7]

I have pushed https://quay.io/repository/external_storage/rbd-provisioner?tab=tags and https://quay.io/repository/external_storage/cephfs-provisioner?tab=tags . Keep in mind these include many months of changes at once. Again, apologies for the delay, having this long a period between releases kind of contradicts the supposed benefits of external provisioners.

Continuing discussion in https://github.com/kubernetes-incubator/external-storage/issues/766