Rke: No default StorageClass after deploying a k8s cluster using RKE

[zedtux]

RKE version:

$ rke --version
rke version v0.1.6 (Darwin)

Docker version: (docker version,docker info preferred)

# docker info
Containers: 24
 Running: 17
 Paused: 0
 Stopped: 7
Images: 16
Server Version: 17.03.2-ce
Storage Driver: overlay2
 Backing Filesystem: extfs
 Supports d_type: true
 Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host macvlan null overlay
 Log:
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 4ab9917febca54791c5f071a9d1f404867857fcc
runc version: 54296cf40ad8143b62dbcaa1d90e520a2136ddfe
init version: 949e6fa
Security Options:
 seccomp
  Profile: default
Kernel Version: 4.9.0-6-amd64
Operating System: Debian GNU/Linux 9 (stretch)
OSType: linux
Architecture: x86_64
CPUs: 1
Total Memory: 1.911GiB
Name: deadlpool
ID: 4L3X:3OO4:IVXL:Q6JX:2GIX:C7CK:MPGM:23IF:XZFU:7HB3:C7XH:SFGY
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Labels:
 provider=generic
Experimental: false
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false

WARNING: No swap limit support

Operating system and kernel: (cat /etc/os-release, uname -r preferred)

# uname -r
4.9.0-6-amd64

Type/provider of hosts: (VirtualBox/Bare-metal/AWS/GCE/DO)

VPS

cluster.yml file:

# default k8s version: v1.8.10-rancher1-1
# default network plugin: canal
nodes:
  - address: XX.XX.XXX.XXX
    port: xxxxx
    ssh_key_path: '~/.ssh/gitlab_ci_id_rsa'
    user: root
    role: [controlplane,etcd]
  - address: YYY.YYY.YY.YY
    port: xxxxx
    ssh_key_path: '~/.ssh/gitlab_ci_worker_id_rsa'
    user: root
    role: [worker]
ingress:
  provider: none

Steps to Reproduce:

1. Install the Kubernetes cluster with RKE
2. List existing StorageClass with kubectl get storageclass

Results:

$ kubectl get storageclass
No resources found.

The result of this is when I'm deploying anything using helm, where a PVC is required, it remains in an error state with pod has unbound PersistentVolumeClaims (repeated 2 times).

[sebastien-prudhomme]

You need to setup a storage provider.

If you don't use a cloud provider, you can try Longhorn. Here is an Helm chart: https://github.com/rancher/charts/tree/master/proposed/longhorn/v0.1.0

There is an option to use it as the default storage class.

[zedtux]

Thank you @sebastien-prudhomme for this suggestion. I'm looking around and found https://www.minio.io/kubernetes.html out of many options.

Why are you recommending Longhorn ?

[zedtux]

BTW my issue is still that I have 0 storage classes which blocks, apparently, any PVC requests which shouldn't be.

2 options for me :

1. A default storage class is installed with RKE (needs to define the one that would be very easy, free and so on) (I.e: passing the --enable-admission-plugins flag to the kube-apiserver with the DefaultStorageClass value, see https://kubernetes.io/docs/reference/command-line-tools-reference/kube-apiserver/).
2. Update the README.md with some details about how to configure it (like redirecting to the Kubernetes doc)

[sebastien-prudhomme]

Longhorn is easy to install (see the documentation) and is developped by Rancher.

You can also try Rook (based on Ceph) as another "in Kubernetes cluster" option.

There is no default as it really depends on your working environment (on AWS ? on Azure? on private datacenter ?)

[zedtux]

Longhorn is easy to install (see the documentation) and is developped by Rancher.

Okay, thank 👍

There is no default as it really depends on your working environment (on AWS ? on Azure? on private datacenter ?)

Then please update the README.md file in order to explain this and give the howto for all the cases :)

New comers will love it.

[galal-hussein]

@zedtux Storageclasses depends on the cloud provider you choose and several other options, its not something that should be installed by default, closing the issue now, please let me know if you have any more questions

[zedtux]

Storageclasses depends on the cloud provider you choose

@galal-hussein I'm not using a cloud provider, I'm deploying RKE on my own servers.

and several other options.

Well that a kind of vague. What other options? Do you have a link describing those options?

its not something that should be installed by default

Depending on the case, I disagree with you. The noob guy using RKE, which is an extremely simple, lightning fast Kubernetes installer that works everywhere, expects to have something fully working, and not spending hours of searches on the Internet to understand what is storage in k8s, what are the available options, and how does they work and so on.

closing the issue now

You're closing an unresolved issue. Your comment isn't helping me, and isn't providing a solution or a workaround to my issue. So what shall I do from here?

---

Basically I stick on RKE should provide a default storage class so that RKE remains an extremely simple Kubernetes deployment tool.
Or in the worst case, should be an example YAML file, showing how to configure a storage class, so that non expert like I was could go lightning fast with their first experience with Kubernetes.

If you take the full example YAML file, you see there are plenty of options to configure many parts of k8s, which have all default values.
I'm expecting an option about storage classes, with a default value, no matter what it is.

Frankly I'm really convinced my issue is highlighting a gap that needs to be addressed.

Now reading again my issue, I have to admin it isn't clear enough, but then ask for more information please instead of closing my issue.

[sebastien-prudhomme]

@zedtux If you want a first experience with Kubernetes, just use Minikube. It has a default storage class.

I'm not associated with Rancher, so i guess i'm neutral, but as i've said before and as @galal-hussein also said, storage configuration really depends on your environment.

If you start installing Kubernetes on your servers, it means you may want a distributed storage solution and there are plenty of them: Longhorn, Rook, OpenEBS, etc

Or perhaps you have an external storage (SAN or a NAS for instance) that you want to use?

As you see it's not easy to provide a default solution.

[zedtux]

And I'm fine with that, I really understand it. BUT RKE has no configuration option for the storage classes.

In my eyes, it should be then a wiki page about it, explaining different cases, covering the one server case, and many servers case too, with an exhaustive list of possible solutions, and how to configure RKE for one or another option.

But today, there's nothing, so RKE is laking a configuration option for a part of Kubernetes.

P.S: And the main point in my previous comment was about closing an unresolved issue.

[galal-hussein]

@zedtux as described in the kubernetes documentation for storage classes, it depends on the provisioner which can be one of nearly 20 options including AWS, Azure, GCE, Local, etc.

for example for AWS environment, it would be something like that:

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: slow
provisioner: kubernetes.io/aws-ebs
parameters:
  type: io1
  zones: us-east-1d, us-east-1c
  iopsPerGB: "10"

even with aws, you still need to define the storage class options which includes the type, zones, iopsPerGB, so not only storage class depends on the environment but also it has multiple variation for the same environment. you can find more information about storage classes in the official docs for kubernetes where every environment is explained and has its own storage class config example, so re-explaining storage classes in the README seems redundant.

As for closing the issue, i only closed the issue because it seems that @sebastien-prudhomme answers to your question were enough but i also explained that please feel free to let me know if you still have more questions regarding the issue.

[zedtux]

Thank you @galal-hussein for your answer. So no plan be able to configure the storage class from RKE, I mean from the cluster.yml file?

[galal-hussein]

@zedtux we enable an addon feature which can be used to create some resources right after creating the cluster for example you can create AWS storageclass in the yaml file like that:

addons: |-
  ---
  kind: StorageClass
  apiVersion: storage.k8s.io/v1
  metadata:
    name: slow
  provisioner: kubernetes.io/aws-ebs
  parameters:
    type: io1
    zones: us-east-1d, us-east-1c
    iopsPerGB: "10"

for more information about addons please refer to RKE documentation.

[zedtux]

Ah ! Then that's my solution 👍.

Okay so maybe a short explanation (like you just did) in the document could help new comers.

Thank you @galal-hussein, that's what I was looking for. Now the issue can be closed 😉