Microk8s: Feature request: allow a way to configure microk8s components with configuration file.

[balchua]

This feature request is to allow users to have ways to configure microk8s cluster components such as apiserver, kube proxy, controller, registry, etc... without resorting to modifying multiple files in /var/snap/microk8s/current file system.

Users should be able to configure microk8s like this microk8s.configure -f config.yaml

This can help administrators or users to easily share configs among their teams and potentially version them too.

What do you think?

List of configurable components.

• [ ] containerd
• [ ] kube api server
• [ ] kube controller
• [ ] kube proxy
• [ ] kubelet
• [ ] csr

[ktsakalozos]

This comes up very frequently (eg https://github.com/ubuntu/microk8s/issues/43 and https://github.com/ubuntu/microk8s/issues/92). It will be great to be able to have a microk8s.init (following the lxd snap paradigm) that would configure MicroK8s.

• microk8s.init (or .setup) would be interactive and ask for some basic configuration options.
• microk8s.init -f config.yaml (or .setup) would use the config file provided
• microk8s.init --auto (or .setup) would reset to the default values
• microk8s.init set <property> <value> (or .setup) could set arbitrary configuration values that could be consumed by the addons, eg microk8s.init set dns.upstream 1.2.3.4 to set the upstream nameservers on dns.

Implementation-wise I was thinking we could store the configuration yaml in $SNAP_DATA and offer a few utility functions to be used by the rest of MicroK8s. It would be up to the wrapper of each service (or addon script) to read that configuration and apply the subset of the configuration values that are relevant to what is wrapped.

[balchua]

@ktsakalozos to be honest, i only thought about mostly cluster configuration. 😁. Im just wary of addons names clashing with other addons.

[ktsakalozos]

We do not need to do everything on the first try, but it is nice to see how we can extend our work.

[balchua]

Sure thing. Let me try to update the issue with list of components that we intend to include. By the way, are we open to implementing this using go or other programming languages?

[ktsakalozos]

are we open to implementing this using go or other programming languages?

This is a good question. As you have seen the MicroK8s deployment you get in your system is a thin layer of bash on top of the K8s upstream binaries. The reason for this is that

• bash keeps the size of the snap package low (as it does not require any runtime)
• presents a low entry barrier for external contributions

For sure larger components that we will eventually need to code will have to be in a programming language (bash is great but you can easily produce code/scripts hard to maintain). The rest of the team here is into python. So python is a strong candidate language. The problem I see with python is that it has a runtime that I am not sure how large it is when it is put inside the snap. I need to do some research.

The option of go also sounds reasonable. It comes naturally since K8s is in go and we have no shortage of go devs in Canonical/Juju. I need to talk with the rest of the team to see if we are comfortable to maintain some go components.

[balchua]

I fully understood the rationale behind using bash. It's super lightweight. That's mostly the reason why i asked the question. 😁.

[balchua]

Just thinking out loud here. This is how i imagine the configuration file.

apiVersion: microk8s.io/v1alpha1
kind: Configuration
spec:
  containerd:
    registry:
      - host: localhost
        insecure: true
      - host: myregistry.org
        insecure: false
  apiServer:
    extraArgs:
      insecure-bind-address=127.0.0.1
      cert-dir=$SNAP_DATA/certs
      etcd-servers='unix://etcd.socket:2379'
      service-cluster-ip-range=10.152.183.0/24
  controller:
      extraArgs:
. . . .

These configurations will then be translated to corresponding components' config format. Perhaps this is a little verbose.

[balchua]

@ktsakalozos i was checking out the clustering branch. I noticed that there are some similarities on what the agent do wrt to configuring the cluster. Is it something that can be extended to standalone microk8s?

Thanks!

[ktsakalozos]

Indeed @balchua the cluster-agent could be used and extended to configure running k8s services. What exactly you have in mind?

[balchua]

I prefer having the same config for both clustered and standalone.

[ktsakalozos]

Yes, that makes sense. When your have a MicroK8s cluster you would expect the user activity to propagate to the nodes (see for example the dns addon). The cluster-agent on the nodes takes care of the service configuration and restarts. It can even fire scripts that come in the form of addons.

We could leverage this for the wide configuration.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.