Kops: Add support for custom configs passed to --runtime-config

[ls-jftheroux]

I need to enable NetworkPolicy right now. My understanding is that I need to pass some options to --runtime-config for kube-apiserver to do so.

It's currently not possible with kops right now.

Having the ability to pass custom configs to k8s components would be great. As right now, although kops is awesome, it's a bit limiting in some cases.

An example of what I'm looking for:

--runtime-config=extensions/v1beta1/networkpolicies=true

[pacuna]

also interested in this. I'm trying to test the new ScheduleJob type and I need to pass options via runtime-config to the api-server

[pacuna]

@justinsb thanks! So, this options should be passed when running the create cluster command?

[justinsb]

For now you need to edit the cluster spec post-creation. We can probably make this better, but this is "step one".

So you would do

kops create cluster; kops edit cluster; kops update cluster

In the edit, you will add a section under the main spec that looks like

  kubelet:
    kubeAPIServer:
      runtimeConfig:
        "extensions/v1beta1/networkpolicies": "true"

But... you may want to let me test this properly first! It's implemented so I automatically closed the ticket, but it isn't fully tested.

[pacuna]

@justinsb thanks. I actually tried that and I wasn't able to turn the config I needed. One thing I notice is that the kubeApiServer should be outside the kubelet section right? Or else it gets deleted

[shrugs]

Just to confirm to any other people that stumble into this via google, the config does need to be at the root of the spec like so:

spec:
  kubeAPIServer:
    runtimeConfig:
      "group/api/resource": "true"

[chrislovecnm]

@Shrugs if you are able to help to update the docs, that would be amazing

[shrugs]

@chrislovecnm Good idea, I'll plan on it.

[alex88]

I've added

spec:
  kubeAPIServer:
    runtimeConfig:
      batch/v2alpha1: "true"

ran kops update cluster --yes but nothing changed, kops rolling-update cluster and kops upgrade cluster doesn't show any pending changes

[shrugs]

@alex88 try forcing the rolling update with --force?

[alex88]

@Shrugs yeah that did the trick, thanks

[shrugs]

I believe this issue is correctly solved now; @ls-jftheroux can you confirm and close if necessary?

[ls-jftheroux]

I'm gonna go ahead and close it as I don't use kops anymore.

[rrichardson]

Is there an issue open for the update not running withouth --force?

The problem is that --force destroys and re-creates all of the nodes as well as the masters.

This is something I would prefer not to do on our production system if I can help it. I would prefer that it detect the kubeAPIServer config change as something that requires only master restarts.

[jnicholls]

I second rrichardson

[nagas]

what @rrichardson said

[sellers]

@rrichardson you can apply to only the master(s) in question with the --instance-group master-us-.... flag passed to the rolling-update directive. (where ... is the remaining part of your master node Instance Group)