Kind: support for custom CNIs

[BenTheElder]

this could potentially even be the default.

more generally we should look into suporting multiple other CNI
while low-prio, it is something we should _eventually_ be doing, for sure

/priority important-longterm
/kind feature

[neolit123]

FYI the kubeadm survey revealed these results about CNI.

weave is still something i recommend to users, though.

[mauilion]

Related #278

[BenTheElder]

After https://github.com/kubernetes-sigs/kind/pull/331 it will be pretty straightforward to load an alternative CNI with a custom image at least, we still need to do #278 as well which should be pretty straightforward, will put that in the next minor milestone after 0.2 tentatively.

[aojea]

I think that one simple solution to this problem can be adding a new field CNI under the networking struct in the new v1alpha3 API #340
This CNI field should be a string pointing to a remote or local manifest, if this field is empty we continue using current implementation.

apiVersion: kind.sigs.k8s.io/v1alpha3
kind: Cluster
networking:
  CNI: https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

For air-gapped environment, we can deliver manifests for each plugin that uses a local registry #110 avoiding to have to recreate continuously the node-image or to ship the images and load them with kind load

[neolit123]

CNI: https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml`

or make the CNI field accept a mount path too...
something to note is that some plugins have more than one YAML file.

For air-gapped environment, we can deliver manifests for each plugin that uses a local registry #110 avoiding to have to recreate continuously the node-image or to ship the images and load them with kind load

we don't have to ship the images as long as the custom CNI process is document in kind.
also a custom registry seems like an overkill for this.

also, we have to keep in mind that custom CNI falls under advanced testing use cases and making this easy to use from kind shouldn't be a primary goal IMO. e.g. "kind does support custom CNI, but you need to do these steps"..

[aojea]

@neolit123 thanks, the mount path works with the current version using the extraMounts option and overwriting the /kind/manifests/default-cni.yaml file

# this config file contains all config fields with comments
kind: Config
apiVersion: kind.sigs.k8s.io/v1alpha2
# 3 control plane node and 3 workers
nodes:
# the control plane node config
- role: control-plane
  replicas: 1
  extraMounts:
  - containerPath: /kind/manifests/default-cni.yaml
    hostPath: /home/aojeagarcia/go/src/github.com/aojea/kindnet/install-kindnet.yaml
    readOnly: true
    type: File
# the three workers
- role: worker
  replicas: 2

[BenTheElder]

no later than 0.4 I'd like to include some way to not have a default CNI installed so you may choose to install your own. "supported" is a strong word, but "possible without modifying the kind binary" definitely will happen.

[BenTheElder]

this is simple enough now, going to add it to 0.3 so we can let anyone with complaints about kindnetd bypass it entirely.

[BenTheElder]

530 should make it a bit more first class.