Kind: Define "unkind"

[fabriziopandini]

As discussed in kind meeting the 29th of January (see https://github.com/kubernetes-sigs/kind/issues/252 for @neolit123 notes), it is necessary to define how to extend kind for supporting additional use cases like e.g. "use kind for upgrade tests" or "use kind for kubeadm development".

Even if details about the possible solutions for the aforementioned use cases are not yet defined, there is a general agreement that the only a "core" part of this should be implemented in the kind main binary, while the "not-core" part should be another go binary that reuses/extends the kind.

The bar of what is "core" and what is "non-core" for kind is not defined yet, but at first following question could be addressed:

Where this binary should live?
a) in the kind repo (my preferred option)
b) in the kubeadm repo
c) in another repository (please, don't...)

What will be the name of this binary 😜?
a) unkind
b) kindest (from kind extensions, my preferred option despite unkind proposal was brilliant)
c) ...

[BenTheElder]

[still thinking about this as well, small notes though]
for the binary:
a) and b) are both reasonable, though b) is a stronger forcing function for making the library good

for the binary name:
kindest is probably confusing since that's the image registry https://hub.docker.com/u/kindest/
so maybe unkind it is 😉 @munnerz I think it was?

[pablochacin]

Even if details about the possible solutions for the aforementioned use cases are not yet defined, there is a general agreement that the only a "core" part of this should be implemented in the kind main binary, while the "not-core" part should be another go binary that reuses/extends the kind.

I'm confused about this statement. I was expecting that non-core o more accurately "use case specific" code should live out-of tree in the project that brings this requirement (For example in kubeadm) or as individual projects.

What I think is important within kind project is to provide the capability to use kind as a library and be able to customize its behavior by means of custom actions. Alternatively, defining a plugin mechanisms that allows those extensions to be dynamically loaded could also be an option.

I think that providing an extended kind binary is non sustainable on the long term.

[inercia]

I was wondering wether it is really necessary to have some external binaries. Do we really need that level of customization? Because interfacing with those external tools would bring some complexity to the acrhitecture...

I think most users would have enough with some mechanism for 1) running some shell commands in containers 2) copying stuff to/from containers. These things could either a) ovewrite currents tasks or b) be run as some hooks at some points in time.

[neolit123]

I think most users would have enough with some mechanism for 1) running some shell commands in containers 2) copying stuff to/from containers. These things could either a) ovewrite currents tasks or b) be run as some hooks at some points in time.

absolutely, this is what i proposed in https://github.com/kubernetes-sigs/kind/issues/252
if we think about nodes over the internet one just needs ssh and scp. for kind we need the same over containers.

[BenTheElder]

if we think about nodes over the internet one just needs ssh and scp. for kind we need the same over containers.

docker cp $NODE_NAME, docker exec $NODE_NAME

[BenTheElder]

I'm confused about this statement. I was expecting that non-core o more accurately "use case specific" code should live out-of tree in the project that brings this requirement (For example in kubeadm) or as individual projects.

Ideally yes, in this case we it's odd.

So for _kubernetes_ the project, it makes sense to do EG kubeadm upgrade testing. We need a place for that code. But it will likely not be generally supported for kind users to upgrade their nodes. The point of ""unkind"" is to have kubeadm-project CI glue that we _do not_ support for re-use. Which is why it should not live in the main CLI at all.

I was wondering wether it is really necessary to have some external binaries. Do we really need that level of customization? Because interfacing with those external tools would bring some complexity to the acrhitecture...

usage as a library is a supported feature. the cluster-api already does this for example. this tool just might not happen to live in another repo for :reasons:

[BenTheElder]

To further clarify: the idea is that we can expose things like running custom logic against the cluster, or other APIs that are difficult to support in the main CLI via the library.

We will then want to leverage some of these specifically to test kubeadm itself. """unkind""" would be an example of being extensible via the library.

This means we can eg: easily have third party users run their custom commands via library calls, without trying to shoe-horn this into the config or flags for kind. The primary CLI will stay simple and maintainable, the library will be a bit more powerful.

[neolit123]

whatever this ends up being, i'm not fond of the name unkind. :)

[BenTheElder]

yeah, kinder might be a better name (kind cli , kinder extras cli, kindest image registry / image tags)

[BenTheElder]

So far the primary use case we've agreed to for this is #256. Let's solve that one instead, and better document what the project scope is in our docs as opposed to just the initial scope ... (working on that).