K3s: config.yaml does not work as documented for new installation

[thewilli]

Environmental Info:
K3s Version:
k3s version v1.18.9+k3s1 (630bebf9)

Node(s) CPU architecture, OS, and Version:

Cluster Configuration:
Linux v22019048243487761 4.15.0-106-generic 107-Ubuntu SMP Thu Jun 4 11:27:52 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux (master)

Describe the bug:

Installation options provided by config.yaml are not used for a new installation.

Steps To Reproduce:

• Start with k3s not being installed
• Create file /etc/rancher/k3s/config.yaml user-readable with the following content:

write-kubeconfig-mode: "0644"
node-name: mymaster

• Install k3s by executing $ curl -sfL https://get.k3s.io | sh -

Expected behavior:

• The file /etc/rancher/k3s/k3s.yaml is created with mode 0644
• The node is named mymaster

Actual behavior:

Neither the custom node name, nor the provided mode to /etc/rancher/k3s/k3s.yaml are set

Additional context / logs:

Documentation:

By default, values present in a YAML file located at /etc/rancher/k3s/config.yaml will be used on install.

[ShylajaDevadiga]

@thewilli config.yaml file support was added in k3s v1.19. In k3s v1.18 or before we will have to pass the flags in the command line.

[brandond]

@davidnuzik we should probably update the docs to mention that this is a 1.19 feature.

[thewilli]

thanks for clarifying

[catherineluse]

Closing this issue because the minimum required version was documented.

[thewilli]

@catherineluse imho the documentation should as well contain the information how an agent configuration would look like, i.e. especially how the installation script distinguish between server and agent installation if only the config file was present.

As of now, the latest version installs a server even for a config.yaml with only token and server keys present. This might be a bug, misuse due to missing documentation or of course just a misunderstanding by me :wink:

[Edit]
When looking at the installation script, it seems the config file is ignored for the installation type selection, even though those parameters are regular CLI ones, which could be set using the config file.

# --- if we only have flags discover if command should be server or agent ---
        (-*|"")
            # this might as well be set by config.yaml
            if [ -z "${K3S_URL}" ]; then
                CMD_K3S=server

[catherineluse]

So we should have an example Rancher agent installation command here as well?

[thewilli]

So we should have an example Rancher agent installation command here as well?

@catherineluse yes, either a full example for an agent installation or the installer modified in a way it works as expected (at least I would have expected it), i.e. the installation is of type agent iff the K3S_URL parameter is provided via environment variable or config.yaml

[brandond]

@thewilli a couple things

1. The installer does not manage config.yaml, nor does it inspect its contents to determine if you want a server or agent. Anything you provide via INSTALL_K3S_EXEC or as args to the script itself will still be set via flags in the systemd unit or openrc init script.
2. k3s server or k3s agent are commands, everything after that are flags. The config file only supports setting flags. You cannot set server or agent via config.yaml. @catherineluse it might be worthwhile to call this out specifically.
3. config.yaml is intended to be either an alternative to or alongside setting flags via install.sh. For example, you could pre-populate config.yaml with common options you want set on all your nodes, and then do curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC=server sh - or curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC=agent sh -

[thewilli]

@brandond

[..] nor does it inspect its contents to determine if you want a server or agent

Exactly, and I find this some kind of inconsistent. If INSTALL_K3S_EXEC is not set, it determines whether I wanted to install a server or agent by checking if the K3S_URL aka --server flag. But it does only check the environmental way of providing this flag, not the config file one.

For example, you could pre-populate config.yaml with common options you want set on all your nodes

I do like the config file approach for automated deployments. Having a single command for the actual installation alongside a dynamically generated configuration file makes it less complex and easier to test.

[brandond]

Properly parsing a YAML file is much more complicated than just checking args and environment variables - at this point I think we'll probably just make sure we document that the install script does not examine files on disk, just arguments and environment variables provided to the script itself.