K3os: Wireguard with K3OS

[khachatur-s]

Hi guys

It's more a question rather than feature request, but
Is it possible to use Wireguard in k3OS?
There's Wireguard option in k3s networking docs, but for that kernel modules are needed.
I can't figure out how to add this modules in kernel, 'cause in regular distros it's done via package manager (suggested in Wireguard docs) and k3OS has no packager manager included.
Or perhaps I misunderstood something?

Of course it would be nice to have Wireguard out of box but seems that's included in kernel starting from version 5.6

Thanks in advance

[PhilsLab]

Based upon that question, how much time do you think it takes until kernel 5.6 will arrive in k3os?

[nazarewk]

Not sure if this works, but the build passed. Might test it later https://github.com/rancher/k3os-kernel/pull/19

[dweomer]

@nazarewk wrote:

Not sure if this works, but the build passed. Might test it later rancher/k3os-kernel#19

This looks cool. I was hoping to just pick up wireguard from the Focal kernel (5.4.0 series) build I am working on but it ain't building for me yet.

---

@PhilsLab wrote:

Based upon that question, how much time do you think it takes until kernel 5.6 will arrive in k3os?

k3OS ships with the Ubuntu LTS/HWE kernel which will be 5.4.0 from Focal Fossa when it ships later this month.

[dclark]

The k3os kernel patch modified from https://github.com/rancher/k3os-kernel/pull/19 is working with an updated k3os including the wireguard tools.

https://github.com/dclark/k3os/tree/wireguard

Add wireguard to k3os.modules
Set flannel-backend=wireguard

wg shows peering as expected from a multi server k3s setup.

[dclark]

Pull Request https://github.com/rancher/k3os/pull/444 with the wireguard-tools added.

[dweomer]

Waiting on rebase and squash for https://github.com/rancher/k3os-kernel/pull/19

[dclark]

https://github.com/rancher/k3os-kernel/pull/19 has been merged, @dweomer issue can be closed?

[xeor]

Should this work out of the box now with v0.10.1-rc1?

I am getting this error on my agent (formatted for readability):

time="2020-05-10T19:52:17.289987079Z" level=fatal msg="
  flannel exited: failed to run command:
    export SUBNET_IP=$(echo $SUBNET | cut -d'/' -f 1);
    ip link del flannel.1 2>/dev/null; echo $PATH >&2;
    wg-add.sh flannel.1 && wg set flannel.1 listen-port 51820 private-key privatekey &&
    ip addr add $SUBNET_IP/32 dev flannel.1 &&
    ip link set flannel.1 up &&
    ip route add $NETWORK dev flannel.1 Err: exit status 1

Output: /var/lib/rancher/k3s/data/e11077249ce746839166d05992f4e5771f9bcd9ba593a9c269fe77c6a14cb60d/bin:/bin:/sbin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin:/var/lib/rancher/k3s/data/e11077249ce746839166d05992f4e5771f9bcd9ba593a9c269fe77c6a14cb60d/bin/aux

modprobe: module wireguard not found in modules.dep
modprobe: module tun not found in modules.dep

Error: Unknown device type.

/var/lib/rancher/k3s/data/e11077249ce746839166d05992f4e5771f9bcd9ba593a9c269fe77c6a14cb60d/bin/aux/wg-add.sh: line 26: boringtun: command not found

/var/lib/rancher/k3s/data/e11077249ce746839166d05992f4e5771f9bcd9ba593a9c269fe77c6a14cb60d/bin/aux/wg-add.sh: line 29: boringtun: command not found

/var/lib/rancher/k3s/data/e11077249ce746839166d05992f4e5771f9bcd9ba593a9c269fe77c6a14cb60d/bin/aux/wg-add.sh: line 32: wireguard-go: command not found"

I got a config like this (truncated):

k3os:
  modules:
    - wireguard
  k3s_args:
    - agent

and a master running like /usr/local/bin/k3s server --disable=traefik --disable servicelb --node-taint k3s-controlplane=true:NoExecute --flannel-backend wireguard

Master is running ubuntu, not k3os

[dclark]

@xeor v0.10.1-rc1 contains the wireguard tools but not the updated kernel with the module. It's been merged in https://github.com/rancher/k3os-kernel/pull/19 so will be included in future k3os releases.

[xeor]

Thanks for the update @dclark! That means that 0.10.1 still can't work with wireguard? That is something that might be worth mentioned in the release-notes maybe?

What about the tools wireguard-go, and boringtun that i'm getting command not found on. Should they also be included before k3os supports wireguard?

[dclark]

@xeor this is mentioned in the release notes:

"Wireguard Tools #444
These are usable now if you happening to be running an overlay with a wireguard-enabled kernel.
The k3OS kernel with Wireguard is Coming Soon tm!"

As per the k3s source: https://github.com/rancher/k3s/blob/2ea7a717d958d98482462f58950230cef5feece9/scripts/wg-add.sh

boringtun is only required if the wireguard module is not available, once present this works as expected.

[dweomer]

Wireguard userspace was added in #444
Wireguard kernel module is included in #524 included in v0.11.0-rc1