Nixpkgs: systemd-networkd's DHCPServer does not start a DHCP server.

[tomfitzhenry]

Summary

In master and 19.09, systemd-networkd with DHCPServer=yes does not result in DHCP being served.

Steps to reproduce

https://github.com/NixOS/nixpkgs/pull/73505 is a NixOS test that reproduces this. The test passes, by applying the workaround described below. To reproduce the bug, remove the workaround.

Mechanism

In systemd 242, systemd would create /etc/localtime and systemd-networkd would grab timezone info from /etc/localtime as part of being a DHCP server. Everything is good so far.

systemd 243 introduced https://github.com/systemd/systemd/commit/09bef967d9cacb95bafab0882606cc3a57ac324e, which removed the creation of /etc/localtime, and thus systemd-networkd fails to get timezone data, and thus fails to start a DHCP server. In particular, not listening on 67/udp.

https://github.com/systemd/systemd/commit/bc9ecd484f1ebfe0de8b567c90f6cd867fbd5894 will fix this by falling back to UTC if /etc/localtime does not exist.

Upstream bug: https://github.com/systemd/systemd/issues/14050

Solution

master: Backport https://github.com/systemd/systemd/commit/bc9ecd484f1ebfe0de8b567c90f6cd867fbd5894 or wait for systemd 244.

19.09: Backport https://github.com/systemd/systemd/commit/bc9ecd484f1ebfe0de8b567c90f6cd867fbd5894 .

Workaround

Set EmitTimezone = false in your network unit, or create /etc/localtime.

Metadata

$ nix run nixpkgs.nix-info -c nix-info -m
 - system: `"x86_64-linux"`
 - host os: `Linux 4.19.80-1.pvops.qubes.x86_64, Debian GNU/Linux, 10 (buster)`
 - multi-user?: `no`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.3.1`
 - channels(user): `"nixpkgs-20.03pre197053.d08a7431561"`
 - nixpkgs: `/home/user/.nix-defexpr/channels/nixpkgs`

Maintainer information:

# a list of nixpkgs attributes affected by the problem
attribute:
# a list of nixos modules affected by the problem
module: systemd-networkd

[arianvp]

I have a workaround here:

https://github.com/arianvp/nixos-stuff/blob/f34eb42447956e62b5a8c0b0419d0102826a6c15/configs/arianvp.me/default.nix#L11

[arianvp]

Let's make timezone mandatory if DHCPServer is enabled? Does that make sense?

[tomfitzhenry]

Let's make timezone mandatory if DHCPServer is enabled? Does that make sense?

I can see how that workaround works, and think it's a good solution.

I don't think timezone should be mandatory once systemd-networkd 244 has been released, however, since it's unnecessary. Is there a way we/maintainers can be reminded to remove this constraint once systemd-networkd 244 is released and in master?

[tomfitzhenry]

An alternative is to have the networkd module set timezone if it isn't set already. I don't think this is a good solution, however, since if/when we remove this workaround, that would be a breaking change (in particular, /etc/localtime would be removed).

[tomfitzhenry]

To save users the bother of setting timezone, how difficult is it to apply https://github.com/systemd/systemd/commit/bc9ecd484f1ebfe0de8b567c90f6cd867fbd5894 as a patch?

[flokli]

I think this is an upstream bug - I asked in https://github.com/systemd/systemd/pull/14010#issuecomment-554738488 for a backport to systemd-stable (which our systemd derivation bases on), otherwise, we could manually apply the patch there, too.

[tomfitzhenry]

I think this is an upstream bug - I asked in systemd/systemd#14010 (comment) for a backport to systemd-stable (which our systemd derivation bases on)

Great. I wasn't aware systemd provided a stable track. I'm happy to wait a day or two, to see if upstream backport the fix.

[flokli]

upstream did backport to systemd/systemd-stable. @andir @Mic92, could you update nixos/systemd to include those patches?

[andir]

"those"? Can you please point out which once exactly?

[flokli]

I wouldn't propose to provide a curated set of commits. We should rewind to systemd-stable, plus our NixOS-specific patches on top (maybe push to a v243-1 branch to not mess with the existing v243 one)

[tomfitzhenry]

@andir it's v243.4 which contains the fix https://github.com/systemd/systemd-stable/commit/f5a4caa5418b87ebfb2c254a6a05e2c7c2ddbef1 .

$ git remote -v
origin  https://github.com/systemd/systemd-stable (fetch)
origin  https://github.com/systemd/systemd-stable (push)
$ git log -n1 f5a4caa5418b87ebfb2c254a6a05e2c7c2ddbef1
commit f5a4caa5418b87ebfb2c254a6a05e2c7c2ddbef1
Author: Lennart Poettering <[email protected]>
Date:   Wed Nov 13 10:32:26 2019 +0100

    time-util: treat /etc/localtime missing as UTC

    (cherry picked from commit bc9ecd484f1ebfe0de8b567c90f6cd867fbd5894)
$ git tag --contains f5a4caa5418b87ebfb2c254a6a05e2c7c2ddbef1
v243.4

[flokli]

systemd 243.4 PR: https://github.com/NixOS/nixpkgs/pull/74207

[tomfitzhenry]

master no longer has this issue, so I'm closing. I will push for the test https://github.com/NixOS/nixpkgs/pull/73505 to be merged, to catch regressions. I have updated the test to no longer require the workaround described in this issue.