Nixpkgs: dhcpcd: Failed to set DNS configuration

[demokritos]

Describe the bug
/etc/resolv.conf doesn't have a name server.

journal:

dhcpcd[685]: Failed to revert interface configuration: Unit dbus-org.freedesktop.resolve1.service not found.

To Reproduce
Steps to reproduce the behavior:

1. nixos-rebuild switch with the latest nixos-unstable channel.
2. reboot
3. cat /etc/resolv.conf

Expected behavior
/etc/resolv.conf should have a nameserver field with my router address.

Screenshots

dhcpcd[685]: Failed to revert interface configuration: Unit dbus-org.freedesktop.resolve1.service not found.
nscd[688]: 688 monitoring file `/etc/hosts` (4)
systemd[1]: Listening on Nix Daemon Socket.
dhcpcd[685]: Failed to revert interface configuration: Unit dbus-org.freedesktop.resolve1.service not found.
nscd[688]: 688 monitoring directory `/etc` (2)
systemd[1]: Reached target Sockets.
dhcpcd[685]: Failed to revert interface configuration: Unit dbus-org.freedesktop.resolve1.service not found.
rngd[689]: Initializing available sources
rngd[689]: [hwrng ]: Initialized
rngd[689]: [jitter]: Initializing AES buffer
rngd[689]: [jitter]: Unable to obtain AES key, disabling AES in JITTER source
rngd[689]: [jitter]: Enabling JITTER rng support
rngd[689]: [jitter]: Initialized
rngd[689]: [pkcs11]: No pkcs11 slots available
rngd[689]: [pkcs11]: Initialization Failed
nscd[688]: 688 monitoring file `/etc/resolv.conf` (5)
systemd[1]: Reached target Basic System.
dhcpcd[685]: Failed to set DNS configuration: Unit dbus-org.freedesktop.resolve1.service not found.
nscd[688]: 688 monitoring directory `/etc` (2)
systemd[1]: Starting Kernel Auditing...
nscd[688]: 688 monitoring file `/etc/services` (6)
systemd[1]: Starting CPU Frequency Setup...
nscd[688]: 688 monitoring directory `/etc` (2)
systemd[1]: Starting DHCP Client...
nscd[688]: 688 disabled inotify-based monitoring for file `/etc/netgroup': No such file or directory
systemd[1]: Starting Name Service Cache Daemon...
7k2j2sl0r491m7l08zfys1kv33knblij-unit-script-network-link-eth0-start[748]: Configuring link...
7k2j2sl0r491m7l08zfys1kv33knblij-unit-script-network-link-eth0-start[748]: bringing up interface... done
nscd[688]: 688 stat failed for file `/etc/netgroup'; will try again later: No such file or directory
systemd[1]: Starting resolvconf update...
s19cz5spvx0y6yaqbvm0arg6i4zi53ps-unit-script-network-link-wlan0-start[753]: Configuring link...
s19cz5spvx0y6yaqbvm0arg6i4zi53ps-unit-script-network-link-wlan0-start[753]: bringing up interface... done
dhcpcd[685]: dev: loaded udev
systemd[1]: Started Hardware RNG Entropy Gatherer Daemon.
dbus-daemon[691]: [system] Activating via systemd: service name='org.freedesktop.resolve1' unit='dbus-org.freedesktop.resolve1.service' requested by ':1.3' (uid=0 pid=729 comm="resolvconf -d eth0.link -f ")
systemd[1]: Started WPA Supplicant.
wpa_supplicant[690]: Successfully initialized wpa_supplicant
systemd[1]: Started D-Bus System Message Bus.
dbus-daemon[691]: [system] Activation via systemd failed for unit 'dbus-org.freedesktop.resolve1.service': Unit dbus-org.freedesktop.resolve1.service not found.
systemd[1]: Started Kernel Auditing.
dbus-daemon[691]: [system] Activating via systemd: service name='org.freedesktop.resolve1' unit='dbus-org.freedesktop.resolve1.service' requested by ':1.7' (uid=0 pid=755 comm="resolvconf -d wlan0.link -f ")
systemd[1]: Started CPU Frequency Setup.
dbus-daemon[691]: [system] Activation via systemd failed for unit 'dbus-org.freedesktop.resolve1.service': Unit dbus-org.freedesktop.resolve1.service not found.
systemd[1]: Started resolvconf update.
dhcpcd[685]: eth0: waiting for carrier
systemd[1]: Reached target Network (Pre).
dhcpcd[685]: wlan0: waiting for carrier
systemd[1]: Starting Address configuration of eth0...
dhcpcd[685]: wlan0: carrier acquired
systemd[1]: Starting Address configuration of wlan0...
wpa_supplicant[690]: wlan0: CTRL-EVENT-REGDOM-CHANGE init=USER type=COUNTRY alpha2=KR
systemd[1]: Starting Link configuration of eth0...
dhcpcd[685]: DUID xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
systemd[1]: Starting Link configuration of wlan0...
dhcpcd[685]: wlan0: IAID xxxxxxxxxx
systemd[1]: Started Address configuration of eth0.
dhcpcd[685]: wlan0: adding address xxxxxxxxxxxxxxxxxxxxxxxxxx
systemd[1]: Started Address configuration of wlan0.
dhcpcd[685]: wlan0: carrier lost
systemd[1]: Starting Networking Setup...
dbus-daemon[691]: [system] Activating via systemd: service name='org.freedesktop.resolve1' unit='dbus-org.freedesktop.resolve1.service' requested by ':1.8' (uid=0 pid=781 comm="resolvconf -d wlan0.link -f ")
systemd[1]: Started Name Service Cache Daemon.
dbus-daemon[691]: [system] Activation via systemd failed for unit 'dbus-org.freedesktop.resolve1.service': Unit dbus-org.freedesktop.resolve1.service not found.
systemd[1]: Reached target Host and Network Name Lookups.

Additional context
The machine is Raspberry Pi 3B. nixos-rebuild switch upgraded dhcpcd from 8.1.2 to 8.1.4 and linux kernel from 5.4.6 to 5.4.7.

Metadata

 - system: `"aarch64-linux"`
 - host os: `Linux 5.4.7, NixOS, 20.03pre207998.e0470e11c7a (Markhor)`
 - multi-user?: `no`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.3.1`
 - channels(root): `"nixos-20.03pre207998.e0470e11c7a"`
 - channels(esrevinu): `"nixpkgs-20.03pre207249.7e8454fb856"`
 - nixpkgs: `/home/esrevinu/.nix-defexpr/channels/nixpkgs`

[izuk]

I'm also seeing this.

[karolchmist]

I have the same problem. In my case, I did a bit of bisecting of the nixpkgs repo, and it seems that it is caused by this commit: cdf79db19d6. Previous commits that I checked, up to dad49d0b5ec (so just one before cdf79db19d6) seem to work fine. HEAD of nixos-unstable have still the same problem (2e8fc97dbfa).

Can you check if it is your case as well?

Pinging @Infinisil, maybe he has some insight to it.

My /etc/resolv.conf is also almost empty on cdf79db19d6:

~ cat /etc/resolv.conf                                                                               ~
# Generated by resolvconf
options edns0

On dad49d0b5ec:

➜  ~ cat /etc/resolv.conf                                                                               ~
# Generated by resolvconf
domain home
nameserver 192.168.1.1
nameserver xxx
nameserver xxx
nameserver xxx
options edns0

Metadata:

nix-shell -p nix-info --run "nix-info -m"                                                          ~
 - system: `"x86_64-linux"`
 - host os: `Linux 5.4.3, NixOS, 20.03.git.cdf79db (Markhor)`
 - multi-user?: `no`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.3.1`
 - channels(karol): `""`
 - channels(root): `""`
 - nixpkgs: `/etc/nixos/nixpkgs`

[Infinisil]

Thanks for the ping, I'll take a look

[Infinisil]

@karolchmist Can you run diff on your system build from https://github.com/NixOS/nixpkgs/commit/dad49d0b5ec032d32f7780d9853ea1cc28a91c37 to https://github.com/NixOS/nixpkgs/commit/cdf79db19d6ce1f305acebd86dabd58edb42e7c0 and show the result? You can find all previous system builds in /nix/var/nix/profiles/system-* so it should be something like

diff /nix/var/nix/profiles/system-100 /nix/var/nix/profiles/system-101

This will show exactly what ended up being different for that commit

[Infinisil]

Can reproduce, so I should be able to run this too

[Infinisil]

Found the culprit! While it does start failing since https://github.com/NixOS/nixpkgs/pull/75031, the actual mistake is not in that PR, it merely exposes it. In that PR I'm mentioning:

• [x] Made sure that this change doesn't introduce weird derivation changes (specifically the commit about submodule packing) -> It does, but nothing significant, only value ordering in lists for the same priority, which is arbitrary anyways

And that is indeed what happened for dhcpcd: the ordering of packages in services.dhcpcd.path (which was never fully specified) was changed such that the default systemd packages in https://github.com/NixOS/nixpkgs/blob/00c813a63628a6bbc315f08687c55a5fb5fa1c91/nixos/modules/system/boot/systemd.nix#L242-L249 newly come before the packages dhcpcd adds in https://github.com/NixOS/nixpkgs/blob/00c813a63628a6bbc315f08687c55a5fb5fa1c91/nixos/modules/services/networking/dhcpcd.nix#L173

However for dhcpcd that's a problem, because both pkgs.systemd and pkgs.openresolv provide a resolvconf binary! So this means that dhcpcd will end up using the one from systemd instead of the one from openresolv, leading to this failure.

I'll submit a PR that fixes this by explicitly putting systemd's packages after others.

[Infinisil]

See above PR

[demokritos]

I confirm that 9327e1c6ba1e6 fixes this issue.