Nixpkgs: systemd-timesyncd.service fails during 19.03 -> 19.09 upgrade

[edolstra]

(This is with b20a0e49c8c0d5c84b917bfbbe3f2ead31886a67 applied.)

# nixos-rebuild test --flake ~eelco/Misc/eelco-configurations/hagbard
...
Job for systemd-timesyncd.service failed because the control process exited with error code.
...
warning: the following units failed: systemd-timesyncd.service

● systemd-timesyncd.service - Network Time Synchronization
   Loaded: loaded (/nix/store/ag67dibj50z39rw1sr39zjd0dx6zcf2d-systemd-243/example/systemd/system/systemd-timesyncd.service; enabled; vendor preset: enabled)
  Drop-In: /nix/store/dyvdcmxf58ch46iypm7x5w8jh999anr2-system-units/systemd-timesyncd.service.d
           └─overrides.conf
   Active: failed (Result: exit-code) since Sun 2019-09-22 20:48:30 CEST; 1s ago
     Docs: man:systemd-timesyncd.service(8)
  Process: 15892 ExecStart=/nix/store/ag67dibj50z39rw1sr39zjd0dx6zcf2d-systemd-243/lib/systemd/systemd-timesyncd (code=exited, status=238/STATE_DIRECTORY)
 Main PID: 15892 (code=exited, status=238/STATE_DIRECTORY)
       IP: 0B in, 0B out
      CPU: 541us

Sep 22 20:48:30 hagbard systemd[1]: systemd-timesyncd.service: Service has no hold-off time (RestartSec=0), scheduling restart.
Sep 22 20:48:30 hagbard systemd[1]: systemd-timesyncd.service: Scheduled restart job, restart counter is at 5.
Sep 22 20:48:30 hagbard systemd[1]: Stopped Network Time Synchronization.
Sep 22 20:48:30 hagbard systemd[1]: systemd-timesyncd.service: Start request repeated too quickly.
Sep 22 20:48:30 hagbard systemd[1]: systemd-timesyncd.service: Failed with result 'exit-code'.
Sep 22 20:48:30 hagbard systemd[1]: Failed to start Network Time Synchronization.
warning: error(s) occurred while switching to the new configuration

The journal also contains:

Sep 22 20:48:30 hagbard systemd[15869]: systemd-timesyncd.service: Failed to set up special execution directory in /var/lib: Not a directory
Sep 22 20:48:30 hagbard systemd[15869]: systemd-timesyncd.service: Failed at step STATE_DIRECTORY spawning /nix/store/ag67dibj50z39rw1sr39zjd0dx6zcf2d-systemd-243/lib/systemd/systemd-timesyncd: Not a directory

[matthewbauer]

@edolstra This looks like the same issue as https://github.com/NixOS/nixpkgs/issues/64922. Apparently it is fixed in system 243.

[edolstra]

Yeah, it actually had nothing to do with not rebooting since the error persisted after a reboot.

[edolstra]

And setting system.stateVersion to 19.03 did fix it.

[rvolosatovs]

@matthewbauer I'm seeing the exact same issue on transition 19.03 -> 19.09.
systemd version is 243, so doesn't look like it's fixed:

$ systemctl --version             
systemd 243 (243)
+PAM +AUDIT -SELINUX +IMA +APPARMOR +SMACK -SYSVINIT +UTMP -LIBCRYPTSETUP +GCRYPT -GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID -ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=hybrid

[ertw]

For what it's worth, I ran into this issue moving a machine from 19.03 -> 19.09, and had to delete /var/lib/systemd/timesync per what I found on the Arch wiki, so I guess this is a larger issue affecting systemd 243. After deleting that file, the upgrade worked fine :heavy_check_mark:

The error I was seeing on attempting nixos-rebuild switch --upgrade was: -timesyncd (code=exited, status=238/STATE_DIRECTORY)