Nixpkgs: NixOS RPi4 after reloading new config would still keep things from the installer config

[diamondburned]

Description

Right now, the sd-image-raspberrypi4.nix imports installation-device.nix, which does multiple things, such as enabling the nixos user or disabling SSH and wpa_supplicant autostart.

According to https://github.com/NixOS/nixpkgs/issues/63576, the end-user should nixos-generate-config and nixos-rebuild switch to that new config. However, after a reboot, I saw both new config attributes (my created user was there) and old attributes (the nixos user was there, SSH was not autostarted, ...).

During the config switch, errors were spotted. However, later switches on the same config did not really do anything (seemingly like the last switch was successful).

Since sd-image-raspberrypi.nix also imports what sd-image-raspberrypi4.nix imports, it's likely that other Raspberry Pis would have the same issue.

Reproducing

Steps to reproduce the behavior:

1. Create a NixOS image which imports sd-image-raspberrypi4 [refer to "Creating an image"]
2. Flash the image to a micro SD card and plug it into the Raspberry Pi 4
3. Go to the Pi's tty/console
4. Generate a new configuration.nix and switch:

sudo nixos-generate-config
sudo nixos-rebuild switch

Expected Behavior

There should've been no traces of the installation-device.nix file, meaning the nixos user shouldn't be there.

Screenshots

Metadata

• system: "aarch64-linux"
• host os: Linux 4.19.75, NixOS, 20.03pre201329.f1682a7f126 (Markhor)
• multi-user?: yes
• sandbox: yes
• version: nix-env (Nix) 2.3.1
• channels(root): "nixos-20.03pre201329.f1682a7f126"
• nixpkgs: /nix/var/nix/profiles/per-user/root/channels/nixos

Addendum: Creating an Image (has to be done on an aarch64 machine)

1. Save this file as ./sd-image.nix

{ config, pkgs, lib, ... }:

{
    imports = [
        <nixpkgs/nixos/modules/installer/cd-dvd/sd-image-raspberrypi4.nix>
    ];
}

1. Run this command

nix-build '<nixpkgs/nixos>' -A config.system.build.sdImage -I nixos-config=./sd-image.nix

[diamondburned]

I have some sort of clues right now. The generation in /boot is up to date, but the generation in the FIRMWARE partition is still the old one.

I'm thinking a fix would be to change this line to "/boot"

[diamondburned]

Another update, seems like populateRootCommands in sd-image.nix is never used?

[diamondburned]

Got SSH working AND config reload working with

{
    sdImage.populateFirmwareCommands = ''
        ${config.system.build.installBootLoader} ${config.system.build.toplevel} -d ./
    '';
    sdImage.populateRootCommands = ''
        mkdir -p ./files/var/empty
    '';

    fileSystems = lib.mkForce {
        "/boot" = {
            device = "/dev/disk/by-label/FIRMWARE";
            fsType = "vfat";
            # we NEED this mounted
        };
        "/" = {
            device = "/dev/disk/by-label/NIXOS_SD";
            fsType = "ext4";
        };
    };
}

I'll make a PR soon.

[mmahut]

@diamondburned awesome work!

[Renha]

Could you please give a link to your PR?

[cmacrae]

@diamondburned :wave: I built an image with your above method (fixes for /var/empty & /boot mount/population included). It did indeed seem to fix the sshd issue, as /var/empty was now present, however when I generate a config, switch to that, after reboot it seems I'm chucked back to the boot configuration of the installer.

Would you mind sharing the configuration.nix you're using for your RPi4? I'm not sure what I'm missing :thinking:

[diamondburned]

Regarding this: with the burden of my procrastination and me keep making new projects, I'm afraid I can't make a PR.

Though I still have my working config; if someone could pick this up, I would really appreciate it: https://gitlab.com/diamondburned/nix-rpi4

@cmacrae ^

[cmacrae]

Cheers @diamondburned :pray: Sure, I'll happily pick this up.
I'll try and get a PR in tonight :)

[Renha]

@cmacrae can you please provide a link to your PR?

[cmacrae]

@Renha I haven't had the time to open it yet, I'm afraid. Free weekend ahead of me though, so I'll set a reminder up!

[cmacrae]

@Renha There you go :point_up: :slightly_smiling_face:

[stale[bot]]

Hello, I'm a bot and I thank you in the name of the community for opening this issue.

To help our human contributors focus on the most-relevant reports, I check up on old issues to see if they're still relevant. This issue has had no activity for 180 days, and so I marked it as stale, but you can rest assured it will never be closed by a non-human.

The community would appreciate your effort in checking if the issue is still valid. If it isn't, please close it.

If the issue persists, and you'd like to remove the stale label, you simply need to leave a comment. Your comment can be as simple as "still important to me". If you'd like it to get more attention, you can ask for help by searching for maintainers and people that previously touched related code and @ mention them in a comment. You can use Git blame or GitHub's web interface on the relevant files to find them.

Lastly, you can always ask for help at our Discourse Forum or at #nixos' IRC channel.