Nixpkgs: Raspberry Pi 3 Support

[fdietze]

So far, I was unable to install nix on rpi3. What I tried:

• Flash Armv7 image to sdcard and boot Pi. Kernel does not boot, missing dtb files. So I copied bcm2710-rpi-3-b.dtb and bcm2837-rpi-3-b.dtb from the official raspberry image on the sdcard.
• It boots but has a black screen.
• I copied my public key into /root/.ssh/authorized_keys, bootet pi and blindly typed systemctl start sshd. It worked! Now I had ssh access to my pi.
• I changed my configuration.nix to:

{ config, pkgs, lib, ... }:
 {
   # NixOS wants to enable GRUB by default
   boot.loader.grub.enable = false;
   # Enables the generation of /boot/extlinux/extlinux.conf
   boot.loader.generic-extlinux-compatible.enable = true;

   boot.kernelPackages = pkgs.linuxPackages_rpi;

   # Manual doesn't currently evaluate on ARM
   services.nixosManual.enable = false;

   nix.binaryCaches = lib.mkForce [ "http://nixos-arm.dezgeg.me/channel" ];
   nix.binaryCachePublicKeys = [ "nixos-arm.dezgeg.me-1:xBaUKS3n17BZPKeyxL4JfbTqECsT+ysbDJz29kLFRW0=%" ];

   # File systems configuration for using the installer's partition layout
   fileSystems = {
     "/boot" = {
       device = "/dev/disk/by-label/NIXOS_BOOT";
       fsType = "vfat";
     };
     "/" = {
       device = "/dev/disk/by-label/NIXOS_SD";
       fsType = "ext4";
     };
   };


   environment.systemPackages = [ pkgs.vim ];
   services.openssh.enable = true;
   users.extraUsers.root.openssh.authorizedKeys.keys =
       [ "ssh-rsa  xxxxxxxxxxxxx" ];
 }

Then:

nixos-rebuild switch --fast --option binary-caches http://nixos-arm.dezgeg.me/channel --option binary-cache-public-keys nixos-arm.dezgeg.me-1:xBaUKS3n17BZPKeyxL4JfbTqECsT+ysbDJz29kLFRW0=%

After success, the newly installed rpi-kernel does not boot. Any Ideas?

[Mic92]

Is this related to https://github.com/NixOS/nixpkgs/pull/21988 ?

[wizeman]

@Mic92: indeed.

@fdietze: at least for me, my Raspberry Pi 3 did not boot until I made the modifications in #21988. Note that those changes haven't been merged into the master branch yet (much less made it onto a stable release).

Apart from those changes, you will also need to disable the generic-extlinux-compatible boot loader, and enable the raspberryPi boot loader instead.

The linuxPackages_rpi kernel is also needed, but you already have that in your configuration.

Although I'm not completely sure, another thing which is probably needed for the Raspberry Pi 2 or 3, but is not usually mentioned anywhere, is that you need to have the following text in your /boot/config.txt:

kernel=kernel7.img
initramfs initrd

Although if someone could confirm whether the above point is really needed or not, that would be great.

[dezgeg]

I am waiting for this kernel patch: https://lkml.org/lkml/2017/1/19/542 to go in to make sd-image-armv7l-linux.img boot out-of-the-box.

[wizeman]

@dezgeg As far as I can see, that only contains the Raspberry Pi 3's DTB for the arm64 architecture, not the arm32 that we are using.

IIRC, there was a similar patch a few months ago which also added the RPi3 DTB for arm32, but there was some pushback, both because of duplicate code, but also because the kernel developers seemed to prefer to support only 64-bit kernels (if the hardware supports it).

Last time I checked, NixOS still doesn't support the arm64/aarch64 architecture, so if I'm not mistaken, the patch you mentioned wouldn't be very useful currently... :disappointed:

[wizeman]

Ah wait, maybe I've misread the patch. It seems that it adds the 32-bit DTB, so indeed it would be very useful. Thanks for keeping track of this, @dezgeg!

[dezgeg]

No, that kernel patch is for 32-bit ARM. The DT is just identical between 32-bit and 64-bit (and it already exists under arch/arm64). It's true that the arm-soc maintainers might still not want that, let's hope they do.

Aarch64 support is implemented here: #22117

[FPtje]

I can confirm the black screen on boot issue on the Raspberry pi 3. U-Boot shows fine, but the screen goes blank when the kernel starts loading. The exact same SD card would boot fine (with HDMI working) on a Raspberry pi 2.

It's a minor problem to me personally, since we don't use the HDMI port anyway.

Also, I've got a Linux kernel override that includes the DTB patch. For those who want to build their own RPI3 image, override the kernel like this:

{
    linux_4_8 = pkgs.linux_4_8.override {
      kernelPatches = [
        { patch = ./rpi3-dtb.patch; name = "rpi3-dtb"; }
      ];
    };
}

rpi3-dtb.patch containing of course this patch.

In other news, the Linux kernel for the raspberry pi is very limited. Things like firewalls, TUN, iptables stuff and I2C don't work out of the box because of missing kernel configs. Is this intentional? I've enabled a bunch of these things in my linux_4_8 override, using the extraConfig field, but I could imagine that it could be desirable to have NixOS support most of the same stuff Raspbian does.

[dezgeg]

FWIW, when I tried RPi3 on Aarch64 the HDMI worked for me. Maybe it got fixed/enabled in kernel 4.9.

For the kernel config question, it's because sometimes (not always) enabling the kernelAutoModules option for the config generator (which tries to build anything that's not already enabled as a module) makes the kernel crash on boot. Really weird problem. But do tell what's missing and I can explicitly enable it.

[FPtje]

My entire override can be found here. There's I2C stuff, which might be useful for tinkering. The INET, IP[6]_..., NETFILTER_, NF_ stuff is for the firewall (and for our VPN).

Many things (like the PPP stuff) are probably not needed. The default Raspbian stuff can be found here, scattered in Kconfig files.

[Mic92]

@fdietze #21988 was merged now. Does nixos now work for you on rpi3?

[dezgeg]

That one is not related to OP's config, as it uses boot.loader.generic-extlinux-compatible.enable = true;

However, this might help:
nixpkgs.config = { platform = pkgs.platforms.raspberrypi2; };

[fdietze]

Sorry for the silence, didn't have time yet to try again. I'll report back when I do.

[fdietze]

So I tried it again:

• downloaded latest armv7 image, still had to copy missing dtb files to boot partition
• It still boots to a black screen.
• copied my public key into /root/.ssh/authorized_keys, bootet pi, blindly typed systemctl start sshd
  I tried to include the suggestions from this thread into my configuration.nix:

{ config, pkgs, lib, ... }:
 {

   # NixOS wants to enable GRUB by default
   boot.loader.grub.enable = false;
   # Enables the generation of /boot/extlinux/extlinux.conf
   boot.loader.generic-extlinux-compatible.enable = false;
   boot.loader.raspberryPi.enable = true;
   boot.loader.raspberryPi.version = 3;
   boot.extraKernelParams = [ "nomodeset" ]; # doesn't have any effect

   boot.kernelPackages = pkgs.linuxPackages_rpi;

   nixpkgs.config = {
    #platform = pkgs.platforms.raspberrypi2; # would produce infinite recursion?!
    allowUnfree = true;
  };

   # Manual doesn't currently evaluate on ARM
   services.nixosManual.enable = false;

   nix.binaryCaches = lib.mkForce [ "http://nixos-arm.dezgeg.me/channel" ];
   nix.binaryCachePublicKeys = [ "nixos-arm.dezgeg.me-1:xBaUKS3n17BZPKeyxL4JfbTqECsT+ysbDJz29kLFRW0=%" ];

   # File systems configuration for using the installer's partition layout
   fileSystems = {
     "/boot" = {
       device = "/dev/disk/by-label/NIXOS_BOOT";
       fsType = "vfat";
     };
     "/" = {
       device = "/dev/disk/by-label/NIXOS_SD";
       fsType = "ext4";
     };
   };


   environment.systemPackages = [ pkgs.vim ];
   services.sshd.enable = true;
   users.extraUsers.root.openssh.authorizedKeys.keys =
       [ "ssh-rsa ..." ];
 }

then run:

nixos-rebuild switch --fast --option binary-caches http://nixos-arm.dezgeg.me/channel --option binary-cache-public-keys nixos-arm.dezgeg.me-1:xBaUKS3n17BZPKeyxL4JfbTqECsT+ysbDJz29kLFRW0=%

and rebooted.

Still black screen after reboot and ssh service does not start automatically (I can still blindly start it via keyboard). How can I know if it worked? Any suggestions?

[fdietze]

It seems to work with a newer kernel:

boot.kernelPackages = pkgs.linuxPackages_4_10;

At least the ssh service is started automatically now.

[fdietze]

Does anyone know why my Pi gets a new MAC-address on every boot?

[dezgeg]

I noticed it as well. I tracked down into an bug in either U-Boot or the mainline DTBs. (Basically U-Boot currently doesn't recognize aliases { ethernet = <&whatever>; }, it wants the alias name to be ethernet0 or similar.)

[dezgeg]

BTW, now that we have aarch64 builds on the official Hydra, I recommend the mainline kernel on aarch64 even more on the RPi3. The images aren't currently being built by Hydra but here's a fairly recent one: https://www.cs.helsinki.fi/u/tmtynkky/sd-image-aarch64-linux.img

Still, for the 32-bit port, there has been some activity from the bcm2835 maintainer in February: https://www.spinics.net/lists/arm-kernel/msg561622.html, so I expect that to land in the 4.12 kernel...

[fdietze]

The aarch64 image works way better. It boots correctly without any hacks and the default binary cache works well. My problem now is that nixos-rebuild switch fails, because there is no space left on /boot. I tried resizing it using gparted, but that didn't work. Any advice on resizing the boot partition?

[FPtje]

My RPI boot partition always fills up when I deploy new versions of the kernel often enough. New boot files are put into /boot when a new kernel is installed. The solution is to perform a sudo nix-collect-garbage -d. WARNING: this will remove all previous generations of NixOS, so you won't be able to boot to an earlier version of the system. A nixos-rebuild switch after collecting garbage should clear the redundant /boot files, clearing up space.

In my personal case I can't switch even after collecting garbage. The message about not enough space being left is thrown before it actually cleans /boot AFAIK. I've had to delete files from /boot before. Be very careful with that, though, since you can render your system unbootable.

[vcunat]

For full /boot see https://github.com/NixOS/nixpkgs/issues/23926

[dezgeg]

In retrospect, actually mounting the FAT partition at /boot was a bad idea... U-Boot can load the kernel+initrd+etc. just fine from the ext4 partition. To do that:

• umount /boot
• comment out the fileSystems."/boot" entry from configuration.nix
• use fdisk to remove the bootable flag from the FAT32 partition, and set it for the ext4 partition
• nixos-rebuild switch

[tohl]

Tuomas Tynkkynen [e-mail] writes:

BTW, now that we have aarch64 builds on the official Hydra, I
recommend the mainline kernel on aarch64 even more on the RPi3.

great, thank you!

The images aren't currently being built by Hydra but here's a fairly
recent one:
https://www.cs.helsinki.fi/u/tmtynkky/sd-image-aarch64-linux.img

I copied the img to sd card and booted. It hangs after the message
about loading the kernel. Am I supposed to do something special for it
to start nixos?

[dezgeg]

It certainly works for me, though it certainly stays there for several seconds. Maybe you have a slow(er) card.

[rickynils]

@tohl I got the behavior you're describing before switching out the power adapter for a more powerful one.

[tohl]

that did the trick, it works now, thank you

[tohl]

Is there an example configuration.nix somewhere?

The one in the sd-image just imports the sd-image expression, but it is
not clear to me, how would I go from there to my own configuration?
E.g. nixos-hardware repo does show only sample config for rpi2 but not
for rpi3. Simply trying to enable ssh with that default import does not
work, for example.

[dezgeg]

Use one from section "NixOS installation & configuration" from https://nixos.org/wiki/NixOS_on_ARM. Don't pick linuxPackages_rpi but rather linuxPackages_latest.

[tohl]

Use one from section "NixOS installation & configuration" from
https://nixos.org/wiki/NixOS_on_ARM. Don't pick linuxPackages_rpi
but rather linuxPackages_latest.

yes, that worked, thank you

also, the custom binary cache configuration can be ignored for aarch64

[tohl]

it seems to work very well, except firewall doesnt work:

firewall = {
  enable = true;
  allowPing = true;
  allowedTCPPorts = [
    22 # sshd
  ];
};

Apr 22 16:59:02 nixos systemd[1]: Starting Firewall...
Apr 22 16:59:02 nixos firewall-start[12601]: iptables: No
chain/target/match by that name.
Apr 22 16:59:02 nixos systemd[1]: firewall.service: Main process exited,
code=exited, status=1/FAILURE
Apr 22 16:59:02 nixos systemd[1]: Failed to start Firewall.
Apr 22 16:59:02 nixos systemd[1]: firewall.service: Unit entered failed
state.
Apr 22 16:59:02 nixos systemd[1]: firewall.service: Failed with result
'exit-code'.

is that expected?

[dezgeg]

Did you try upgrading channel? I think it's fixed in the latest kernel builds.

[tohl]

Did you try upgrading channel? I think it's fixed in the latest kernel builds.

How can I upgrade the channel?

There isn't seem to be any by default and when I add one, it cannot
unpack the channels.

Tomas

===

[root@nixos:~]# nix-channel --list

[root@nixos:~]# nix-channel --add https://nixos.org/channels/nixos-unstable
nixos

[root@nixos:~]# nix-channel --list
nixos https://nixos.org/channels/nixos-unstable

[root@nixos:~]# nix-channel --update
downloading Nix expressions from
‘https://d3g5gsiof5omrk.cloudfront.net/nixos/unstable/nixos-17.09pre105622.f0fac3b578/nixexprs.tar.xz’...
downloading
‘https://d3g5gsiof5omrk.cloudfront.net/nixos/unstable/nixos-17.09pre105622.f0fac3b578/nixexprs.tar.xz’... [8009/8486
KiB, 1136.6 KiB/s]
unpacking channels...
builder for
‘/nix/store/s8kvq42vx1dxl8d3n5kcsymipw2ahig0-user-environment.drv’
failed to produce output path
‘/nix/store/9vgcjy5xgn2yjclp66k4kpad99npm11c-user-environment’
error: build of
‘/nix/store/s8kvq42vx1dxl8d3n5kcsymipw2ahig0-user-environment.drv’
failed
cannot unpack the channels at /run/current-system/sw/bin/nix-channel
line 169.

[root@nixos:~]# nixos-rebuild switch --upgrade
downloading Nix expressions from
‘https://d3g5gsiof5omrk.cloudfront.net/nixos/unstable/nixos-17.09pre105622.f0fac3b578/nixexprs.tar.xz’...
downloading
‘https://d3g5gsiof5omrk.cloudfront.net/nixos/unstable/nixos-17.09pre105622.f0fac3b578/nixexprs.tar.xz’... [8006/8486
KiB, 1136.5 KiB/s]
unpacking channels...
builder for
‘/nix/store/s8kvq42vx1dxl8d3n5kcsymipw2ahig0-user-environment.drv’
failed to produce output path
‘/nix/store/9vgcjy5xgn2yjclp66k4kpad99npm11c-user-environment’
error: build of
‘/nix/store/s8kvq42vx1dxl8d3n5kcsymipw2ahig0-user-environment.drv’
failed
cannot unpack the channels at /run/current-system/sw/bin/nix-channel
line 169.

[dezgeg]

I can't replicate that problem. Maybe you had something getting corrupted on the partition when the boot crashed due to the power problem.

Of course, building from a git checkout (-I nixpkgs=/path/to/nixpkgs) should also work.

[purefn]

The wiki has been shutdown. Will someone put up an ARM page on the https://github.com/nixos-users wiki, with example configuration.nix?

[Mic92]

you can.

[makefu]

@purefn Current status is that the aarch64 image works pretty much out of the box (except for wifi). I've documented this in the unofficial nixos wiki at https://nixos.wiki/wiki/NixOS_on_ARM#Raspberry_Pi_3 .

I am running nixos-unstable (2017-10-08).

[fdietze]

Same for me! Things work out of the box (using the configuration.nix from the wiki article). I think this can be closed.

[mogria]

I'm running the AArch64 image as described on the unoffical wiki.
Does anyone have the Audio working via the jack port?

I tried putting dtparam=audio=on in config.txt and copying the .dtb files from the raspberrypifw package into /boot. But that doesn't seem to have any effect.

No sound cards are detected. I've also tried manually loading the snd_bcm2385 module, but with no effect at all. I tried to debug using the tools in raspberrypifw but they don't seem to work as of now (See https://github.com/NixOS/nixpkgs/issues/30026).

[florianjacob]

@mogria same problem here ☹️ also tried audio_pwm_mode=2 and kernel 4.14.
This gentoo image claims they have it working, though.

[ghost]

I have had no luck with audio as well.

The main problem I am having is hard crashing on u-boot initializing the OS. I have had this with multiple re-images of a few revisions of dezgeg's aarch64 image, I am not sure how to troubleshoot this really.

Seems like a memory issue possibly & occasionally the tty seems to crash to the U-boot init text, but the Alt-Fn shortcuts allow switching back to a functioning tty.

[iblech]

@hcmensch: Are you sure that there are no problems with the power supply? The phenomenon you describe is precisely what is stated at the bottom of https://nixos.wiki/wiki/NixOS_on_ARM/Raspberry_Pi about problems with the power supply.

[ghost]

I started off with bad power supplies, then upgraded to a correct spec supply from one of the official RPI re-sellers after reading about the potential problems. I think I will try to test the supply at my university's ee lab and get a different one in case I received a fake or faulty unit.

[garbas]

could we add the rpi3 hardware configuration here -> https://github.com/NixOS/nixos-hardware/tree/master/raspberry-pi?

[bachp]

I was also playing with rpi3 and an aarch64 image. Almost everything seems to work out of the box.

I now to updated the raspberrypifw packet to see if I can get the camera working, but the change in firmware doesn't seem to be picked up.

After some analysis I figured found the options:

boot.loader.raspberryPi.enable = true;
boot.loader.raspberryPi.version = 3;

and from looking at the source of the module I think it should copy the new firmware to the /boot partition. But it doesn't.

Does anybody know how the rpi3 firmware loading is supposed to work?

[bachp]

It turns out I had boot.loader.generic-extlinux-compatible.enable = true; set which prevents boot.loader.raspberryPi.enable = true; from taking effect.

After disabling boot.loader.generic-extlinux-compatible the new firmware files were copied to /boot but the system didn't boot anymore as the bootloader entry didn't update.

I did some additional digging and found out the the initial firmware comes from sd-image-aarch64.nix. It seems to be copied there when the image is initially created but I couldn't find any mechanism that would allow to update the firmware on a running NixOS.

Is this is just not implemented yet or am I missing something?

[dezgeg]

@bachp Yes, the answer currently is essentially 'not implemented yet'. To upgrade the firmware, replace these files in /boot with new versions: bootcode.bin fixup_cd.dat fixup.dat fixup_db.dat fixup_x.dat start_cd.elf start_db.elf start.elf start_x.elf

[bachp]

@dezgeg Ok thanks for the info. Do you already have something in mind how this could be done? I might find some time to work on this.

[florianjacob]

With the firmware upgrade instructions, I tried again to get analog audio to work with 18.03 pre, 4.15.12 kernel, manually downloaded firmware 20180328, dtparam=audio=on, modprobe snd_bcm2835 but no luck. :frowning_face:

(To people wondering why someone would want to use the analog audio of a RaspberryPi 3: It's not as bad as it used to be, and especially with audio_pwm_mode=2 the noise nterference is gone. :wink: )

[bachp]

@florianjacob I was trying to get the camera working but no luck so far too. I also tried with the latest firmware 20180328.

Did anybody manage to get the camera working under NixOS?

[bkchr]

So, I got the camera working.

The following patch is required: https://patchwork.kernel.org/patch/9626797/

And you need to add the following to /boot/config.txt:

gpu_mem=128
start_x=1

[florianjacob]

No sound cards are detected. I've also tried manually loading the snd_bcm2385 module, but with no effect at all. I tried to debug using the tools in raspberrypifw but they don't seem to work as of now (See #30026).

@mogria now that #30026 seems to be mostly resolved, could you name which tool(s) you intended to use for debugging the sound issues? I'm having trouble to find something useful in that area.

[bachp]

@bkchr What firmware version have you been using?

[bkchr]

@bachp the one you created.

[bachp]

@bkchr I should read more carfully. I'm probably missing the patch you mentioned. Anyway as you say it is working with the new FW I submit is as PR.

[timclassic]

I tried booting the sd-image-aarch64-linux.img from https://www.cs.helsinki.fi/u/tmtynkky/nixos-arm/installer/ on a Raspberry Pi 3B+ and it failed to boot at all. I then updated the firmware files in the /boot filesystem to the latest from https://github.com/raspberrypi/firmware/tree/master/boot and this allowed U-boot to load. However, U-Boot tried (and failed) to load "bcm283x-rpi-other.dtb" which appears to be a placeholder name to indicate an unknown model. I suspect like I'm out of luck for the moment.

However, I found a few related patches to U-Boot:

• https://lists.denx.de/pipermail/u-boot/2018-April/324722.html
• https://lists.denx.de/pipermail/u-boot/2018-April/324724.html

Is there already work underway to support the Pi 3B+ in NixOS, and I just haven't found it yet? I'm happy to open a separate issue if appropriate.

[bkchr]

@timclassic did you tried booting with the raspberry pi kernel? Upstream probably does not have the required devicetree yet.

[dezgeg]

Actually it's U-Boot which needs to be upgraded to know about RPi 3B+; I saw some patches related to that on the mailing list so hopefully it's in U-Boot 2018.05.

[bkchr]

Using https://github.com/NixOS/nixpkgs/pull/38490 and https://github.com/NixOS/nixpkgs/pull/38342, the camera module should be work out of the box. You will need the following options in your configuration:

boot.loader.raspberryPi.enable = true;
boot.loader.raspberryPi.version = 3;
boot.loader.raspberryPi.uboot.enable = true;
boot.loader.raspberryPi.uboot.startx = true;
boot.loader.raspberryPi.uboot.gpu_mem = 256;

[makefu]

@bkchr so many great hints and pointers, would you mind putting this info into either the wiki or nixos-hardware?

[bkchr]

I will try to put it into the wiki. :)

[bkchr]

@makefu I added it to the wiki :) https://nixos.wiki/wiki/NixOS_on_ARM/Raspberry_Pi#Camera

[makefu]

@bkchr yes i already saw that via the wiki notification bot in freenode#nixos-wiki , thanks!

[bkchr]

@makefu @dezgeg someone of you could merge the two pull requests?

[makefu]

@bkchr unfortunately i am nothing more than an ordinary nixos user. @Mic92, @dezgeg pretty please? :+1:

[dezgeg]

I will look at them once I have some time.

[bachp]

For those interested in order to get the camera to show up as /dev/video0 you need to load the bcm2835-v4l2 module.

I also updated this in the wiki: https://nixos.wiki/wiki/NixOS_on_ARM/Raspberry_Pi#Camera

[bachp]

Did somebody get an xserver running on rpi3 with aarch64?

[dezgeg]

Yes it works fine (including OpenGL) with the modesetting video driver.

[bachp]

@dezgeg Are there any special settings you need to provide like cma size on the commandline or gpu_mem size in config.txt.

It might be worth to add this to the wiki too.

[dezgeg]

Right, the cma setting might be needed (in order to even get a virtual console at all). But if that's needed we should rather bump the default CMA size up in the kernel config so things work out-of-the-box. I don't think gpu_mem needs to be changed.

[bachp]

I had to set cma=128M to get the xserver running. Everything lower wouldn't work.

[timclassic]

@bkchr No, I have not tried the Raspberry Pi kernel. Do you think that it's necessary, even with the U-Boot updates I likely need?

@dezgeg I'd like to try building the latest U-Boot myself. I know that I can set up cross-compliation for aarch64 in general, and I've also found the instructions for building just U-Boot at https://nixos.wiki/wiki/NixOS_on_ARM#Building_u-boot_from_your_NixOS_PC. I'm not sure if either of these alone is enough, though. Can you point me at the process for generating the installation images you host at http://nixos-arm.dezgeg.me/installer, and I'll just start from there?

[bkchr]

No, I think then first try uboot and see what happens :)

[timclassic]

@dezgeg Feel free to ignore my question. I've been able to generate my own SD card image via an older rpi3. Hopefully I can work from this to bootstrap rpi3+ support.

[hlolli]

Useing the lastest sd-image-aarch64-linux.img from @dezgeg for rpi3+. Keep getting stuck on the rainbow splash screen (don't have the serial cable, so I rely on hdmi + keyboard). Just reading very related threads, I have good suspicion that the release v2018.05 of u-boot from earlier today https://github.com/u-boot/u-boot/releases/tag/v2018.05 solves the rpi3+ booting problems (given that I'm encountering one). I hope the images get updated in next days.

[dezgeg]

Waiting for kernel 4.18 to be released is also required, I think.

[hlolli]

@dezgeg I base my suspicion primarily on this reddit comment, and the comment under it
https://www.reddit.com/r/raspberry_pi/comments/88k5bl/rpi_3b_and_uboot_problems/dwlb5u6

but no stress, my rpi3+ is completly hobby related :Þ

[bkchr]

Does someone is able to boot the raspberry pi 3 with latest master branch?

I looked into the last generation that worked for me and I think that it is either glibc 2.27 or binutils 2.30 that prevents my raspberry pi from booting.

Here is my log of the failing boot:

<<< NixOS Stage 1 >>>

loading module dm_mod...
running udev...
kbd_mode: KDSKBMODE: Inappropriate ioctl for device
Gstarting device mapper and LVM...
[    1.969164] Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
[    1.969164] 
[    1.978476] CPU: 0 PID: 1 Comm: init Not tainted 4.16.8 #1-NixOS
[    1.984580] Hardware name: Raspberry Pi 3 Model B (DT)
[    1.989801] Call trace:
[    1.992301]  dump_backtrace+0x0/0x1c8
[    1.996025]  show_stack+0x24/0x30
[    1.999396]  dump_stack+0x9c/0xc0
[    2.002766]  panic+0x124/0x294
[    2.005872]  complete_and_exit+0x0/0x30
[    2.009771]  do_group_exit+0x40/0xa8
[    2.013406]  get_signal+0x280/0x5b0
[    2.016954]  do_signal+0x88/0x240
[    2.020325]  do_notify_resume+0xd8/0x130
[    2.024311]  work_pending+0x8/0x10
[    2.027774] SMP: stopping secondary CPUs
[    2.031763] Kernel Offset: disabled
[    2.035308] CPU features: 0x0802004
[    2.038850] Memory Limit: none
[    2.041963] ---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
[    2.041963] 
[    2.865264] random: crng init done

I also searched for the failure and some suggested that the SD card could be the problem. I have run fsck multiple times and it does not reported any big failures.

[bkchr]

I opened a pull request that fixes glibc 2.27 for raspberry pi: https://github.com/NixOS/nixpkgs/pull/40572

[qolii]

Urrrggghhhh. Is anyone willing to share a ready-to-go U-Boot for the RPi 3B+? I actually only got this board to bootstrap compiling U-Boot for my EspressoBin (since I can't for the life of me get aarch64 cross-compiling on NixOS to work for it from the guides I've seen); I saw that Raspberry Pi 3B was supported "out of the box" and thought it couldn't go wrong. Turns out the "+" is important, and now I'm stuck with two unbootable boards! I naively assumed the 3B and 3B+ were totally compatible.

I followed pretty much exactly the path of @timclassic (did you get your own bootstrapping process to work?), although I additionally tried copying the bcm2710-rpi-3-b-plus.dtb into the expected bcm283x-rpi-other.dtb, just in case. It then hangs trying to start the kernel (which may not be a surprise to anyone who knows what they are doing)

Anyhow, I'll be eternally grateful if anyone has:
1) A u-boot-rpi3.bin I can just write onto my SD card to get me going.
2) Instructions for how to cross-compile a U-Boot for AArch64 (from either AMD64 or ARMv7)
3) Some other magic.

Here's hoping!

[mestaritonttu]

@qolii did you try this for cross-compiling: https://github.com/ElvishJerricco/cross-nixos-aarch64

[timclassic]

@qolii I took the cowards way out and decided to wait until the upstream kernel supported more 3B+ features. I happen to have a couple of 3Bs that I'm using in the meantime. From reading other threads, I realized that I'd likely have network trouble next even if I got one my 3B+s to boot with the upstream kernel.

I'm sorry I don't have better news :(

[qolii]

@mestaritonttu, that looks really interesting, thanks. I have not tried it, nope. But I'll look at that.

@timclassic, hehe, that's exactly what I would have done :)

@dezgeg, you are a total hero! Works perfectly. Network is great, not seen any problems yet.

[sdevmaster]

I just am not able to get audio working, does it work for anyone?! Even with new image from @dezgeg

[florianjacob]

@sdevmaster no, as far as I know still no one has managed to get analog audio from a raspberry pi on NixOS.

[bkchr]

@sdevmaster: Are you using the latest upstream kernel? If yes, I get the analog interfaces to show up in Alsa with the following commit: https://github.com/bkchr/nixpkgs/commit/3c1b35e694d3a54c8de483780e58011c8e389999
If you use the raspberry pi kernel, you probably need to add dtparam=audio=on to your /boot/config.txt.

[bkchr]

I can also confirm, that the analog output is working for me.

[jfrankenau]

Using https://github.com/ElvishJerricco/cross-nixos-aarch64 I've sucessfully compiled an SD image for my Raspberry Pi 3 which works fine so far.

I'd like to use the lirc-rpi module however adding dtoverlay=lirc-rpi to the config.txt has no effect. I've also copied the overlays folder from the raspberrypifw package to /boot but this doesn't change anything.

[makefu]

@jfrankenau you may need to build your kernel with CONFIG_LIRC=y (see https://discourse.nixos.org/t/the-correct-way-to-override-the-latest-kernel-config/533 )

[jfrankenau]

@makefu, thank you for the quick answer. This may indeed be it. At least /proc/config.gz tells me that CONFIG_LIRC is not set. I will recompile my kernel and report back.

[jfrankenau]

@makefu, I've found out that with kernel 4.16 lirc-rpi is no longer supported. Instead there is a gpio-ir overlay which also doesn't need LIRC but instead directly registers as an input device which can be configured using ir-keytable which is much simpler and quicker in my opinion than setting up a new IR remote with LIRC.

For us however there are still a few hoops to jump through. Loading overlays using dtoverlay= in /boot/config.txt has to supported by the kernel (and/or U-boot?) which isn't the case for our mainline kernel. I believe with the Raspberry Pi kernel this should work, haven't tried it however.

What I did instead was to take the base device tree bcm2837-rpi-3-b.dtb and manually apply the gpio-ir.dtbo overlay using the dtmerge tool. I then put the resulting dtb file at /boot/nixos/*-linux-4.17.8-dtbs/broadcom/ replacing the original dtb. To be able to use dtmerge though, the dtb/dtbo files need to be compiled with symbol names which in turn means that I had to use the original Raspberry Pi kernel sources and run make DTC_FLAGS=-@ ARCH=arm dtbs in order to get dtb/dtbo files which are ready to be used by dtmerge.

So, I don't know now what the progress is on having native overlay support in NixOS but if using U-Boot or configfs doesn't work out this may be an alternative.

[florianjacob]

I found out that the audio issues boil down to the same essential problems as @jfrankenau explained: dtparam=audio=on is depending on device tree overlays as well, and is therefore totally ignored by the current vanilla kernel.

The kernel patch from @bkchr essentially enables dtparam=audio=on as a default, I can confirm it works.
Do you intend to upstream it as NixOS kernel patch?

[bkchr]

I'm not sure. I could upstream it to Nixos, but maybe we want to wait for upstream?

[florianjacob]

Does someone know whether dynamic device tree configuration is something that is in development for the upstream linux kernel? Could not find much information on that.

[bachp]

@florianjacob There is always discussion about it a Linux Conferences but there are some concerns about security and stability. I don't have any resources describing the status of upstream work in that direction.

[tuuzdu]

Why doesn't work bootdelay=0 in u-boot?
Parameters in kernelParams = [""] don't impact on boot process too.
How is possible disable interrupt on autoboot by serial input?

[bkchr]

Hey, does someone has problems with the latest kernel and wifi? Somehow my wifi is not working anymore..

[milnet2]

I have the same issue as @bkchr. After downgrading to boot.kernelPackages = pkgs.pkgs.linuxPackages_4_18; seems to work again.

[bkchr]

Yeah, I just switched one minor version back. There needs to be a regression upstream.

[florianjacob]

As the link to the audio patch is broken: The branch https://github.com/bkchr/nixpkgs/commits/raspberrypi contains the commit "Raspberry Pi audio", which is currently at https://github.com/bkchr/nixpkgs/commit/53939657a8609dfafea9e7e89aae4babedee1e03 .

[devhell]

Same WiFi issue here, there is no mention now in dmesg of brcmfmac getting loaded (or even recognized).

[bkchr]

Here is probably a fix for you problem. I will try to test it out, if my time permits it :)

[bkchr]

I can confirm, that the following commit fixes wifi with latest kernel for me:
https://github.com/bkchr/nixpkgs/commit/609e4bfe99f79b7b40dec07c53dc6e6718712543

[Mic92]

@bkchr we can add this to master, if it works for you.

[domenkozar]

Step by step guide for RBPI 3B+: https://github.com/zupo/nix

[anderslundstedt]

Has anyone succeeded in booting NixOS from USB without any SD-card?

To enable USB boot, I followed the instructions on

https://www.raspberrypi.org/documentation/hardware/raspberrypi/bootmodes/msd.md

If I flash Raspbian to an USB device then it boots but if I flash a NixOS install image to the same USB device it does not boot. I am using a "Sandisk Cruzer Fit 16GB" which is a compatible USB device according to

https://www.raspberrypi.org/blog/pi-3-booting-part-i-usb-mass-storage-boot/

[samueldr]

@anderslundstedt yes, I successfully booted NixOS from USB without SD card on a Raspberry Pi 3B (not +).

Could you elaborate on "it does not boot"?

With a specific USB drive (Cruzer Edge 32GB from Sandisk) it would fail with u-boot detecting zero USB devices and the error message "unable to get device descriptor (error -110)". I am not 100% positive, but I remember it worked with other drives, one I'm unsure of, and a Samsung 64GB BAR Plus.

Noting here, as it's relevant, the way the Raspberry Pi images is built is that it first boots into u-boot, which boots into the (mainline) kernel, so it's possible that a display that works fine with another Raspberry Pi distribution has troubles being initialized by our image (that's a certainty, I have a display here with issues).

If you have serial output, could you confirm that at least u-boot starts, and share the error message it returns?

Alternatively, could you confirm the image work on an SD card for you? That's a much more usual way to boot, and such is pretty well tested. If somehow your image doesn't boot on an SD card for you, there might be something else than USB boot at play here.

[anderslundstedt]

@samueldr thanks for offering your help, I now have somehow got USB boot to work. Unfortunately I do not know exactly what I did that made it work. But what I have now and that works is the following boot layout:

• /dev/sda1: vfat with bootable flag set
• /dev/sda2: ext4 with bootable flag set
• sda2 mounted at /
• /boot/ on the ext4 partition, with contents /boot/extlinux and /boot/nixos.
• sda1 unmounted but with contents
  bootcode.bin config.txt fixup_cd.dat fixup.dat fixup_db.dat fixup_x.dat start_cd.elf start_db.elf start.elf start_x.elf u-boot-rpi3.bin
  (Note that I removed the extlinux and nixos folders from sda1. I do not know whether this mattered.)

To get to the above stage without using an SD-card I had to do a lot of trial and error to get NixOS booting. In my case, I could boot from time to time when I had a USB keyboard plugged in. With only the USB stick plugged in (and sometimes with more devices plugged in) all I got was the same error message and result as @samueldr mentions above: "unable to get device descriptor (error -110)".

Here are some links that helped me get this to work:

• https://nixos.wiki/wiki/NixOS_on_ARM#Disable_use_of_.2Fboot_partition
• https://github.com/NixOS/nixpkgs/issues/22014#issuecomment-290152982
• https://github.com/NixOS/nixpkgs/issues/51924

[samueldr]

Hi, I'm willing to assume that none of that is required for USB boot, that the main issue is with u-boot and "unable to get device descriptor (error -110)", which might be intermittent with your setup. In my experience, simply using dd with the sd_image on a USB device is enough once the USB drive is known to work without issues with the current u-boot (or alternatively using a u-boot with the regression reverted).

Additionally, you should keep only the ext4 partition marked as bootable in a setup where you don't keep kernels and initrds in the FAT /boot partition. This is used by u-boot, once started, to determine whether it can or not look into that partition to find the files it will read to boot (here the extlinux config files).

[anderslundstedt]

Well, my current setup of NixOS on USB has so far always booted successfully. With the NixOS install image flashed to the same USB I am unable to boot at all ("unable to get device descriptor (error -110)") unless I plugin additional USB devices (in which case it randomly succeeds in booting). So I guess that for some reason something in my current setup helps with successfully booting.

Anyway, for me the problem is solved, at least for now. But I can provide additional info if that would be of any use.

Additionally, you should keep only the ext4 partition marked as bootable in a setup where you don't keep kernels and initrds in the FAT /boot partition. This is used by u-boot, once started, to determine whether it can or not look into that partition to find the files it will read to boot (here the extlinux config files).

As I understand it (partly based on https://github.com/NixOS/nixpkgs/issues/51924) the Raspberry Pi booting process reads the FAT partition, which in my case then triggers u-boot to start. So for this reason should it not be marked as bootable?

[samueldr]

The raspberry pi booting process ignores the "Boot" flag¹. The official documentation does not go in details about the partitions, the only details are:

It is no longer necessary for the first partition to be the FAT partition, as the MSD boot will continue to search for a FAT partition beyond the first one.
The boot ROM also now supports GUID partitioning and has been tested with hard drives partitioned using Mac, Windows, and Linux.

So basically, first FAT partition with bootcode.bin(?)² is used. In our case, this boots u-boot.

Finally, The wiki section about disabling the use of the FAT partition for /boot, describes the process this way:

Use fdisk or cfdisk to remove the bootable flag from the FAT32 partition, and set it for the ext4 partition

In the sample, it shows the original, then the updated partition table, where at each point only one of the partitions has the "Boot" flag set.

---

1. It is not defined as ignoring the "Boot" flag, but it is neither defined as checking for its presence in the documentation.
2. I am unsure about the "with bootcode.bin". The text makes it hard to know for sure whether it stops at the first FAT partition it finds, or goes through all FAT partitions.

[anderslundstedt]

Thanks for the explanation. I have now removed the bootable flag from the FAT partition and it still boots fine.

[erobl]

I tried to boot NixOS on my Raspberry Pi A+, but it hangs on the rainbow/color test boot screen. Does it need a different u-boot file? I checked the raspbian image but there's nothing there specifically for the A+, and it has the same processor as the B+. Are there plans to support it?

[samueldr]

@erobl maybe it does, but I wouldn't know for sure. Let's start with a simple question: do you have the Raspberry Pi 3 A+? (There is the Raspberry Pi 1 A+ too.) And another question, which image have you been using? The older images hosted by dezgeg are currently a bit out of date, and the upstream hydra has been producing images for the current stable (18.09) and unstable.

---

The "3-a-plus", let's call it, has no visible specific device tree for it in u-boot, but I wouldn't know for sure if it needs it; here what I mean comes from the current master Makefile:

dtb-$(CONFIG_ARCH_BCM283X) += \
    bcm2835-rpi-a-plus.dtb \
    bcm2835-rpi-a.dtb \
    bcm2835-rpi-b-plus.dtb \
    bcm2835-rpi-b-rev2.dtb \
    bcm2835-rpi-b.dtb \
    bcm2836-rpi-2-b.dtb \
    bcm2837-rpi-3-b.dtb

Though, it doesn't seem to be building a specific device tree for the 3-b-plus either, which I think works...

Additionally, if it booted up to u-boot, it would apparently be using a 3-a-plus device tree:

    [0xE] = {
        "3 Model A+",
        DTB_DIR "bcm2837-rpi-3-a-plus.dtb",
        false,
    },

Which would fail to load (but might still go through). The kernel currently only has rpi-3-b[-plus].dtb files for the "3" family (and the cm-3 files).

[samuel@bart:~]$ find /nix/store/*4.19* -iname '*bcm*pi*.dtb'
/nix/store/nzmsgxbqdn1m0wdw1vdxiy3wdb3q39sx-linux-4.19.4/dtbs/broadcom/bcm2837-rpi-3-b.dtb
/nix/store/nzmsgxbqdn1m0wdw1vdxiy3wdb3q39sx-linux-4.19.4/dtbs/broadcom/bcm2837-rpi-3-b-plus.dtb

So I'd say it is inconclusive (to me) until someone with the hardware can verify.

[erobl]

Yes, it's the Raspberry Pi 3 A+, I had a typo there. I used the image that was built on hydra, specifically nixos-sd-image-18.09.2102.38a99219f3c-aarch64-linux.img, this one. How would I know if it booted up to u-boot? I had it plugged into a serial to USB cable and it gave me no output.

Edit: I should probably mention that raspbian itself (which I have tested and does boot on my device) has no separate dtb files for the 3 A+, so it might be a case of just pointing to the right one.

[samueldr]

I can't find confirmation that it should work, so I'll assume that the lack of serial and HDMI output might point towards it being just different enough that Raspberry Pi 3B[+] support is not enough.

Though (now that I have confirmation it was indeed the 3B+) I have checked some more, and it looks like the 3 A+ thing was added in December to u-boot.

Let's build you a recent u-boot!!

---

Apply PR #55369 to a fairly recent nixpkgs checkout

curl -L https://github.com/NixOS/nixpkgs/pull/55369.patch | git apply

Cross-build u-boot (maybe the invocation is clumsy, but heck, it works)

nix-build -I nixpkgs=/Users/samuel/tmp/nixpkgs/nixpkgs -E 'let plat = (import <nixpkgs> {}).pkgsCross.aarch64-multiplatform; in plat.ubootRaspberryPi3_64bit'

Not sure if the build will be for more than u-boot itself, but if it's u-boot itself, don't even put the kettle one, it's going to end quickly.

~/tmp/nixpkgs/nixpkgs $ ls -l result/
total 452
-r-xr-xr-x 1 root root 462384 Dec 31  1969 u-boot.bin

Simply replace the u-boot.bin from the FAT partition with this one, and report back.

[erobl]

It seems it's still not working, there's no change from before or after. I'm assuming the u-boot.bin file it uses is u-boot-rpi3.bin, at least that's what it says on the config.txt.

[erobl]

I tried the image for the unstable channel and it boots with no changes, however, u-boot does fail to start the operating system.

The output when it boots is the following:

U-Boot 2019.01 (Jan 14 2019 - 22:02:36 +0000)

DRAM:  448 MiB
RPI 3 Model A+ (0x9020e0)
MMC:   mmc@7e202000: 0, sdhci@7e300000: 1
Loading Environment from FAT... *** Warning - bad CRC, using default environment

In:    serial
Out:   vidconsole
Err:   vidconsole
Net:   No ethernet found.
starting USB...
USB0:   scanning bus 0 for devices... Timeout poll on interrupt endpoint
Failed to get keyboard state from device 1ea7:0912
1 USB Device(s) found
       scanning usb for storage devices... 0 Storage Device(s) found
Hit any key to stop autoboot:  0
switch to partitions #0, OK
mmc0 is current device
Scanning mmc 0:1...
Found /extlinux/extlinux.conf
Retrieving file: /extlinux/extlinux.conf
793 bytes read in 3 ms (257.8 KiB/s)
------------------------------------------------------------
1:  NixOS - Default
Enter choice: 1:    NixOS - Default
Retrieving file: /extlinux/../nixos/qk6yc4hyb72xwfsz2kba0mk3976q6l8b-initrd-initrd
7054116 bytes read in 296 ms (22.7 MiB/s)
Retrieving file: /extlinux/../nixos/66n96qy788xvq3783m4gn1gciym7nb1g-linux-4.14.98-Image
27181568 bytes read in 1131 ms (22.9 MiB/s)
append: systemConfig=/nix/store/j8ysbg9626znmm18gjiqj65s3ra85ylr-nixos-system-nixos-19.03pre168466.47aad6e9b54 init=/nix/store/j8ysbg9626znmm18gjiqj65s3ra85ylr-nixos-system-nixos-19.03pre168466.47aad6e9b54/init loglevel=7 cma=32M console=ttyS0,115200n8 console=ttyAMA0,115200n8 console=tty0
Retrieving file: /extlinux/../nixos/66n96qy788xvq3783m4gn1gciym7nb1g-linux-4.14.98-dtbs/broadcom/bcm2837-rpi-3-a-plus.dtb
** Unable to read file /extlinux/../nixos/66n96qy788xvq3783m4gn1gciym7nb1g-linux-4.14.98-dtbs/broadcom/bcm2837-rpi-3-a-plus.dtb **
Skipping nixos-default for failure retrieving fdt
SCRIPT FAILED: continuing...
Card did not respond to voltage select!

Device 0: unknown device
No ethernet found.
missing environment variable: pxeuuid
Retrieving file: /extlinux/pxelinux.cfg/00000000
No ethernet found.
Retrieving file: /extlinux/pxelinux.cfg/0000000
No ethernet found.
Retrieving file: /extlinux/pxelinux.cfg/000000
No ethernet found.
Retrieving file: /extlinux/pxelinux.cfg/00000
No ethernet found.
Retrieving file: /extlinux/pxelinux.cfg/0000
No ethernet found.
Retrieving file: /extlinux/pxelinux.cfg/000
No ethernet found.
Retrieving file: /extlinux/pxelinux.cfg/00
No ethernet found.
Retrieving file: /extlinux/pxelinux.cfg/0
No ethernet found.
Retrieving file: /extlinux/pxelinux.cfg/default-arm-bcm283x
No ethernet found.
Retrieving file: /extlinux/pxelinux.cfg/default-arm
No ethernet found.
Retrieving fileextlinux/pxelinux.cfg/default
No ethernet found.
Config file not found
No ethernet found.
No ethernet found.
U-Boot> 

It seems like ** Unable to read file /extlinux/../nixos/66n96qy788xvq3783m4gn1gciym7nb1g-linux-4.14.98-dtbs/broadcom/bcm2837-rpi-3-a-plus.dtb ** is the part where it fails to boot.

The new u-boot version has this line as ** Unable to read file /extlinux/../nixos/66n96qy788xvq3783m4gn1gciym7nb1g-linux-4.14.98-dtbs/broadcom/bcm283x-rpi-other.dtb **, which means the new version is probably required.

Edit: I renamed whatever/bcm2837-rpi-3-b.dtb to whatever/bcm2837-rpi-3-a-plus.dtb and it booted to the normal shell. I'm not sure what would let us point to that, since presumably there's a bcm2837-rpi-3-b-plus.dtb to which u-boot points to bcm2837-rpi-3-b.dtb, but at least it works now.

[samueldr]

You're right! I had assumed incorrectly that u-boot would continue without a device tree blob to load.

Retrieving file: /extlinux/../nixos/66n96qy788xvq3783m4gn1gciym7nb1g-linux-4.14.98-dtbs/broadcom/bcm2837-rpi-3-a-plus.dtb
** Unable to read file /extlinux/../nixos/66n96qy788xvq3783m4gn1gciym7nb1g-linux-4.14.98-dtbs/broadcom/bcm2837-rpi-3-a-plus.dtb **

This is... uh... annoying. Especially since for the moment, the device trees directory is a bit hardcoded. Though, once mainline adds the proper device tree, it should continue fine, or otherwise, fixing the build to allow a custom dtbDir and adding what you need would work too; the latter option would be needed for things like customising the device trees anyway, and might be due to be done.

[erobl]

The wifi also doesn't work, which is a big problem since the device only has wifi in terms of network connection. I'm not sure if network.wireless.enable does anything other than starting wpa_supplicant on boot, but any command related to using the wlan0 interface fails, for example:

$ iwlist wlan0 scan
[time] brcmfmac: brcmf_run_escan: error (-4)
[time] brcmfmac: brcmf_cfg80211_scan: scan error (-4)
wlan0 Interface doesn't support scanning : Interrupted system call

It might have something to do with this incantation but I can't install it because I have no network connection.

[erobl]

@samueldr I looked around the code that generates the *.dtb files, and looking at this line, it copies the bcm2708-rpi-b-plus.dtb to bcm2835-rpi-a-plus.dtb, for the old model. This does work on my model (at least using it manually), so maybe someone could commit the extra line that adds copyDTB bcm2710-rpi-3-b-plus.dtb bcm2837-rpi-3-a-plus.dtb to that file? It's an easy fix, at least for now.

[samueldr]

This is only true when using the non-mainline linuxPackages_rpi which uses the raspberry pi foundation kernel instead of the mainline Linux kernel. This wouldn't fix the default built image, and wouldn't fix it for users using the mainline kernel. (In case you didn't know, the aarch64 image is generic, it can be used across a range of different aarch64 SBCs.)

Though, I'm sure that if you were to open a PR with this addition, it would be an easy merge.

[bkchr]

@erobl we also copy these files in boot.loader.raspberrypi. As far as I remember.

[bkchr]

Hey, did anyone knows how to enable overlays with the upstream kernel?

[samueldr]

@bkchr funnily, I'm working on figuring this all out these current days.

Leaving the interesting bits I found here if it helps you figure things out quicker than I am

---

• https://github.com/ikwzm/dtbocfg
• WIP derivation

Device Tree Overlay is a mechanism that enables dynamic loading/unloading of a new device tree blob on top of the kernel device tree. Device Tree Overlay was first introduced in Linux Kernel version 3.19.

With this project I am able to make things look like they'll work. I was at the cusp of making this work, but didn't have time to finish working on it.

Using the device-native DTB, which the mainline kernel can boot from, I was able to use the downstream spi0-cs-overlay.dts to make the kernel react and add the SPI interface, but that setup failed with error -517. I am suspecting that using the built-in DTB of the Raspberry Pi is the issue, and now switched to a test image which uses u-boot which uses the mainline built DTB.

Though this fails, and someone else apparently figured things out:

• https://github.com/notro/tinydrm/wiki/board:-Raspberry-Pi#device-tree-overlay-in-mainline

I'm currently "here", in the process, just about to rebuild the device trees with the compiler configured to output symbols, and probably will digress in adding support to the extlinux generation for an alternative source for the DTBs, as of right now it's pretty hardcoded.

---

UPDATE, late 2019

For anyone seeing this comment, #60422 has been merged. The dtbs are less hardcoded. This is a nixos-rebuild / boot time thing.

[bkchr]

@samueldir ahh nice. I also was at the point that the symbols are missing and added a patch to solve this :D After that I realised that the upstream kernel does not contain the stuff in dtb that I need. So, I switched to the rpi kernel. This one contains the symbols and the other stuff that is required. The overlay is still not loaded by the bootloader. So, I currently switched to use dtmerge to merge the overlay and the kernel dtb and putting the resulting dtb into the boot partition.

[fdietze]

I wanted to note, that using an android phone with USB tethering works great as a wifi replacement.

[ldesgoui]

I don't have an SD card port on my PC, I flashed the sd card ISO on an USB and it boots just fine, I have managed to install NixOS by copying the contents of the USB to the SD card using dd. Is it possible to go through the normal NixOS installation process instead? I have tried with this config, it would not boot.

boot.kernelPackages = pkgs.linuxPackages_latest;
boot.kernelParams = [ "cma=32M" "console=tty0" ];
boot.consoleLogLevel = 7;

boot.loader.generic-extlinux-compatible.enable = true;
boot.loader.grub.enable = false;
boot.loader.raspberryPi = {
    enable = true;
    uboot.enable = true;
    version = 3;

    firmwareConfig = ''
        gpu_mem=16
    '';
};

fileSystems."/" = {
    device = "/dev/disk/by-label/nixos";
    fsType = "ext4";
};

EDIT: Alright, it seems like u-boot won't boot off ext4, I just had to add a /boot vfat partition and it boots fine.

[samueldr]

EDIT: Alright, it seems like U-Boot won't boot off ext4, I just had to add a /boot vfat partition and it boots fine.

To add some precision, the raspberry pi firmware files, and what they boot, have to be on a FAT32 partition. When the Raspberry Pi is used in conjunction with U-Boot, this means that U-Boot has to be on the FAT32 partition.

U-Boot itself will look at and boot ext4 partitions, as long as the partition is marked as bootable.

Ref for the Raspberry Pi boot flow

• https://www.raspberrypi.org/documentation/hardware/raspberrypi/bootmodes/bootflow.md

[AleXoundOS]

To add some precision, the raspberry pi firmware files, and what they boot, have to be on a FAT32 partition. When the Raspberry Pi is used in conjunction with U-Boot, this means that U-Boot has to be on the FAT32 partition.

I'm not sure what exactly do you mean, but it's possible to get rid of FAT32 partition completely for NixOS on Raspberry Pi (I confirm, it works for me). Also, there is a https://github.com/NixOS/nixpkgs/issues/22014#issuecomment-290152982 in this thread.

[samueldr]

Not completely, the FAT32 partition must exist to contain the firmware files for the Raspberry Pi, and u-boot in our use case. Though, the kernels and generations that u-boot will look at can be on the ext4 partition and is the new default on the sd image since recently on unstable, and will be part of 19.09.

[AleXoundOS]

Ah, thank you for the clarification. Indeed the partition is still there, but I guess once we boot NixOS has no knowledge of it.

[samueldr]

Exactly, with the recent changes, it acts just as if it was an opaque blob, except that it's a FAT32 formatted opaque blob.

[anderslundstedt]

I cannot figure out how to get analog audio output working on the Raspberry Pi 3B+. HDMI audio output works. Any advice?

UPDATE: Found the previous audio discussion in this thread. (GitHub apparently hides old messages, so CMD+F 'audio' did initially not give any results). Now trying out https://github.com/bkchr/nixpkgs/commit/53939657a8609dfafea9e7e89aae4babedee1e03 to see if that helps. Currently compiling ...

[anderslundstedt]

Update on my previous comment (regarding analog audio output): Compilation failed (probably out of memory or disk space) and I decided to give up.

Per the above, https://github.com/bkchr/nixpkgs/commit/53939657a8609dfafea9e7e89aae4babedee1e03 does solve the problem. Should we not try to merge those changes?

[bkchr]

I'm still in hope that this is merged in mainstream, but yeah :(

[mogria]

Wifi works but the following option needs to be enabled for the wireless driver to be installed:

hardware.enableRedistributableFirmware = true;
# networking.wireless.enable = true; # for wpa_supplicant

(you probably also need to allow non-free/proprietary packages for this to work)

After running nixos-rebuild swich and rebooting the interface wlan0 showed up on my raspberry pi when running ifconfig

I found this on the NixOS Community Wiki: https://nixos.wiki/wiki/NixOS_on_ARM/Raspberry_Pi

[c00w]

I opened https://github.com/NixOS/nixpkgs/pull/69231 which allows the rpi3a+ to boot with the linux_rpi kernel package.

[stale[bot]]

Thank you for your contributions.
This has been automatically marked as stale because it has had no activity for 180 days.
If this is still important to you, we ask that you leave a comment below. Your comment can be as simple as "still important to me". This lets people see that at least one person still cares about this. Someone will have to do this at most twice a year if there is no other activity.
Here are suggestions that might help resolve this more quickly:

1. Search for maintainers and people that previously touched the
   related code and @ mention them in a comment.
2. Ask on the NixOS Discourse. 3. Ask on the #nixos channel on
   irc.freenode.net.

[AleXoundOS]

Still important to me.

[Mic92]

Is there anything still not working on the rpi3?

[mebubo]

Is there anything still not working on the rpi3?

HDMI only working on "cold" boot:

https://github.com/NixOS/nixpkgs/issues/82455#issuecomment-609423999

[samueldr]

The B+ is different enough that it shouldn't block this issue from being closed. Considering there is already an issue tracking its issues.

Unless this can be reproduced also on the 3B. This issue was opened well before the 3B+ was released.

[JohnAZoidberg]

I haven't had any problems the last time I reinstalled and booted my RPi3. If anyone else has, please let us know.