Nixpkgs: NixOS UEFI installation instructions incomplete

[lukego]

I installed NixOS from USB key in UEFI mode. There were a few things that were not clear to me from the installation instructions:

1. That I have to create a boot partition.
2. What is a reasonable size for the boot partition (default/min/max).
3. That I have to mount /dev/sdaN /boot myself or else nixos-install will fail.

Otherwise I have been _extremely_ impressed with the installation procedure and documentation. (I previously did a non-UEFI install and that was flawless. For some reason this second server did not offer to boot the USB stick in BIOS mode and this seemed to force me to install NixOS under UEFI.)

[heydojo]

I would change :

Instead of fdisk, you should use gdisk to partition your disks. You will need to have a separate partition for /boot with partition code EF00, and it should be formatted as a vfat filesystem.

To something similar to :

Instead of fdisk, you should use gdisk to partition your disks. You will need to have a separate partition for /boot with partition code EF00, it should be formatted as a vfat (fat32) filesystem, be of a minimum size of 100MB and the entire partition table layout of the disk must be in gpt format and not msdos. The gparted partitioning application which can be found on the NixOS desktop livecd has the ability to create gpt partitioning layouts.

And :

After having mounted your installation partition to /mnt, you must mount the boot partition to /mnt/boot.

To :

After having mounted your installation partition to /mnt, you must first mount the EFI system partition you created using the mount point /mnt/boot before running the nixos-install program.

In :
https://nixos.org/nixos/manual/sec-installation.html#sec-uefi-installation
but I do not have or want wiki access. Maybe someone who has; will see this and update. EFI is pretty much the normal way to install operating systems on any X86_64 computer now and has been for some time. So it is pretty important the information is concise and accurate.

Anyone have any opinions about whether it is better to recommend that the EFI firmware partition should be mounted as /boot/EFI and not mounted as the entire boot directory? Is it a gummiboot thing?
I only mount /boot/EFI on my other Fedora flavoured system but it uses grub2-efi. Not very familiar with gummiboot.

[wkennington]

The problem with gummiboot is that it supports reading far fewer
filesystems. I personally would recommend doing a dual bios / efi install
with grub and gpt. So you use gdisk to create a 2M bios boot
partition(ef02), 200MB or more /boot which is an efi system partition(ef00)
formatted as vfat, then the rest of the disk in any configuration. This
should give you maximal machine compatibility with barely any space
overhead. You can then use the same configs for both efi and bios.

On Sat, Sep 19, 2015, 19:06 Tony [email protected] wrote:

I would change :

Instead of fdisk, you should use gdisk to partition your disks. You will
need to have a separate partition for /boot with partition code EF00, and
it should be formatted as a vfat filesystem.

To something similar to :

Instead of fdisk, you should use gdisk to partition your disks. You will
need to have a separate partition for /boot with partition code EF00, it
should be formatted as a vfat (fat32) filesystem, be of a minimum size of
100MB and the entire partition table layout of the disk must be in gpt
format and not msdos. The gparted partitioning application which can be
found on the NixOS desktop livecd has the ability to create gpt
partitioning layouts.

And :

After having mounted your installation partition to /mnt, you must mount
the boot partition to /mnt/boot.

To :

After having mounted your installation partition to /mnt, you must first
mount the boot partition using the mount point /mnt/boot before running the
nixos-install program.

In :
https://nixos.org/nixos/manual/sec-installation.html#sec-uefi-installation
but I do not have or want wiki access. Maybe someone who has; will see
this and update. EFI is pretty much the normal way to install operating
systems on any X86_64 computer now and has been for some time. So it is
pretty important the information is concise and accurate.

Anyone have any opinions about whether it is better to recommend that the
EFI firmware partition should be mounted as /boot/EFI and not mounted as
the entire boot directory? Is it a gummiboot thing?
I only mount /boot/EFI on my other Fedora flavoured system but it uses
grub2-efi. Not very familiar with gummiboot.

—
Reply to this email directly or view it on GitHub
https://github.com/NixOS/nixpkgs/issues/9096#issuecomment-141727598.

[heydojo]

pkgs.grub2_efi exists so maybe the instructions should describe the installation procedure for installing NixOS using that. Especially seeing as there is also a TPM module available and grub2 supports many more file systems than gummiboot.
I don't subscribe to the dual setup as I have never needed to do it since EFI arrived and believe strongly in the technical advantages of EFI (even if it is not perfect.)

[fuzzy-id]

I stumbled over this issue, too. I was accustomed from other distros to have a separate /boot and /boot/efi partition…

I think the changes to the manual as proposed by heydojo should be merged as soon as possible.

Switching to grub is a process which involves further steps and should thus be discussed in a separate ticket/issue.

[TTimTT]

I have been hanging around in irc but didn't found any help though,

I realised that there is a boot.loader.grub.efiSupport option existing, Although when calling nixos-install, grub_legay is used and not grub_efi.

This can be force by overriding pkgs: grub = grub_efi.
Though it seems dirty.

Is it a new feature? I have spent few days now on booting NixOs with grub efi in an easy way without any success and was wondering if there was any "in progress" documentation?

[wkennington]

If you make /boot an esp formatted as fat and set
boot.loader.grub.efiSupport = true and set boot.loader.grub.device =
"nodev" it should produce an efi only install.
On Sat, Oct 24, 2015 at 10:50 AM TTimTT [email protected] wrote:

I have been hanging around in irc but didn't found any help though,

I realised that there is a boot.loader.grub.efiSupport option existing,
Although when calling nixos-install, grub_legay is used and not grub_efi.

This can be force by overriding pkgs: grub = grub_efi.
Though it seems dirty.

Is it a new feature? I have spent few days now on booting NixOs with grub
efi in an easy way without any success and was wondering if there was any
"in progress" documentation?

—
Reply to this email directly or view it on GitHub
https://github.com/NixOS/nixpkgs/issues/9096#issuecomment-150837057.

[CMCDragonkai]

According to this http://askubuntu.com/questions/660023/how-to-install-ubuntu-14-04-64-bit-with-a-dual-boot-raid-1-partition-on-an-uefi/660038?noredirect=1#comment1018427_660038 it shows an easy way to setup EFI System Partition on multiple drives.

This means I'm going to ignore BIOS booting and will not have a biosboot partition. I will try putting ESP on /boot/efi. While /boot will just be on the main data partition which will be handled by ZFS.

So with 2 drives, this means I will only need to occasionally sync the /boot/efi only when I'm changing the bootloader itself. If I update the kernels which will be /boot/kernels that will not require any syncing of the ESP across the disks.

How would boot.loader.grub.efiSupport deal with such a configuration? Or what about gummiboot?

[kcomplexes]

I'm trying to dualboot Windows 10 with NixOS. W10 is currently installed, and I'm quite unsure on how to proceed here. I've searched the documentation of NixOS, Gentoo and Ubunutu; looked through general UEFI documentation, and scoures stackexchange, reddit and other forums.

What needs to be done to install NixOS from USB to my single drive Windows 10 Thinkpad. I want to keep W10 as my default and avoid adding unnescary folders and options during boot.

Do I use Grub2? Gummiboot? EFI-boot-stub? REFined? Do I need a seperate /boot partition?

Thanks

[domenkozar]

I wonder, is separate /boot partition really required for UEFI?

[domenkozar]

@shlevy might know :)

[CMCDragonkai]

The requirements for /boot is dependent on the EFI firmware. However there will always be a separate EFI System Partition that needs to be FAT32.

[shlevy]

@domenkozar Technically your firmware is allowed to support other filesystems, but in practice I don't know of any that support, say, ext2 (Macbooks support hfs, of course). All support fat32. You need to have your kernel and bootloader on a firmware-supported filesystem marked as an EFI system partition to do EFI boot.

For what it's worth I'm not sure what exactly this issue is asking about. What needs to change, specifically?

[domenkozar]

@shlevy we should update NixOS manual to include instructions that /boot needs a separate filesystem. It's currently not noted anywhere.

[shlevy]

In nixos/doc/manual/installation/installing-uefi.xml

<listitem>
    <para>Instead of <command>fdisk</command>, you should use
    <command>gdisk</command> to partition your disks. You will need to
    have a separate partition for <filename>/boot</filename> with
    partition code EF00, and it should be formatted as a
    <literal>vfat</literal> filesystem.</para>
  </listitem>

[jokogr]

The mountpoint could be set to a path other than /boot, it's set in boot.loader.efi.efiSysMountPoint.

[obadz]

See 1c9ac8aa & eda4f5d

[shlevy]

I'm still not sure why this issue exists. As I posted, there's already instructions about this in the manual.

[fuzzy-id]

@shlevy Yeah, I just reread the instructions and was quite astonished why I ran into this issue when I installed NixOS for the first time. Then I reviewed the entire installation instructions. I guess the problem was/is the overall structure. First, we explain the entire installation process in general. Then we give a few bullet points on UEFI. Then we explain how to boot from USB and from PXE. After that another section on installation in a VirtualBox.

We should definitely improve on this. I hope I find some time to restructure the documentation in the next couple of days.

[sboosali]

@obadz for the size of the boot partition, where does the number 100 MB come from? I found 512 MB as the minimum: https://wiki.archlinux.org/index.php/partitioning#Single_root_partition

(I am new to this stuff, so I might be completely mistaken, it's my first time dual booting any distribution in years)

[obadz]

@sboosali, nobody seems to agree on minimum size. https://docs.fedoraproject.org/en-US/Fedora/23/html/Installation_Guide/sect-installation-gui-manual-partitioning-recommended.html says 50MB for instance. 128MB has definitely worked for me.

[globin]

@obadz is your commit referenced here ready to be merged?

[obadz]

@globin, not really: https://github.com/NixOS/nixpkgs/pull/18731#issuecomment-248244929

[bledari]

Still open, really?

[grahamc]

Hi, @bledari, I'm not sure why you say that. Can you help me understand what you're surprised by?

As noted elsewhere in this discussion, the instructions _are_ complete. A follow-up comment indicates the issue is now with organization and flow of the documentation, _not_ an issue of missing documentation.

[dalaing]

I just put together this and then stumbled over this issue while wondering what I should say about the /boot partition.

I'm considering reworking my PR to contain suggestions about what to do with the /boot partition, and to merge the BOIS / UEFI instructions, with the appropriate if-then-else bits of information at the relevant steps. I figure that would help with the flow of information for the two main methods of installation.

My understanding is that fdisk now works for setting up UEFI boot partitions, so that probably will probably help as well.

[bjornfor]

parted can also be used for both mbr and gpt. (I use that when installing nixos.)

[dalaing]

For sure. I'm mostly trying to minimize the changes from the existing docs, and so stuck with fdisk since it was already in there.

[shlevy]

Once again, voting to close.

[backuitist]

Would be nice to have instructions for amending an existing EFI partition. I want to install NixOS along with Ubuntu which previously configured the EFI partition to run grub on its partition (at least that's how I understand it).

In the existing EFI partition, I can see EFI/ubuntu/grub.cfg with the following:

search.fs_uuid 32bc63c8-a61d-4a8c-bbe8-e9978541ae84 root 
set prefix=($root)'/boot/grub'
configfile $prefix/grub.cfg

Should I amend this file?

[CMCDragonkai]

There are 4 things that aren't clearly explained (and have controversial solutions).

1. What to do when you have multiple drives and you want multiple EFI System Partitions to be able to boot off (boot partition redundancy). How to maintain sync between them (manual cloning/mdadm <= 1.0). And whether to do this eagerly (at nixos-rebuild) or lazily (after a successful boot). There's no nixos configuration that supports all these kinds of solutions. (Even for efibootmgr which needs to be used to allow for the possibility of booting for the redundant boot partitions, and the normal bootloader installation module in NixOS doesn't know about these extra partitions (and I couldn't find config that allows you to declaratively specify multiple boot partitions)).
2. How big your EFI system partition should be, this is complicated by the fact that when you run nixos-rebuild, old boot partition generations are not deleted/nor garbage collected, your EFI system partition could get full, and your nixos-rebuild just fails. It's not clear to a new user what to do to fix this problem. I've encounters this in the past, and my EFI System Partition is now 1GB to reduce the chance of meeting this problem. Also I have forgotten how you actually force garbage collection on a full EFI system partition.
3. How the above 3 interacts with LUKS encryption + ZFS or native ZFS encryption. And initrd ssh support.
4. How to recover from corrupted bootloaders. Turns out you need a separate USB with the latest NixOS installed on it. Boot from the USB. Get your filesystem ready (either by decrypting them if necessary and putting into a zpool or logical volume) and mount it at /mnt. You then need to use nixos-install --chroot Which will switch roots into the /mnt, and make it as if you are booted into your system normally, using the --chroot call short circuits the script and doesn't do any other installation. You may also need to use NIXOS_CONFIG environment variable to point to the actual configuration.nix you're using. Finally you may need to mkdir -p /run/user/0, as it hasn't created the run directory for the root user. From here you can run nixos-rebuild boot with NIX_PATH set correctly (if it was custom set), and rebuild your bootloader partition. If this is successful (you will most likely need a network connection), then you can reboot using your fixed boot partitions.

[bjornfor]

What to do when you have multiple drives and you want multiple EFI System Partitions to be able to boot off (boot partition redundancy). How to maintain sync between them (manual cloning/mdadm <= 1.0).

Using bootctl (systemd-boot) and mdadm does seem to work, but it requires some manual steps: https://lists.freedesktop.org/archives/systemd-devel/2017-December/039962.html (Thread subject: "bootctl install" on mdadm raid 1 fails).