Nixpkgs: QA ISO images

[worldofpeace]

Describe the bug
Some amount of manual QA is needed to verify these images are in good shape. In 20.03 we had a pretty silly bug slip through and got dragged on HN :rofl:

Even without that as encouragement this is a good idea.

The main goal is to determine the following:

It needs to be verified that all ISO's that are using a release blocking desktop boot cleanly in their supported configurations. They must boot live when written to a USB stick with any of our supported methods documented in the NixOS manual.

I've drafted some criteria to look out for when testing a desktop. The version for this exact issue can be seen at this gist.
After having testing please comment with a checklist on the items like

- [x] Shutdown, reboot, login, logout
...

The following images need testing:

• [x] Plasma5 ISO
• [x] GNOME ISO

Here are links to the latest images:
https://channels.nixos.org/nixos-20.09/latest-nixos-plasma5-x86_64-linux.iso
~https://channels.nixos.org/nixos-20.09/latest-nixos-gnome-x86_64-linux.iso~

The gnome iso isn't added to the channel atm:
You can get it from an eval of it on hydra: https://hydra.nixos.org/job/nixos/release-20.09/nixos.iso_gnome.x86_64-linux
picking a build https://hydra.nixos.org/build/128080348
and selecting Link from Build Products.

Remaining Q.A tasks

Plasma

• [x] Printing https://github.com/NixOS/nixpkgs/issues/98536
• [ ] Keyboard layout configuration
  
  
  
  • [ ] unlocking encrypted storage volumes
  
  
  • [x] console - this is tested automatically
  
  
  • [ ] When logging in via the default login manager for a release-blocking desktop
  
  
  • [ ] After logging in to a release-blocking desktop, if the user account does not have its own keyboard layout configuration for that desktop (if there is such a user/desktop-specific configuration, it must be used when that user logs in to that desktop)
  
  
• [ ] Desktop Keyring
• [ ] User Switching?

GNOME

• [x] configured keyboard layout (using nixos options) is used when unlocking encrypted storage volumes
• [x] Desktop Keyring

[davidak]

This is very important! I plan to test the images.

Very good that you also document the process.

Does "Desktop Release Blocking Requirements" mean testing the live environment or a new install?

We should extend it with instructions to how test these cases. Like "User switching" should provide a config with multiple test users. In the end, it should be a checklist that you can work off without thinking about it.

We should also test install und upgrade.

(the link is for 20.03)

https://channels.nixos.org/nixos-20.09/latest-nixos-plasma5-x86_64-linux.iso
https://channels.nixos.org/nixos-20.09/latest-nixos-gnome-x86_64-linux.iso (don't exist)

https://releases.nixos.org/nixos/20.09/nixos-20.09beta538.0cfe5377e89

---

WIP

Desktop Release Blocking Requirements

Tested using nixos-plasma5-20.09beta538.0cfe5377e89-x86_64-linux.iso
on Lenovo Thinkpad X230

Shutdown, reboot, login, logout

Shutting down, rebooting, logging in and logging out must work using standard console commands and the mechanisms offered (if any) by all release-blocking desktops.

• [x] shutdown now
• [X] reboot
• [x] login?
• [x] logout?

Using GUI

• [x] shutdown
• [x] reboot
• [x] logout
• [x] login

what is command for login and logout?

Required Applications

It must be possible to run a browser and a terminal applications from all release blocking desktops.

• [x] Terminal
• [x] Browser

Desktop Background

The desktop background should be different than the last release.

• [X] KDE has a new default

last: https://distrowatch.com/images/screenshots/nixos-20.03-menu.png

Artwork

Proposed artwork for the new NixOS release should be used as the background on release-blocking desktops and there should be a new logo.

KDE uses it's default background i guess, not NixOS artwork :exclamation:

where should be a new logo? i see the default NixOS logo in the info app, but i think that is good?

the logo seems squashed tho

Release-blocking images must boot

It needs to be verified that all ISO's that are using a release blocking desktop boot cleanly in their supported configurations.

They must boot live when written to a USB stick with any of our supported methods documented in the NixOS manual.

USB Drive created with dd on linux (NixOS)

• [X] boots on BIOS
• [x] boots on EFI

I don't have an up to date Mac to test.
I don't want to do the obscure install from another linux instructions. How is that officially supported?

Release-blocking images must not be excessively large

TODO: define this? We can easily get comparisons of how large an image was in the release prior.

I guess up to 2 GB is OK? Better near to 1 GB for slow downloads.
It should not be much bigger than the previous without good reason.

got numbers with ls -lah, file browser shows different

nixos-minimal-20.03.3087.b4db68ff563-i686-linux.iso 539M
nixos-minimal-20.09beta538.0cfe5377e89-i686-linux.iso 563M

nixos-minimal-20.03.3087.b4db68ff563-x86_64-linux.iso 552M
nixos-minimal-20.09beta538.0cfe5377e89-x86_64-linux.iso 576M

nixos-plasma5-20.03.3087.b4db68ff563-x86_64-linux.iso 1.2G
nixos-plasma5-20.09beta538.0cfe5377e89-x86_64-linux.iso 1.3G

nixos-20.03.3087.b4db68ff563-x86_64-linux.ova 1.4G
nixos-20.09beta538.0cfe5377e89-x86_64-linux.ova 1.5G

first release
nixos-gnome-20.09beta980.1b40fb34fdb-x86_64-linux.iso 1.4G

• [X] seems OK

does hydra have a graph for that? yes

https://hydra.nixos.org/job/nixos/release-20.09/nixos.iso_minimal.i686-linux#tabs-charts
https://hydra.nixos.org/job/nixos/release-20.09/nixos.iso_minimal.x86_64-linux#tabs-charts
https://hydra.nixos.org/job/nixos/release-20.09/nixos.iso_plasma5.x86_64-linux#tabs-charts
https://hydra.nixos.org/job/nixos/release-20.09/nixos.iso_gnome.x86_64-linux#tabs-charts

it takes very long to load

Working sound

Installed system should be able to play back sound

• [X] works

open youtube in browser

Non-crashing desktop

Not any part of the release blocking desktop should crash when interacted with, on startup, or be non-functional.

• [X] haven't noticed any issues, but also not used exessively

Automatic mounting

Automatic mounting of removable media on insertion must work in release-blocking desktops.

• [X] works

maybe it's actually mounted when opened in file browser, but that makes sense

Printing

Printing must work in release-blocking desktops on at least one printer available to those performing validation testing.

i don't have a printer

nix-env

Software should be able to be installed, removed, and updated with nix-env.

• [X] install
• [X] update
• [X] remove

[nixos@nixos:~]$ nix-env -iA nixos.htop nixos.micro
installing 'htop-3.0.1'
installing 'micro-2.0.5'
these paths will be fetched (3.58 MiB download, 12.68 MiB unpacked):
  /nix/store/86ix01ka3b9z055a0mhhwhkpcziiy9bf-htop-3.0.1
  /nix/store/q2jca1wkyd7dkr9gg6bcdlhwxcli9l3m-micro-2.0.5
  /nix/store/r2wvgnr54vmwnjvzyqdixv8xbn362jgh-mailcap-2.1.48
copying path '/nix/store/r2wvgnr54vmwnjvzyqdixv8xbn362jgh-mailcap-2.1.48' from 'https://cache.nixos.org'...
copying path '/nix/store/86ix01ka3b9z055a0mhhwhkpcziiy9bf-htop-3.0.1' from 'https://cache.nixos.org'...
copying path '/nix/store/q2jca1wkyd7dkr9gg6bcdlhwxcli9l3m-micro-2.0.5' from 'https://cache.nixos.org'...
building '/nix/store/7d0hzl82r31ychwj8bzzascrkjsaap28-user-environment.drv'...
created 10 symlinks in user environment

[nixos@nixos:~]$ sudo nix-channel --update
unpacking channels...
created 1 symlinks in user environment

[nixos@nixos:~]$ nix-env -u
upgrading 'htop-3.0.1' to 'htop-3.0.2'
these paths will be fetched (0.08 MiB download, 0.22 MiB unpacked):
  /nix/store/xl6nwhwrb5mrzrb0p1i7c85w2l4f20lj-htop-3.0.2
copying path '/nix/store/xl6nwhwrb5mrzrb0p1i7c85w2l4f20lj-htop-3.0.2' from 'https://cache.nixos.org'...
building '/nix/store/4mvm9y6i3z4gcqm6llcns901k1q2xa8j-user-environment.drv'...
created 10 symlinks in user environment

Keyboard layout configuration

If a particular keyboard layout has been configured for the system, that keyboard layout must be used:

• When unlocking encrypted storage volumes during boot (but see footnotes)
• When logging in at a console
• When logging in via the default login manager for a release-blocking desktop
• After logging in to a release-blocking desktop, if the user account does not have its own keyboard layout configuration for that desktop (if there is such a user/desktop-specific configuration, it must be used when that user logs in to that desktop)

needs install i guess
please provide instructions

Desktop Keyring

Saving passwords to and retrieving passwords from the default keyring must work for all release-blocking desktops.

please provide instructions

User switching

User switching must work using the mechanisms offered (if any) by all release-blocking desktops in their default configuration.

needs instructions

[worldofpeace]

Hmm, I will see what the link to the gnome image is. It should be distributed now.

[worldofpeace]

Okay, it's not being put in the channel :frowning: https://releases.nixos.org/nixos/20.09/nixos-20.09beta977.ad3a5d5092e

[worldofpeace]

I opened an issue, we need to have that in the channel.
I will get a link from hydra for now in the issue description.

[rissson]

USB Drive created with dd on linux (NixOS)

You can tick "boots on EFI" I tried that last Saturday :D

[edolstra]

FWIW I did a successful installation from USB using a Plasma5 ISO. Didn't test it very systematically but the desktop environment and wireless (nmcli) seemed to work fine.

The only issue was that the initrd couldn't mount the root filesystem automatically because I didn't label the USB stick, but I think that's a long-standing issue.

[davidak]

The only issue was that the initrd couldn't mount the root filesystem automatically because I didn't label the USB stick, but I think that's a long-standing issue.

how have you created the stick? i think this don't happen when you follow the manual und use dd

[birlorg]

I just did a successful installation today from USB using the Plasma5 ISO(https://channels.nixos.org/nixos-20.09/latest-nixos-plasma5-x86_64-linux.iso)

Worked fine used dd from a macOS machine to write the ISO to a USB stick, didn't have any trouble finding the root FS.

Thanks everyone for your work so I could have happy installs!

[worldofpeace]

So here are the last few items that need testing in the plasma5 image, I seemed to have missed these from @davidak review:

• [ ] Printing
• [ ] Keyboard layout configuration
• [ ] Desktop Keyring
• [ ] User Switching

The points that had needs instructions, I will try to link to Plasma docs:

• https://docs.kde.org/index.php?language=en&package=kde-workspace
• https://docs.kde.org/stable5/en/kde-workspace/plasma-desktop/index.html

and add Instructions to some of the sections below.

Printing

Printing must work in release-blocking desktops on at least one printer available to those performing validation testing.

Keyboard layout configuration

If a particular keyboard layout has been configured for the system, that keyboard layout must be used:

• When unlocking encrypted storage volumes during boot
• When logging in at a console
• When logging in via the default login manager for a release-blocking desktop
• After logging in to a release-blocking desktop, if the user account does not have its own keyboard layout configuration for that desktop (if there is such a user/desktop-specific configuration, it must be used when that user logs in to that desktop)

This one is kinda complex and probably does require an install to verify.
There are keymap tests https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/keymap.nix, but these aren't for how keymaps are handled in plasma5 or gnome. So you can cross off When logging in at console because that is validated automatically.

This is how you configure your keymap in nixos:

console.keyMap = "dvorkak";
services.xserver = {
  layout = "us";
  xkbVariant = "dvorak";
};

that's an example for us+dvorak.

In addition to this, a user can configure their layout without those options from systemsettings in plasma and gnome-control-center in gnome. The third point When logging in via the default login manager for a release-blocking desktop is actually very important and I've actually seen this regress in some places. It would look like "I configured dvorak", "but at display-manager it's colemak" or whatever layout. But in the actual desktop it's the configured layout.

I hope this is helpful and let me know if you have any other questions.

Desktop Keyring

Saving passwords to and retrieving passwords from the default keyring must work for all release-blocking desktops.

Instructions

In plasma they use kwallet:

• https://docs.kde.org/stable5/en/kdeutils/kwallet5/index.html

Using the kwallet application should suffice.

The way I usually test this is by using networkmanager and creating a new connection and using store only for this user, which should be default IIRC. I believe on an installed system the default wallet will be created from your login credentials, so you shouldn't have to create it. As for in the ISO, there is a default password but it is empty, so I'm wondering if this is going to work.

In gnome they use gnome-keyring:

• https://wiki.gnome.org/Projects/GnomeKeyring/

Opening seahorse and checking if the Login keyring is created
and unlocked. If it isn't unlocked this means the PAM module is broken somehow.

You can also test this with networkmanager like how I mentioned above.

User Switching

User switching must work using the mechanisms offered (if any) by all release-blocking desktops in their default configuration.

Instructions

This is best tested on an installed system with users.users creating a secondary user, and then using the switch user dialog. I know in gnome you can create a user imperatively (I know terrifying). I've never looked if you could do that in Plasma on NixOS, but it should be there.

[worldofpeace]

NixOS 20.09 GNOME Q.A

Shutdown, reboot, login, logout

Shutting down, rebooting, logging in and logging out must work using standard console commands and the mechanisms offered (if any) by all release-blocking desktops.

• [x] shutdown/poweroff
• [x] reboot
• [x] login
• [x] logout

I also tested suspend, but that can be fickle.

Required Applications

It must be possible to run a browser and a terminal applications from all release blocking desktops.

• [x] browser

Firefox started.

• [x] terminal

gnome-terminal is tested to start anyways.

Desktop Background

The desktop background should be different than the last release.

This was more of a todo, so skipping.

Artwork

Proposed artwork for the new NixOS release should be used as the background on release-blocking desktops and there should be a new logo.

This was also more of a todo, but we do use a NixOS wallpaper

• [x] uses nixos wallpaper

Release-blocking images must boot

It needs to be verified that all ISO's that are using a release blocking desktop boot cleanly in their supported configurations.

They must boot live when written to a USB stick with any of our supported methods documented in the NixOS manual.

USB Drive created with dd on linux (NixOS)

• [X] Boots on EFI
• [x] Boots on BIOS

Release-blocking images must not be excessively large

Firefox told me the download was 1.3 GiB, that's pretty good.

Working sound

Installed system should be able to play back sound

I tested this in various ways, since I think it should be able to play back sound in several ways

• [x] Played a video in firefox
• [x] system bell and volume sound
• [x] Plugged in a headphones and listened to same video
• [x] Downloaded video with youtube-dl and listening in gnome-music
• [x] The above with headphones

Non-crashing desktop

Not any part of the release blocking desktop should crash when interacted with, on startup, or be non-functional.

• [x] doesn't crash

Automatic mounting

Automatic mounting of removable media on insertion must work in release-blocking desktops.

• [x] Mounted a LUKS encrypted USB

Printing

Printing must work in release-blocking desktops on at least one printer available to those performing validation testing.

• [x] Printed a test page on installed system

nix-env

Software should be able to be installed, removed, and updated with nix-env.

• [x] Install
• [x] Remove
• [x] Update

Keyboard layout configuration

• [ ] unlocking encrypted storage volumes - didn't test :x:
• [x] console - this is tested automatically
• [x] When logging in via the default login manager for a release-blocking desktop
• [x] After logging in to a release-blocking desktop, if the user account does not have its own keyboard layout configuration for that desktop (if there is such a user/desktop-specific configuration, it must be used when that user logs in to that desktop)

In this case I made a new user, configured it for dvorak, and it was dvorak when I logged in.

Desktop Keyring

Saving passwords to and retrieving passwords from the default keyring must work for all release-blocking desktops.

• [ ] :red_circle: It asks me to create Default Keyring in live environment

I will mention that we actually fixed this a couple cycles ago, so I feel pretty confident in saying that on the installed system it will have a working desktop keyring.

User switching

User switching must work using the mechanisms offered (if any) by all release-blocking desktops in their default configuration. This requires creating another user.

• [x] User switching

GNOME doesn't seem to provide a "Switch User", so logging and and logging out seem to suffice. And I could create a new user for this.

GNOME specific etc.

GNOME initial setup shows up which is actually very very helpful in a LIVE environment.

• services.xserver.libinput.tapping which enables tap-to-click is not respected. I think we can use a dconf override in the gnome3.nix module to make this work for now.

org.gnome.desktop.peripherals.touchpad tap-to-click true

• We need to override favorite-apps to use firefox. This will make it a default item in the dock.

• ~Don't ship epiphany anymore~

This browser is not doing good in nixos atm https://github.com/NixOS/nixpkgs/issues/98819
Fixed in https://github.com/NixOS/nixpkgs/commit/13b192749c2312683ad5db285de66f0352b67f98

• firefox was not using Cantarell

It was using DejaVu instead. Why isn't firefox preferring the system font?

• subpixel rendering?

With the default font of Cantarell it is noticable that lcd subpixel rendering is better. With fonts like Inter I don't really see a different between grayscale or subpixel.

[birlorg]

Everything below is from the booted nixOS ISO image (not booted from the internal hard drive):

KWallet

Kwallet seems to work OK. I had to "force close". I've never used Kwallet before, but I could create a wallet, create a password int he wallet save it all, close Kwallet, open the wallet and view my password.

Keyboard Layout

Keyboard Layout Config: via plasma's system settings I changed to English UK, applied and the changes took effect in the desktop, but did not take effect in the console.

wallpaper

Wasn't on your list, but my Plasma ISO does not use the NixOS wallpaper or logo anywhere within Plasma. The default wallpaper (desktop background) image is labeled "Next"

Printers

I don't have a printer around, but launching the printer config dialog I get a box saying: "Print Service is Unavailable, Bad File Descriptor"

If I click the printer icon on the bottom right, I get a nix store error: "Error loading QML file: then a nix store path ending in "printmanager.qml" looks like it's trying to load from package "printmanager-20.08.1"

full error:

Error loading QML file: file:///nix/store/5byybbmrl6x408ykwg46xb49jxszb47b-print-manager-20.08.1/share/plasma/plasmoids/org.kde.plasma.printmanager/contents/ui/printmanager.qml:64:34: Type PopupDialog unavailable
file:///nix/store/5byybbmrl6x408ykwg46xb49jxszb47b-print-manager-20.08.1/share/plasma/plasmoids/org.kde.plasma.printmanager/contents/ui/PopupDialog.qml:81:23: Type PrinterItem unavailable
file:///nix/store/5byybbmrl6x408ykwg46xb49jxszb47b-print-manager-20.08.1/share/plasma/plasmoids/org.kde.plasma.printmanager/contents/ui/PrinterItem.qml:28:1: Type PlasmaExtras.ExpandableListItem unavailable
file:///nix/store/79fasdyzfj50ybrygl90qy0aj8i3j2wb-plasma-framework-5.73.0-bin/lib/qt-5.12.7/qml/org/kde/plasma/extras/ExpandableListItem.qml:20:1: module "QtQuick" version 2.14 is not installed

Users

There doesn't appear to be a way to add/switch users from within KDE, while booted from the ISO.

Conclusion

I've never really used KDE/Plasma before, so I have no idea what I'm doing. Overall it all seemed to work. I pushed a bunch of buttons and generally tried to mess things up.

Again, everything above was done from the live ISO, not booted from the HDD.

I won't test the gnome image, my inet isn't capable of downloading 2 ISO's in the same month.

[worldofpeace]

I've thought to seed the configuration.nix from nixos-generate-config with the config for the desktop environment https://github.com/NixOS/nixpkgs/pull/100199. Probably should add that to requirements that the generated hardware and configuration.nix are all build-able and match the ISO environment as close as possible so the user can reproduce it easily.

[davidak]

• [X] I also bootet the plasma iso on EFI successfully on my system
• [X] and GNOME iso on BIOS

with GNOME desktop, you don't have the manual and terminal on desktop, so you might get lost. an introductions opens, but that is just an intro. to GNOME. maybe we can inject a link to the manual there?

[worldofpeace]

with GNOME desktop, you don't have the manual and terminal on desktop, so you might get lost. an introductions opens, but that is just an intro. to GNOME. maybe we can inject a link to the manual there?

We could add it as a favorite https://github.com/NixOS/nixpkgs/blob/230728216ab79a7579b6549a91380aa9a1a3a0a0/nixos/modules/services/x11/desktop-managers/gnome3.nix#L46

[worldofpeace]

Tracking the issue with plasma5 printing here https://github.com/NixOS/nixpkgs/issues/98536. I'm going to move what's left for plasma5 QA etc. into the OP

[davidak]

GNOME Software is not usable in the live environment and i think also in GNOME unless you activate Flatpak, so maybe don't have it by dafault in GNOME? People will review NixOS and say it's broken...

Also, i noticed GNOME uses Wayland now. Does that generally work, for example with games using Steam?

[worldofpeace]

GNOME Software is not usable in the live environment and i think also in GNOME unless you activate Flatpak, so maybe don't have it by dafault in GNOME? People will review NixOS and say it's broken...

Also, i noticed GNOME uses Wayland now. Does that generally work, for example with games using Steam?

There's Xwayland :+1: We thought to default to the xorg session, but that's old news (it's been default for a bit) and the Wayland session should be fine for the live system. I've played stuff using Steam on the Wayland session just fine, I believe it will just use Xwayland when necessary.

[worldofpeace]

GNOME Software is not usable in the live environment and i think also in GNOME unless you activate Flatpak, so maybe don't have it by dafault in GNOME? People will review NixOS and say it's broken...

I think in the future we should document this. Though I can't help but feel there was some other reason we did this, like maybe it was for shell extensions and opening .flatpakref files cc @hedning @jtojnar

[hedning]

like maybe it was for shell extensions

Yeah, IIRC that was the reasoning, but Software no longer provides shell extensions, so it's fairly useless without flatpak at the moment.

Perhaps a services.flatpak.guiPackages option makes sense, that way the gnome desktop could add gnome-software and other desktops could add their preferred options without any clashes. And people would get a flatpak gui automatically if they enabled flatpak.

[worldofpeace]

like maybe it was for shell extensions

Yeah, IIRC that was the reasoning, but Software no longer provides shell extensions, so it's fairly useless without flatpak at the moment.

Perhaps a services.flatpak.guiPackages option makes sense, that way the gnome desktop could add gnome-software and other desktops could add their preferred options without any clashes. And people would get a flatpak gui automatically if they enabled flatpak.

Sounds slick, that would mean Plasma Discover users could benefit from, and pantheon with appcenter and... I think that makes the point pretty clear :grin:

[worldofpeace]

I'm considering this as close to finished as it will be since we only have a few days. Thanks everyone :+1: