Nixpkgs: Thunderbird U2F support doesn't work

[IreneKnapp]

Describe the bug
When attempting to log onto a Google email account within Thunderbird, Google's systems direct Thunderbird to use OAuth2. For users who have configured their Google accounts to require a U2F credential (such as a Yubikey), the U2F prompt occurs during this flow. The flow works normally up to the U2F step, which I am not able to complete.

To Reproduce
Steps to reproduce the behavior:

1. Prepare a Google account configured with at least one U2F hardware token as a second factor for logging in, and ensure that gmail is an active service in the account.
2. Launch Thunderbird. Add a new email account. Enter any human name, and the email address of the prepared Google account, leaving the password black. Click "done" a few times while Thunderbird discovers the correct way to log in.
3. A window will pop up showing Google's login page. Enter the username and password at the prompts, until you reach the prompt to use a security key.
4. Observe that the security key does not flash its lights (as it normally would to indicate that a touch is requested). Also observe that, though the on-screen prompt to touch the security key appears normally, after about a second it is replaced by the words "Something went wrong. Remove your Security Key and try again."

As this message is produced by Google's code, not by Thunderbird's, it is likely that the exact text will change over time.

Expected behavior
At step 4, the security key plugged into my USB port would flash its lights, I would touch it to authorize the login, and the login would complete successfully, allowing me to check my email in Thunderbird.

Screenshots

Additional context
Most users encountering this problem will be able to work around it by using Google's app-specific passwords feature; if you're having this problem, consult the Google help center for advice. This workaround is not available to users using the highest level of security setting, Advanced Protection, since that setting intentionally disables the app-specific passwords feature.

U2F during the OAuth2 flow is ostensibly supported in Thunderbird since 60.0, and my tests are with Thunderbird 68.7.0, the latest packaged in NixOS 20.03.

This same login flow also appears when trying to log into the same Google account as a normal user in Firefox. In that context, it works as expected for me. It is likely that Google is doing some user-agent detection as well as some API-version detection, but I played around with both of those a bit using Thunderbird's developer tools and was unable to find a variation that produced different behavior.

I suspect that this may be an issue related to a similar bug in Firefox on NixOS, #34747, that was fixed two years ago. In that bug, the issue was the absence of libudev. I read the Thunderbird code and found that, indeed, it has a Rust component which supports U2F and relies on being able to dlopen("libudev.so.1") at runtime (see thunderbird-68.7.0/dom/webauthn/libudev-sys/src/lib.rs). I attempted to resolve the issue by adding libudev to the Thunderbird buildInputs, but this didn't appear to change anything.

I am happy to assist in gathering more information. After discussion in the Thunderbird Matrix chatroom, it was suggested to me that this is more likely to be a NixOS issue than a Thunderbird issue, so I'm following up here. My main goal is to make sure this is being tracked, and to help however I can; I don't have a time-critical need to make it work.

Notify maintainers
@edolstra @lovesegfault @nbp

Metadata

• system: "x86_64-linux"
• host os: Linux 5.4.35, NixOS, 20.03.1619.ab3adfe1c76 (Markhor)
• multi-user?: yes
• sandbox: yes
• version: nix-env (Nix) 2.3.4
• channels(root): "nixos-20.03.1619.ab3adfe1c76, nixos-unstable-20.09pre223023.fce7562cf46"
• nixpkgs: /nix/var/nix/profiles/per-user/root/channels/nixos

Maintainer information:

# a list of nixpkgs attributes affected by the problem
attribute: thunderbird
# a list of nixos modules affected by the problem
module:

[arianvp]

Thanks for the clear report. Just for sanity's sake; did you have

hardware.u2f.enable = true;

in your NixOS config? This is needed to load required udev and hwdb rules

[06kellyjac]

As a note I'm having the same issue. Yubikey works in Firefox but not in Thunderbird

nixos-option hardware.u2f.enable
Value:
false

Default:
false
//...

Looks like I don't have that enabled but that doesn't explain why it works for Firefox but not for Thunderbird :thinking:

I'll enable it and try some time soon.

---

nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 5.4.35, NixOS, 20.09pre223023.fce7562cf46 (Nightingale)`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.3.4`
 - channels(root): `"home-manager, nixos-20.09pre223023.fce7562cf46, nixos-hardware"`
 - channels(username): `""`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

Firefox and thunderbird are just as is from pkgs

nixos-option environment.systemPackages | grep -i thunderbird
  text = "[ pkgs.firefox pkgs.thunderbird ]";

[emilazy]

Maybe we should default hardware.u2f.enable to on, or just remove the toggle entirely. Does it increase closure size significantly?

[emilazy]

That said, I have it on and can reproduce this problem on my system, so unfortunately it's not a fix for this issue.

[lovesegfault]

I've had this same issue from day one of being on NixOS and I fiddled around a bit without success trying to fix the issue.

[prusnak]

Maybe we should default hardware.u2f.enable to on, or just remove the toggle entirely. Does it increase closure size significantly?

It's literally just adding a single libu2f-host package.

[IreneKnapp]

@arianvp I had actually been using the following code manually, since my use of U2F goes back to before the hardware.u2f.enable option existed.

  services.udev.packages = with pkgs; [
    libu2f-host
  ];

  environment.systemPackages = with pkgs; [
    libu2f-host
  ];

Looking at the implementation of hardware.u2f.enable, this looks more than sufficient (I probably didn't need to add it to systemPackages).

I have now re-tested with hardware.u2f.enable = true. I get the same behavior as in my original report.

[IreneKnapp]

I am logging my progress on this to motivate myself, and in case anybody else chooses to work on it. I don't know when or whether I'll have time to finish.

I wanted to verify my hypothesis that libudev isn't being found at runtime. I made the following change to the Thunderbird source:

diff --git a/src/dom/webauthn/libudev-sys/src/lib.rs b/src/dom/webauthn/libudev-sys/src/lib.rs
index c422399d85..3999a95505 100644
--- a/src/dom/webauthn/libudev-sys/src/lib.rs
+++ b/src/dom/webauthn/libudev-sys/src/lib.rs
@@ -56,9 +56,17 @@ impl Library {
     let name = CString::new(name).unwrap();
     let name = name.as_ptr();

-    Library(unsafe {
+    let result = Library(unsafe {
       ifnull!(dlopen(name, flags_noload), dlopen(name, flags))
-    })
+    });
+
+    match result {
+        Library(pointer) => {
+            println!("IAK libudev.so.1 {:?}", pointer)
+        }
+    };
+
+    result
   }

   fn get(&self, name: &'static str) -> *mut c_void {

When I ran it, this printed out the following output:

IAK libudev.so.1 0x0

That's a void pointer. So I'm now reasonably confident that this dlopen() failure is the problem. I think simply putting libudev in the buildInputs isn't sufficient for the Thunderbird build system to find it. I'll try my best to understand the build system and figure out what else is needed.

[06kellyjac]

Quite a few packages that mention libudev symlink it in the installPhase or they have a patch that adds a flag that lets you point at it

ln -s ${systemd.lib}/lib/libudev.so.1 $out/lib/libudev.so.0
''-DNIXPKGS_LIBUDEV="${getLib systemd}/lib/libudev"'';

[IreneKnapp]

I have a PR that fixes this coming soon. I'm not able to make an automated test for the behavior, so I think it's likely that this issue thread will be important to have around should there ever be a regression. Therefore, I'm documenting an observation about development workflow, hoping that nobody will ever need it, but just in case.

The observation is this: If you have successfully logged in once, the credential will be cached. If you need to test again after doing that, the steps to reproduce above won't work because you won't be able to force a login. To work around that, the file to get rid of is ~/.thunderbird/<profile>/logins.json.