Homebrew-core: Proposoal to resurrect p11-kit

[toonetown]

In January, p11-kit was sent to the boneyard, apparently because of upstream's lack of maintenance for OS X.

However, the two unit test failures that happen in that upstream bug appear to be due to the fact that we are running the build as non-root, or in some sort of sandboxed environment. The code for those unit tests p11-kit actually contains an option (set by environment variable) to skip those particular tests:

https://cgit.freedesktop.org/p11-glue/p11-kit/tree/common/test-compat.c#n138
https://cgit.freedesktop.org/p11-glue/p11-kit/tree/p11-kit/test-conf.c#n451

Information on why these tests fail in certain environments is pretty sparse - but from what I can figure (*see below), it appears to be something to do with setguid and sandboxing or something like that.

In the original (homebrew-legacy) issue from January where it was decided to boneyard, @apjanke noted that:

$ brew uses p11-kit
gnutls

And this is where _I_ ended up getting involved. I'm interested in adding PKCS#11 support in to openconnect - but according to @dwmw2 (from the openconnect project), it already exists there..._IF_ gnutls is installed using p11-kit. That got me thinking that perhaps it could be used by more applications if gnutls were compiled with it:

$ brew uses gnutls
anjuta                      libvirt                     qemu                      
bitlbee                     libxmlsec1                  task                      
freediameter                loudmouth                   taskd                     
git-annex                   mailutils                   ucommon                   
glib-networking             minbif                      upscaledb                 
gnu-smalltalk               ncdc                        vte3                      
gtk-vnc                     openconnect                 weechat                   
gwenhywfar                  pianobar                    wine                      
knot                        pianod                      wireshark                 
libinfinity                 pidgin                      homebrew/fuse/s3fs        
libsoup                     profanity                   homebrew/versions/gnupg21

One of the main benefits of including p11-kit (and support for it in gnutls) would be the ability for openconnect to use certificates from the OS X Keychain if an appropriate PKCS#11 driver is installed (likely KeychainToken - but perhaps others).

Prior to knowing that p11-kit was in the boneyard (I didn't check there first 😬), I had created a formula that I wanted to submit as a PR: https://github.com/homebrew/homebrew-core/compare/49bd559...toonetown:p11-kit

However, when running brew audit --new-formula p11-kit, I ran into:

$ brew audit --new-formula p11-kit
p11-kit:
  * p11-kit seems to be listed in tap_migrations.json!
    Please remove p11-kit from present tap & tap_migrations.json
    before submitting it to Homebrew/homebrew-core.

That (long explanation) brings me to the point of this issue. I would like to propose resurrecting p11-kit from the boneyard and adding support for it back in to gnutls (ideally as an :recommended dependency). I just do not know if there is some kind of official "proposal" mechanism, or if I just remove it from homebrew-boneyard (and create a PR there) and then remove it from tap_migrations.json and include that in my PR to homebrew-core.

The next step that I would perform (and as a separate PR with a separate discussion) would be re-enabling p11-kit functionality in gnutls.

Specifically, I'd be interested in hearing from those who worked on the issue around boneyarding p11-kit (@apjanke, @dunn, @rakkesh, @DomT4, and @MikeMcQuaid) to see if perhaps I'm missing something here. I am _very_ new to this side of things (PKCS#11/gnutls/etc) and am the first one to say I am _not_ a security expert - so perhaps there are other considerations than what I have spelled out here. Plus, I'm not opposed to just running my own formulas on my own machine for my own purposes if they aren't helpful to a broader audience.

---

*Additional information I found about the tests that were failing in p11-kit:

• Patch which introduced the tests
• Very brief mailing list post which indicates that it has something to do with setguid
• Ubuntu changelog which lists the change as being a part of 0.18.5-1 (Which jives with the comment that nothing newer than 0.18.4 works on OS X)

[dwmw2]

I can't comment on https://github.com/Homebrew/legacy-homebrew/issues/47124 but in there, @rakkesh says:

PKCS#11 in general is not targeted for desktop apps

For Linux desktops at least, that isn't true. We do expect that you can plug in a token like a Yubikey, and it should transparently be available to _all_ applications, desktop and otherwise. Fedora policy is that anything which would allow you to use a certificate from a file, should also transparently accept a PKCS#11 URI in place of the filename.

And PKCS#11 isn't limited to hardware support; KeychainToken can also expose the certificates/keys in the OSX Keychain(s) via the PKCS#11 API. So if we get the pieces in place, it means that any "well-behaved" application which supports PKCS#11 out of the box, will also support using keys from the Keychain. That seems like a desirable goal, right?

I had an OSX user come to me last week and ask about using Keychain certs in OpenConnect; I didn't have a good answer. It would be great if it Just Worked, and it looks like it can.

[DomT4]

https://github.com/Homebrew/homebrew-core/pull/5351 is your friend. You were beaten to the punch. Will close here as that PR solves your question, but feel free to poke additional discussion there.

I would like to propose resurrecting p11-kit from the boneyard and adding support for it back in to gnutls (ideally as an :recommended dependency).

IRT this I'm probably a 👎 on :recommended, at least until p11-kit is proven to be used enough.

[toonetown]

I would like to propose resurrecting p11-kit from the boneyard and adding support for it back in to gnutls (ideally as an :recommended dependency).

IRT this I'm probably a 👎 on :recommended, at least until p11-kit is proven to be used enough.

Would you be OK with :optional? If so, I'll work up a PR for that.

[DomT4]

Yup! No problem with :optional at all once https://github.com/Homebrew/homebrew-core/pull/5351 is merged.

[toonetown]

Yup! No problem with :optional at all once #5351 is merged.

OK - I'll work up a new PR for that once #5351 is merged. Where would be the appropriate place to discuss :optional vs :recommended on that? I'm guessing that here (on a closed, unrelated bug) is not the place - but I have a couple of arguments that I'd like to put forth toward using :recommended. Is there a mailing list that would be more appropriate to carry on that discussion? Or should I open a new issue and carry on the discussion there? Or do we have the discussion on the PR that I will submit? Or should I just hop on the IRC channel and talk about it there?

(I'm just trying to know where the best place to carry on such discussions may be)

[DomT4]

? Or do we have the discussion on the PR that I will submit?

This one.

[dwmw2]

From my experience with trying to make PKCS#11 _Just Work™_ for Linux users — which in your case translates to making keys in the Keychain _Just Work™_, I'd suggest that you really do want to recommend p11-kit and that applications build with PKCS#11 support.

For keys in hardware tokens, we end up with a string which they can use which identifies the key therein — for example pkcs11:manufacturer=piv_II;id=%01 is a typical example a Yubikey. And they can consistently use that same string with _all_ well-behaved applications. But that only works if you recommend that this support gets included.

For OSX with KeychainToken, you end up using this not just for hardware support, but also for referencing keys in the Keychain, which is really kind of important for system integration. One might argue that _all_ well-behaved applications should be capable of using keys from the Keychain, if they are capable of using keys from files.

[MikeMcQuaid]

From my experience with trying to make PKCS#11 Just Work™ for Linux users — which in your case translates to making keys in the Keychain Just Work™, I'd suggest that you really do want to recommend p11-kit and that applications build with PKCS#11 support.

Considering it was just put in the boneyard: there's no way we're going to make it :recommended before it's been :optional for a while first.

[dwmw2]

Considering it was just put in the boneyard: there's no way we're going to make it :recommended before it's been :optional for a while first.

Absolutely; that makes complete sense! Let's actually get it _working_ first, too :)

So the next thing is to add KeychainToken. @toonetown do you have that one lined up in a PR somewhere? Make sure you include a p11-kit .module file, installed into the directory indicated by running:

pkg-config --variable=p11_module_configs p11-kit-1

Note: _Every_ PKCS#11 provider you package should do this. So if you have OpenSC, Coolkey, SoftHSM, and anything else that provides a PKCS#11 module, they should do it too.

Then, basically any application that is built against GnuTLS will Just Work™ when you give it a PKCS#11 URI instead of a filename for a certificate. Anything using OpenSSL needs to be using the PKCS#11 engine, but that's not particularly complex. Things like Curl, wpa_supplicant, etc. already support it.

With very little actual work _(assuming KeychainToken is working)_ you should quickly get to the point where all applications you ship can use keys from the Keychain...

[toonetown]

@dwmw2
So the next thing is to add KeychainToken. @toonetown do you have that one lined up in a PR somewhere?

Yes - that is what I'm going to be working on next. (Well, getting gnutls to have :optional dependency on p11-kit).

However, as I'm reading through the GnuTLS README for instructions on how to build, it actually does list p11-kit as a required dependency (they don't even list it as optional). The only places where I can see them using it as --without-p11-kit is when compiling for Windows.

Given that, perhaps it might be better to treat it as :required even...

@MikeMcQuaid
Considering it was just put in the boneyard: there's no way we're going to make it :recommended before it's been :optional for a while first.

That is an argument I'd actually make in _favor_ of making it a requirement for gnutls...it _used_ to exist in homebrew-core (up until recently), and had been used by gnutls, but was retired because of the perception it wasn't being maintained (when, in reality, the main problem was that we just needed to run the correct unit tests in our build).

There is one question I do have, however. Is it possible to create a formula that has a dependency upon another formula with a specific option enabled? For example, if p11-kit is an :optional to gnutls, is there a way to specify in the openconnect formula that it depends_on gnutls[with-p11-kit] or something? (I have no idea what the syntax of such a specification would be.

If that is _not_ the case, then I would say that's even more of an argument for having p11-kit be a :required dependency...because there already is a (fairly common, IMO) use case requesting it.

I am also currently grokking the gnutls code to see what sort of additional stuff is done when p11-kit is included. I know that it will build the p11tool binary...I don't know if it materially changes much else in the final output - but I'll report back with what I find.

[dwmw2]

Note that you _can_ build OpenConnect without PKCS#11 support. I just don't recommend it because this stuff SHOULD JUST WORK, DAMMIT! :)

But if the configure script finds that the gnutls_pkcs11_add_provider() function isn't present in GnuTLS, it'll infer that GnuTLS was built without PKCS#11 support and it'll build OpenConnect without PKCS#11 support accordingly.

I'm not entirely familiar with your policies, but I suspect you probably don't want an option for the OpenConnect formula which makes it support PKCS#11 or not; it will just automatically use what GnuTLS offers.

[toonetown]

So - looking through the gnutls source code, it _does_ appear that the only things that change are:

1. A new binary p11tool is created in addition to the library
2. Additional pkcs11 functions are available and exposed in the library

I think it would be fairly safe to list it as a :required dependency...at the very least a :recommended dependency. But I'm working up a PR right now with it as an :optional dependency so that we can continue moving forward with our testing.

[DomT4]

You might have been beaten to it again in https://github.com/Homebrew/homebrew-core/pull/5416 🙈. Not having a very lucky week with Homebrew contributions.

[nmav]

As the maintainer of gnutls, I'd strongly suggest making the p11-kit variant the default. The PKCS#11 functionality is essential in modern gnutls versions, especially for primary platforms like OSX which you would like your users to use smart cards, HSMs and the keychaintoken.

[toonetown]

@DomT4
You might have been beaten to it again in #5416 🙈. Not having a very lucky week with Homebrew contributions.

Maybe I'll create my PR right now - it's just one file change (just gnutls).

[dwmw2]

Once we have KeychainToken working, doesn't that expose the system's default trust database too, and you'll want to configure GnuTLS to use it for default trust? What are you lot doing for default trust right now anyway? Something non-integrated with OSX, I'd guess?

[dunn]

@toonetown

There is one question I do have, however. Is it possible to create a formula that has a dependency upon another formula with a specific option enabled? For example, if p11-kit is an :optional to gnutls, is there a way to specify in the openconnect formula that it depends_on gnutls[with-p11-kit] or something? (I have no idea what the syntax of such a specification would be.

Yes, you can do depends_on "gnutls" => "with-p11-kit"

[nmav]

Once we have KeychainToken working, doesn't that expose the system's default trust database too, and you'll want to configure GnuTLS to use it for default trust? What are you lot doing for default trust right now anyway? Something non-integrated with OSX, I'd guess?

It could use it if the CA certificates are available in a special token. You'd then have to specify --with-default-trust-store-pkcs11=URI to configure (and this token must be marked as trusted in p11-kit config)