Void-packages: [RFC] Switching back to OpenSSL

[Johnnynator]

OpenSSL nowadays doesn't have the issues anymore it had back in 2015 and significantly more Contributors watching and improving it. LibreSSL usually lacks behind in terms of supported algorithms and doesn't support the same API. Libs like Qt dropping support for OpenSSL 1.0 makes it significantly harder to maintain a (correct) patchset for LibreSSL support.

OpenSSL Pros

• Not that many (potentially wrong) patches needed, proper upstream support for nearly ever lib/program
• Potentially faster on non x86_64 platforms
• Access to newer Algorithms earlier
• No ABI breakage every 6 month

LibreSSL Pros

• Potentially safer by default (?)

[travankor]

Another advantage: OpenSSL is switching to a license OpenBSD considers non-free (Apache-2.0, which Void considers free). This means the codebase between openssl and libressl is more likely to diverge.

I think having better software (ie: haskell ssl library keeps breaking with libressl) and hardware support (ie: aarch64 crypto acceleration) is more useful for Void than security (not that openssl is super insecure these days).

[q66]

I'd argue that OpenSSL is safer, since it just gets a lot more attention and audit nowadays. Since heartbleed a lot of attention has gone to OpenSSL, it's probably one of the better-audited projects nowadays.

The performance increase on non-x86_64 platforms is not "potential", it's there; OpenSSL has optimized assembly code for most architectures, in addition to plain C fallbacks - LibreSSL does not have them, they all got dropped with the exception of the x86_64 ones.

So, +1 from me.

Perl being required for build is a non-problem, it's already required for build in several other bootstrap packages, including coreutils, gcc and glibc.

[pullmoll]

I had myself several times where it was difficult to see in which way patching a source for libressl would be correct. This is because I do not know every detail of the differences between the openssl versions 1.0.x and 1.1.x, and the libressl API lies somewhere in between the two.

So from my point of view using openssl could save us lots of work, and if a majority thinks that openssl is audited well enough nowadays, I'm pro switching.

[Hoshpak]

Are you talking about xbps as a project or the Void Linux xbps package? Switching all packages to openssl and still forcing every Void system to still install libressl in parallel through xbps would make it kind of pointless to switch in the first place.

I generally agree that we should switch to openssl. Libressl not supporting the openssl 1.1 API is increasingly holding packages back (I think I had issues when trying to update postfix in the past) and cannot be trivially patched. The slow movement of libressl development also bothers me and led me to not use it on my server. I am now able to connect to this server via TLS 1.3, just not from any of my Void machines.

[Johnnynator]

Simply don't switch xbps.

This would also imply to build libarchive against LibreSSL, but nevertheless I dislike having both LibreSSL and OpenSSL at the same time in the base system.

[Johnnynator]

We do have already multiple implementations at the same time, see mbedtls, gnutls, libressl, etc.

But not in the base system, there we only have LibreSSL as of now.

[Johnnynator]

E.g. I need to decide if ca-certificates depends on LibreSSL or OpenSSL (in theory I might be able to patch update-ca-certificates to work with both)

[travankor]

Anyway I've been thinking about it and maybe I'll switch xbps to use mbedtls. Not sure yet. So go ahead!

@xtraeme What about bearssl? In the link you provided above, it resists the attacks.

[travankor]

no, i meant for xbps, as an alternative backend

[Johnnynator]

@Johnnynator hmm I would not do this way. Each ssl implementation must depend on ca-certificates.

Yes, all ssl implementation depend on ca-certs but ca-certs depends on only one SSL implementation.
But the update-ca-certificates script right now ONLY works with libressl. And the openssl command does not have a proper way of querying whether it is OpenSSL or LibreSSL. (It always exits with 0, even when the command was not found..., so I need to judge it by what is print to stdout, argh...).

Edit: correction, OpenSSL exits with 1 on invalid commands, LibreSSL is the one that always exits with 0.

[Johnnynator]

in fact I haven't tried with openssl >= 1.1, but I think it would need minimal changes...

XBPS did compile and run fine for me locally.

[Johnnynator]

@Johnnynator we could use alternatives for the openssl command, and then use the specific impl cmd, i.e for openssl "openssl", for libressl "openssl-libressl" or whatever.

Probably the most sane way, I will prepare it like that.

@Johnnynator cool! I'll update the README then.

So I'm not against it, but what bothers me about openssl is the perl build dependency... it DOES matter while bootstrapping. I would take the alpine patch to get rid of it.

Alpine also needs perl for bootstraping, and the perl c_rehash runtime script is not needed in our case, since our ca-certifcates package is not using it, so we can simply ignore it. Also as q66 pointed out, we already have a few packages that need perl for bootstrapping (e.g. glibc, gcc), so I don't see an issue with OpenSSL needing it.

[Johnnynator]

Anyways, stick with libressl 3.1 for now, it's about to get released and adds more compatibility for openssl 1.1.

There are still significant gaps in the API.

[travankor]

stuff that uses libtls will need libressl, too

[q66]

Perl does not matter while bootstrapping, stop insisting that it does. I still don't see any reason to package both of them either, as @Johnnynator said it would require libarchive to be built against it and complicate everything. The "just in case" argument doesn't make any sense, either of them could mess up something and you have no way to know which.

[q66]

Does anything actually use libtls? Since it's a libressl specific api and most distros don't package it at all, I don't think we need to worry

[q66]

It is, since coreutils needs it, as well as GCC.

[q66]

Libcrypto contains the majority of the asm acceleration code, including for sha*. Wrt libtls: so... other openbsd projects (duh) - I doubt it'd required, as e.g. Debian packages openntpd without libressl, and s6-networking, which can also use bearssl, which is a better choice either way.

[q66]

Anyway, openssl needs Perl for good reasons, it uses it to deal with processing the assembly code for different targets. Libressl was only able to rip it out because they ripped out the asm

[q66]

C implementation of what?

[Johnnynator]

They still have c implementation of c_rehash but as I said, it is not really needed and we can ignore it, since we use the debian ca_certificates update script.

[q66]

There are much worse bootstrap dependencies we could have than Perl anyway, as far as I know Perl has never been problematic on anything, has been around for decades and is completely portable. And pretty much every single distro out there ships it.

[q66]

1) that doesn't even matter for this topic, since Perl is a host dependency. 2) the cross-compiling logic at least for current Perl isn't even complicated. Past doesn't matter, at all. 3) I don't care how many commits you have

[Ypnose]

I'm no longer a package maintainer, but from an user perspective libressl is sometimes painful when specific openssl options are needed and not included. There is an example here : https://github.com/libressl-portable/portable/issues/544
If it can save maintainers time, go for it.

[mobinmob]

[juan@leysa ~]$ xbps-query -Rs libtls.so -p shlib-requires
acme-client-0.1.16_4: libtls.so.19 (/mnt/storage/hostdir/binpkgs)
libressl-3.0.2_2: libtls.so.19 (/mnt/storage/hostdir/binpkgs)
libressl-netcat-3.0.2_2: libtls.so.19 (/mnt/storage/hostdir/binpkgs)
openntpd-6.2p3_5: libtls.so.19 (/mnt/storage/hostdir/binpkgs)
s6-networking-2.3.1.2_1: libtls.so.19 (/mnt/storage/hostdir/binpkgs)
[juan@leysa ~]$

s6-networking works with bearssl. Upstream marks bearssl support as beta but both Alpine and Adelie use it.

[howtologinquickwiththirtyninecharacters]

@Johnnynator you may want to update your package to 1.1.1g, versions d, e and f are affected by this vulnerability. (Is this the right place to comment on this or should I have commented on the New package request? I'm still new to this).

[Hoshpak]

The version remark would have been better in the PR however the vulnerability itself is highly relevant to this discussion since the number of vulnerabilities in each library is an important decision criterion for a TLS library.

[q66]

All software has vulnerabilities. I seriously doubt libressl has fewer of them.

[the-maldridge]

My 2 cents. I am opposed to software monocultures, they stifle attempts to produce new and better implementations and tend to breed discontent among developers that wish to do something different.

If we were to accept OpenSSL I would recommend doing so in the same way we have gcompat. It can be used in places where there is need for its specific interface, but otherwise not. My preferred SSL implementation is BoringSSL, though it is unsuitable for use in a distribution.

[eli-schwartz]

Alpine also needs perl for bootstraping, and the perl c_rehash runtime script is not needed in our case, since our ca-certifcates package is not using it, so we can simply ignore it.

They still have c implementation of c_rehash but as I said, it is not really needed and we can ignore it, since we use the debian ca_certificates update script.

Note that there's probably never a good excuse to use c_rehash at all, whether you use the debian ca_certificates script or not... because https://www.openssl.org/docs/man1.1.1/man1/openssl-rehash.html

tl;dr /usr/bin/openssl rehash and /usr/bin/c_rehash do the same thing, one in C and one in perl. It's unclear when you'd ever want to use the latter, and I think you might be hard-pressed to find software which invokes it. Someone tried to rewrite it in bash and PR it to openssl, but the PR was closed as "perl is easier to build on OpenVMS, that being said we might be able to just drop it entirely since you should just use the openssl app's rehash command".

[fosslinux]

I agree with @the-maldridge, after some hard thinking and a discussion on IRC.

I'm not really concerned about OpenSSL in the repositories.

However, LibreSSL should be of first-class support, and OpenSSL should only be used where necessary for maintainability (eg, qt5). IMO, OpenSSL shouldn't be in the base system - xbps should stay with LibreSSL; no real reason to move it off it.

Saying all this, it is essential that the maintainers come to a decision how OpenSSL should be used before it is merged, and what will happen to LibreSSL (once again, I will strongly advocate for LibreSSL not being removed - rather still being first-class).

I see a number of options, ranked from most LibreSSL to most OpenSSL.

_No OpenSSL_

1. Do not merge OpenSSL.
   _User choice, first-class support for LibreSSL; OpenSSL not well supported_
2. Merge OpenSSL, but do not have any packages depend upon it. Have it as a choice. Maintain full compatibility with LibreSSL, but don't require current packages to support OpenSSL. Do not include OpenSSL in the base system (default LibreSSL).
   _User choice, first-class support for both_
3. Merge OpenSSL, but do not have any packages depend upon it. Have it as a choice. Maintain full compatibility with LibreSSL; quickly ensure all current packages to support OpenSSL. Do not include OpenSSL in the base system (default LibreSSL).
   _Maintainer choice, but LibreSSL for base system_
4. Merge OpenSSL. Allow packages to depend upon it, and optionally drop LibreSSL specific patches. Packages will pull in either of OpenSSL or LibreSSL as required. Both could be installed on the same system. However, base packages should only include LibreSSL. Do not include OpenSSL in the base system.
   _Maintainer choice, including base system - both in base system_
5. Merge OpenSSL. Allow packages to depend upon it, and optionally drop LibreSSL specific patches. Packages, including base packages, are allowed to pull in either of OpenSSL or LibreSSL as required. Both could be installed on the same system - and both will be installed as part of the base system.
   _Maintainer choice, but OpenSSL for base system_
6. Merge OpenSSL. Convert all base system packages to use OpenSSL only (including xbps). Allow packages to depend upon it, and optionally drop LibreSSL specific patches. Base system should only use OpenSSL. Both could be installed on the same system, but only OpenSSL will be in the base system. Maintainers can still choose to use LibreSSL, and most software can continue to do so (ex. base system).
   _User choice, first-class support for OpenSSL; LibreSSL not well supported_
7. Merge OpenSSL. Convert all base system packages to use OpenSSL only (including xbps). All packages must work with OpenSSL - make this a priority - but not all have to work with LibreSSL. Include OpenSSL in the base system, and make it the default. Maintainers must use OpenSSL.
   _OpenSSL only; no LibreSSL_
8. Merge OpenSSL. Convert all packages to use OpenSSL only. All packages must work with OpenSSL. Roadmap for LibreSSL to be removed from the repositories.

6 is likely to end up at 7 eventually.

I, personally, would hate 7 or 8. My opinion is 4. 3 and 5 would create too much maintainer work, 6 would lead to an extreme drop of support of LibreSSL in general, and would eventually lead to 7. 1, 2 and 3 I would also be happy with (but 3 would create poor maintaership).

I would strongly recommend against 2 and 7 because all it's going to add is complex, dodgy code, broken software, and worse packaging.

[Duncaen]

Running both is not an option all reverse dependencies need to use the same one otherwise we get runtime errors.

Edit: Excluding the few limited cases that require libtls. A per package decision on using libressl or openssl is a logistical nightmare.

[q66]

Mixing libressl and openssl in one system is a recipe for disaster as they share symbols.

Also, sticking primarily with libressl does not solve the problem of the experience being poor outside of x86_64.

[Ypnose]

Please, can you elaborate your comment ?

[the-maldridge]

I believe q66 is referring to openssl containing large amounts of hand optimized assembly for both modern and long dead platforms which accelerates certain arithmetic functions. LibreSSL works primarily on x86_64.

Perhaps a better question to ask about this is why Void is seeing poor performance on non-x86 platforms. OpenBSD builds on a number of different targets, and there aren't reports of poor performance that I'm aware of from a very cursory search.

[q66]

@the-maldridge It's not just about hand optimized, it's also about access to hardware crypto, which libressl outside of x86_64 does not have, which results in significantly poorer throughput

1) openbsd builds a lot fewer targets than Linux, e.g. they don't have 64-bit ppc of any kind
2) people using openbsd don't care about performance a whole lot, e.g. there is still no reasonable SMP in openbsd
3) openbsd still uses the perl infra from openssl to generate asm for targets they build, while libressl-portable just has the x86_64 ones generated ahead of time

[q66]

I ran some tests on ppc64le for comparison: https://gist.githubusercontent.com/q66/4f4dc63565cdfafb10c6dde1d3067648/raw/8d2243c22324212af35d3133455c0c7067ab088f/bench.txt

[Vaelatern]

Should note that dfly, an OS that cares a lot about SMP, does use libressl, but they are only x86_64.

It may be that openssl is no longer the tire fire it was when Void Linux switched. More importantly, it may be that adoption of OpenSSL is more in line with Void's philosophy than staying on LibereSSL.

But there is about to be a new LibreSSL release. I'd propose that we wait for that release and the rebuild following before we make a decision, to see if things are better or diverging openssl and libressl APIs are making things so much worse.

[CameronNemo]

Regarding libtls, we may have an option in this library (note: I have not vetted this):

https://sr.ht/~mcf/libtls-bearssl/

[marmeladema]

By the way, new release of LibreSSL is out since early April:

• https://ftp.openbsd.org/pub/OpenBSD/LibreSSL/libressl-3.1.0-relnotes.txt
• https://ftp.openbsd.org/pub/OpenBSD/LibreSSL/libressl-3.1.0.tar.gz
• https://ftp.openbsd.org/pub/OpenBSD/LibreSSL/libressl-3.1.0.tar.gz.asc

Well ... apparently its not a stable version. Sorry for the confusion.

[concatime]

(https://www.nccgroup.trust/us/about-us/newsroom-and-events/blog/2019/february/downgrade-attack-on-tls-1.3-and-vulnerabilities-in-major-tls-libraries/)
@travankor, to be fair, BearSSL does NOT implement TLS 1.3.

It would have been cool if they also tested MatrixSSL.

[travankor]

@concatime I only suggested to xtraeme that he considers a bearssl backend for xbps. Since he's gone, it's up to the community to decide.

I doubt anyone would port xbps to MatrixSSL, given that it changes the effective license of the derived work.

[imrn]

21994: Is it related with libressl? Any comments?

[Vaelatern]

@imrn probably not related.

[marmeladema]

Has any consensus been reached?
On a personal note, I am starting to struggle using Void Linux on a daily basis because more and more things rely on recent protocols/algorithms not provided by libressl. For example, i have to either build openssl/cURL myself or rely on a docker version of cURL.

[fosslinux]

What is the issue with cURL?

[CameronNemo]

@marmeladema Since this issue was created:

• A vulnerability in OpenSSL's TLS 1.3 implementation was found
• A new release of LibreSSL with TLS 1.3 was released
• q66 enabled hardware accelerated cryptographic primitives for PowerPC and ARM systems

Those are the major changes to the situation from my perspective. I would be curious to know which algorithms and protocols are missing from LibreSSL. Many of the algorithms removed from LibreSSL were removed deliberately from what I have read.

[q66]

umm... my halfassed assembly stuff is hardly a solution for anything, it was never meant to be anything more than a stopgap solution until this is solved

openssl still has much better performance in various things (e.g. chacha20 which is default for openssh), libressl is still missing things (e.g. python hashlib doesn't have bcrypt and other things when built with libressl), libressl-portable is still a second tier codebase (it's awfully clear that libressl is written by openbsd for openbsd, and the "portable" distribution has its own specific bugs) and the divergence is only going to get larger, since openbsd doesn't take apache 2.0 code, and all fixes, enhancements (including new assembly code etc) and so on going into openssl are apache 2.0

openssl also still gets significantly more auditing and has a larger testsuite, at least in the portable libressl distribution the testsuite is rather superficial and that's a red flag for security too

also, the tls 1.3 support in libressl is client only as far as i can tell

nothing has changed on my stance since starting this issue, i've only become more convinced

[fosslinux]

The Apache 2.0 licensing issue is going to come with an entire host of problems. LibreSSL will surely diverge further from OpenSSL and this will create more burden for package maintainers.

If there is one reason apart from the portable libressl distribution's issues, including the lack of a test suite, strange feature detection, often poor coding style, and poor performance, I think that the burden on maintainers is the primary problem with libressl going forward.

I didn't think we should switch back to OpenSSL at the beginning of the issue but I am quite convinced now.

[q66]

i wouldn't call libressl poor, it's a good project and i appreciate the effort, but for openbsd - portable seems to be getting attention/maintenance mostly from... 3 people overall as far as i can see, and I'm not super comfortable sticking with it in long term - especially if we want to take portability seriously - and "but heartbleed" doesn't sound like very convincing argument to me anymore, considering how much attention openssl has been getting the last half a decade...

[bugcrazy]

OpenSSL has a lot of legacy OS discontinued, quantity of developers in OpenSSL does not mean quality in the code, OpenSSL has a design problem, it is susceptible to serious flaws, not that LibreSSL has no vulnerabilities, but it has cleaner code and with a focus on safety. This link shows that LibreSSL has more development than https://cpp.libhunt.com/compare-openssl-vs-libressl

Here on this Gentoo link that has a LibreSSL port project, which has patches and fixes to ensure software compatibility with LibreSSL. https://gitweb.gentoo.org/repo/proj/libressl.git

In this old link, it compares OpenSSL vs LibreSSL, showing how security is applied in LibreSSL code.
https://resources.infosecinstitute.com/libressl-the-secure-openssl-alternative/

[Duncaen]

This link shows that LibreSSL has more development than https://cpp.libhunt.com/compare-openssl-vs-libressl

https://github.com/openssl/openssl/graphs/commit-activity
https://github.com/libressl-portable/portable/graphs/commit-activity

Here on this Gentoo link that has a LibreSSL port project, which has patches and fixes to ensure software compatibility with LibreSSL. https://gitweb.gentoo.org/repo/proj/libressl.git

Those are 3 people with like 30 commits this year.
They do exactly the same as the Void Team just at a smaller scale, they are not magically more competent or better. I would argue that their patches are a lot less used than Voids package repository.

[fosslinux]

This link shows that LibreSSL has more development than https://cpp.libhunt.com/compare-openssl-vs-libressl

Untrue, look at commit histories.

In this old link, it compares OpenSSL vs LibreSSL, showing how security is applied in LibreSSL code.
https://resources.infosecinstitute.com/libressl-the-secure-openssl-alternative/
OpenSSL has a design problem, it is susceptible to serious flaws

(emphasis mine); yes, it is old, and that's the problem. OpenSSL's codebase quality and security auditing has increased greatly in the last 5 years. I would have agreed with you some time ago. Nowdays, LibreSSL gets much less auditing, has much fewer developers working on LibreSSL-portable, and has far less architecture support.

quantity of developers in OpenSSL does not mean quality in the code

I agree, but it does mean that something has to lose attention. Often, this is performance, or architecture support, and even build support/code quality (especially in libressl-portable), as @q66 can attest to.

OpenSSL has a lot of legacy OS discontinued

Care to elaborate?

[bugcrazy]

This link shows that LibreSSL has more development than https://cpp.libhunt.com/compare-openssl-vs-libressl

Untrue, look at commit histories.

In this old link, it compares OpenSSL vs LibreSSL, showing how security is applied in LibreSSL code.
https://resources.infosecinstitute.com/libressl-the-secure-openssl-alternative/
OpenSSL has a design problem, it is susceptible to serious flaws

(emphasis mine); yes, it is old, and that's the problem. OpenSSL's codebase quality and security auditing has increased greatly in the last 5 years. I would have agreed with you some time ago. Nowdays, LibreSSL gets much less auditing, has much fewer developers working on LibreSSL-portable, and has far less architecture support.

quantity of developers in OpenSSL does not mean quality in the code

I agree, but it does mean that something has to lose attention. Often, this is performance, or architecture support, and even build support/code quality (especially in libressl-portable), as @q66 can attest to.

OpenSSL has a lot of legacy OS discontinued

Care to elaborate?

LibreSSL has an independent audit, which can be seen on this link, with a report by a member of Void Linux.
https://blog.doyensec.com/2020/04/08/libressl-fuzzer.html

This academic thesis "Analysis of software vunerabilities through historical data" shows comparative graphs of CVEs. http://lup.lub.lu.se/student-papers/record/8923711/file/8923713.pdf

OpenSSlL has a bad design, which favors serious vulnerabilities, this has not been fixed, as this is part of the structural code of OpenSSL, over the years, it has maintained serious vulnerabilities that affect linking software.

https://news.ycombinator.com/item?id=22935221

https://github.com/openssl/openssl/issues/4729

https://github.com/openssl/openssl/issues/4733

https://hownot2code.com/2016/06/03/evaluate-the-string-literal-length-automatically/#more-172

https://www.viva64.com/en/b/0183/

In these links you can compare the number of CVEs between OpneSSL and LibreSSL.

https://www.cvedetails.com/vulnerability-list/vendor_id-217/Openssl.html

https://www.cvedetails.com/vulnerability-list/vendor_id-97/product_id-30688/Openbsd-Libressl.html

[fosslinux]

CVEs are an interesting thing, and actually a point I brought up much earlier, I think on IRC, when I wasn't convinced of moving back to OpenSSL. Basically, OpenSSL is a far more audited codebase and receives more auditing attention than LibreSSL.

I don't see linters/static analysis code checking tools, alone, as evidence of poor coding practices. An OpenSSL dev said in one of those threads that many are false positives.

In addition, each of the articles you linked above (excluding the recent vuln, which was discussed earlier in this thread) are 3+ years old. Again, I am of the opinion that the code quality has improved in that time.

Anyway, I don't think I'll go back and forth, let others lay down their opinions on your data if they would like.

Thanks for the threads, btw, interesting reads, which do reinforce Void's position for originally changing to LibreSSL... at the time.

[concatime]

I would recommend:

1. changing xbps to bearssl. It has really straight API. One downside is that it does not support TLS 1.3 yet. It’s really easy to build/bootstrap, no perl nor cmake, just pure make.
2. replace LibreSSL by OpenSSL 1.1
3. drop all packages that require OpenSSL 1.0

LibreSSL is intended to be used in OpenBSD. I don’t even know if LibreSSL follows OpenSSL 1.0 or 1.1 API. I had a bug with OCSP in Nginx and it took them 8 months to fix it. See https://github.com/libressl-portable/portable/issues/532. LibreSSL is not a bad project at all, it’s just that it’s meant for OpenBSD.

[q66]

well, ideally we'd go straight with openssl 3.0; it should be out soon, AFAIK.

[fosslinux]

changing xbps to bearssl

Two ssl's is probably a recipe for disaster.

replace LibreSSL by OpenSSL 1.1
drop all packages that require OpenSSL 1.0

Agreed.

LibreSSL is not a bad project at all, it’s just that it’s meant for OpenBSD.

100% agree

[q66]

i think they meant making xbps use only bearssl, which would be fine - you already have multiple TLS implementations in your system (there's openssl/libressl, but also nss, gnutls, etc.)

bearssl is nice, small, and explicitly geared towards security (it performs no dynamic memory allocations for example, AFAIK)

[fosslinux]

Ah, I see.