Vcpkg: Upgrade OpenSSL to 1.1.1

[fcharlie]

Good News OpenSSL 1.1.1 has been released, it supports TLS1.3 RFC 8446.
OpenSSl 1.1.1 is New LTS. OpenSSL 1.1.1 is API and ABI compliant with OpenSSL 1.1.0.

Support for various new cryptographic algorithms including:

• SHA3
• SHA512/224 and SHA512/256
• EdDSA (including Ed25519 and Ed448)
• X448 (adding to the existing X25519 support in 1.1.0)
• Multi-prime RSA
• SM2
• SM3
• SM4
• SipHash
• ARIA (including TLS support)

In fact, some ports cannot be upgraded to the latest version because the version of openssl of vcpkg is too low, such as libssh.

Our previous LTS release (OpenSSL 1.0.2) will continue to receive full support until the end of this year. After that it will receive security fixes only. It will stop receiving all support at the end of 2019. Users of that release are strongly advised to upgrade to OpenSSL 1.1.1.

It's time to upgrade OpenSSL.

See: https://www.openssl.org/blog/blog/2018/09/11/release111/

Here are the ports that depend on openssl:

|ports|support OpenSSL 1.1.1|
|---|---|
|librabbitmq|✔|
|aws-sdk-cpp|✔|
|wt|✔|
|azure-c-shared-utility|✔|
|libimobiledevice||
|yara|✔|
|libwebsockets|✔|
|podofo|✔|
|thrift|✔|
|ffmpeg|✔|
|qpid-proton||
|folly|✔|
|libssh|✔ (0.8.* Only 1.1.*) (2019-07 move to mbedTLS)|
|paho-mqtt||
|websocketpp|✔|
|apr-util|✔|
|libarchive|✔|
|cppcms|✔|
|libmysql|✔|
|mongo-c-driver|✔|
|fastrtps|✔|
|freerdp|✔|
|qt5-base|✔ (5.13)|
|caf|✔|
|curl|✔|
|opusfile|✔|
|librtmp||
|libevent|✔|
|mosquitto|✔|
|uwebsoockets|✔|
|libgit2|✔|
|libpq|✔ (11.4 test OK, ubuntu 18.04 apt install openssl 1.1.1)|
|boost-asio|✔|
|cpprestsdk|✔|
|libtorrent||
|wangle|✔|
|grpc|✔|
|libssh2|✔|

[fcharlie]

@ras0219-msft @alexkaratarakis

[madig]

I think this calls for splitting the openssl package into a 1.0.x and 1.1.x branch. The API changed considerably in 1.1.x and older applications must be updated...

[zhulika]

In case someone else is tripped up by this, I'd like to point out that this problem may also cause vcpkg's integration for Visual Studio to break other projects. It may not spring to mind when you first see the errors, because openssl tends to be installed implicitly as a dependency by vcpkg, but vcpkg's version of OpenSSL will take precedence during the link phase over any local copy of OpenSSL that is part of the project's distribution.

I have just spent some time tracing the cause of a bunch of duplicate-symbol linker errors when trying to build NodeJS and it turned out to have been the copy of OpenSSL 1.0.2 that is being inserted at the top of MSBUILD's stack by vcpkg's integration with Visual Studio.

[fcharlie]

@ras0219-msft @alexkaratarakis Only a handful of ports do not support OpenSSL 1.1.1

[alexkaratarakis]

Very nice! qt5 concerns me a bit; the others look more easily patchable.

[fcharlie]

@alexkaratarakis https://bugreports.qt.io/browse/QTBUG-67463

[anluoma]

Just asking that are we waiting for Qt bug to finish before this gets fixed? We are interested of updating to 1.1.1 as well. Or is my assumption wrong?

[jozefizso]

Is there OpenSSL 1.1 port available which can be used internally?

It looks like Qt will block this port for some time.

[Rastaban]

it looks like PR #4983 may be dependent on this.

[mjaafar]

Hi there I want to know if there is any chance that it supports EVP_sha256. Thanls cause I do need it in my integration https://github.com/Microsoft/vcpkg/issues/5529

It the other hand If there is any possibility to check if there is an option in the confiuration easy and fast to activate.

[fcharlie]

@mjaafar See:
https://github.com/openssl/openssl/blob/OpenSSL_1_1_1b/include/openssl/evp.h#L700-L718

[mjaafar]

The issue is related to configure the default options put sha256 in no option I hope in openssl 1.1.1 integration it will be activated @fcharlie

[illera88]

any update here? there are several ports that depends on this

[fcharlie]

@illera88 It seems difficult, OpenSSL 1.1.1 for the UWP transplant for a long time without movement, I recommend that you build your own OpenSSL 1.1.1, which is much better than waiting to be built with vcpkg.

[HLXEasy]

Is there some kind of a roadmap? It would be very cool to use the whole vcpkg content but as long as OpenSSL is stuck on 1.0.x, it's impossible to migrate.

[MouriNaruto]

I have opened a new PR in the offical OpenSSL repo. (https://github.com/openssl/openssl/pull/8917)

[HLXEasy]

Any news here?

[illera88]

Waiting for OpenSSL devs to merge https://github.com/openssl/openssl/pull/8917

[HLXEasy]

Nice, openssl/openssl#8917 get's merged some minutes ago.

[HLXEasy]

@MouriNaruto What do you think? Which steps are the next ones to get this PR closed?

[MouriNaruto]

@HLXEasy I have open an issue. (https://github.com/openssl/openssl/issues/9125)

[illera88]

https://blog.qt.io/blog/2019/06/17/qt-5-12-4-released-support-openssl-1-1-1/

Qt can now use OpenSSL 1.1.1

[kelteseth]

@illera88 Qt 5.13 Apps now __require__ OpenSSL 1.1.1 and Qt itself does not ship it with the binaries. I was hoping to simply use vcpkg for this...

[fcharlie]

Libssh now starts using mbedtls instead of OpenSSL. see: https://github.com/microsoft/vcpkg/pull/4983#issuecomment-508518581. Note that the author of that PR is the maintainer of libssh.

[illera88]

It's better moving out from using OpenSSL than leading with their maintainers... Good decision from libssh maintainer

[fcharlie]

Building openssl 1.1.1 without using vcpkg is easier. I built curl using my own script (depending on openssl 1.1.1). https://github.com/fstudio/clangbuilder/blob/master/sources/wincurl/wincurl.ps1

No matter what package management tool you use, you may end up being bothered by the fact that this main dependency cannot be upgraded in time. Therefore, it may be better to build from source.

[jozefizso]

Any updates for supporting the OpenSSL 1.1.1 port?

[wegylexy]

It's been a year. I realize that OpenSSL changed a lot since 1.1.0 for building on Windows. Has someone got the time to make any progress?

[fcharlie]

OpenSSL 1.0.2 EOL 2019-12-31

Reference: https://www.openssl.org/news/secadv/20190910.txt

OpenSSL 1.0.2 is currently only receiving security updates. Support for 1.0.2
will end on 31st December 2019.

[fcharlie]

The current upgrade to openssl 1.1.1 has been successful, but many ports have not been tested.
Branch: https://github.com/fcharlie/vcpkg/tree/openssl
RP: https://github.com/microsoft/vcpkg/pull/8142

[alcroito]

Would like to chime in and say that Qt 6 built with CMake would also benefit from having openssl 1.1 in vcpkg, instead of forcing users to download and install a package manually.

[dilin-MS]

Hi, any update on openssl exposing option to override version instead of hardcoded to 1.0.2s?

We rely on azure iot c sdk which depends on openssl. This openssl vcpkg hardcoded to 1.0.2 cause azure iot c sdk to be unable to be compiled with openssl 1.1.1. (Cross-compile application with openssl 1.0.2 fails in openssl 1.1.1 environment #1341)

Any plan or update on enabling openssl 1.1.1? Thanks!

[fcharlie]

@dilin-MS

See: https://github.com/microsoft/vcpkg/pull/8566

Some ports have not been migrated and upgrade to openssl 1.1.1 is blocked

[SilviuArdelean]

Looking forward to see the vcpkg based on openSSL 1.1.1+.
Accidentally, it caused me some headaches the last period.
https://silviuardelean.ro/2019/12/14/openssl-vs-vcpkg-some-strange-experiences/

[qis]

I think this calls for splitting the openssl package into a 1.0.x and 1.1.x branch. The API changed considerably in 1.1.x and older applications must be updated...

Stalling this issue solved the problem of requiring to split it into two versioned ports. The best course of action would be to just merge it and fix remaining problems afterwards.

Since I don't believe that there will be any major development done in the next 46 hours, which you have left until 1.0 is officially unsupported anymore, you could just merge it and be done with it!

P.S.: I'm talking about https://github.com/microsoft/vcpkg/pull/8566.

[dhkatz]

Was going to write a port for Valve's GameNetworkingSockets but the build fails because it requires at least OpenSSL 1.1.0.

This is just going to end up causing more and more problems for upgrading and adding new ports in the future the longer OpenSSL isn't updated.

[JackBoosY]

If the interface changes too much after openssl update, I think it is better to create a new port name openssl1_1.

[qis]

If the interface changes too much after openssl update, I think it is better to create a new port name openssl1_1.

@JackBoosY I strongly disagree and urge you not to do it! This is not a simple major release!

Using OpenSSL 1.0 from now on is not only a bad choice, it's outright dangerous for both, developer and his customers. Leaving it as the default and requiring the user to tag 1_1 will result in tragedy (lost money, lost reputation, lawsuits, etc.).

Look at Linux distros: they didn't add OpenSSL 1.1 - they replaced OpenSSL and added OpenSSL 1.0:

openssl/bionic-updates,bionic-security,now 1.1.1-1ubuntu2.1~18.04.4 amd64 [installed,automatic]
  Secure Sockets Layer toolkit - cryptographic utility

openssl1.0/bionic-updates,bionic-security 1.0.2n-1ubuntu5.3 amd64
  Secure Sockets Layer toolkit 1.0 - cryptographic utility

Using OpenSSL 1.0 is a risk now. Let the users make a conscious decision if they want to do it. I'd even suggest outright removing 1.0 and treating it like all version upgrades in vcpkg: You want the old stuff? Use an old vcpkg commit!

[JackBoosY]

@qis If we completely upgrade openssl without keeping the old version, we must fix many ports that currently do NOT support openssl 1.1.
Updates for each port in vcpkg must be compatible with ports that depend on it.

So my suggestion is to add a new port first, support the new version of openssl as other ports update, and then remove the old version of openssl.

[dhkatz]

But library maintainers have known for over a year that OpenSSL was ending its support for 1.0.2. 1.0.2 ended main LTS support in 2018! It was only receiving security updates in 2019.

As the main post shows, a lot more ports support 1.1.1 than don't. So many newer ports would also probably require at least 1.1.0. I don't see the point of holding back this change for a few stragglers.

[jozefizso]

The package name should stay openssl.

As 1.0 is totally unsupported now, it does not make sense to build other packages against such version.

Does somebody really compile for outdated security library? They can checkout earlier git release and/or maintain their own local version.

PS: we must maintain local version of openssl 1.1 because vcpkg was not able to provide the most recent release for several months now. And this is huge pain for us.

[fcharlie]

@jozefizso This is actually a common problem with package management software, and specific software packages are always difficult to upgrade in time. Actually when I use openssl, I always manually build from source.

[qis]

@jozefizso I think you misunderstood dhkatz - you two argue for the same outcome.
@fcharlie We currently use the PR to build OpenSSL with vcpkg and it works quite well:

git clone [email protected]:microsoft/vcpkg && cd vcpkg
git remote add neumann [email protected]:neumann-a/vcpkg
git fetch neumann
git merge --strategy-option theirs neumann/libpq_with_openssl_111d

[JackBoosY]

I hope #8566 can solve these problems completely.

[MikeGitb]

Here is another vote for upgrading the openssl package. If absolutely necessary add a openssl-1.0 package, but don't let the unsupported version stay the default.

The other way around only creates more work (first pointing compatible packages to 1.1 and then back to main), more people with insecure software configurations and less visibility of the problem.

[dhkatz]

Just because I care so much I decided to go through every single port that depended on OpenSSL in some way and check if it had support for OpenSSL > 1.1

I wouldn't consider this an extremely thorough search, but my process was essentially:

• Check the README for mentions of OpenSSL version
• Check the code for mentions of OpenSSL version or function calls
• Check commit and PR history for addition of 1.1 support

Any port with a check I'm pretty sure will support 1.1, and with a question I couldn't really tell but nothing suggests it wouldn't work. Any port with an X, I believe may not work due the age of the library and its use of OpenSSL.

Please note that this isn't suggesting the current versions of all ports with checks will work with OpenSSL, I'm sure many require updates but there are at least newer versions that DO support OpenSSL 1.1

Port| OpenSSL 1.1 Support | Port Updated (Supports 1.1+)
:-----:|:-----:|:-----:
ace| ✔️| ✔️
amqpcpp| ✔️| ✔️
apr-util| ✔️ | ❔
aws-sdk-cpp| ✔️ | ✔️
azure-c-shared-utility| ✔️ | ✔️
boost-asio| ✔️ | ✔️
caf| ✔️ | ✔️
cppcms| ✔️ | ❔
cppfs| ❔ | ❔
cpprestsdk| ✔️ | ✔️
curl| ✔️ | ✔️
evpp| ❌ | ❌
fastrtps| ✔️ | ❌
ffmpeg| ✔️ | ❌
fizz| ✔️ | ❔
fmi4cpp| ✔️ | ❔
folly| ✔️ | ❔
freerdp| ✔️ | ❔
freetds| ✔️ | ❔
grpc| ✔️ | ❔
hiredis| ❌ | ❌
ixwebsocket| ✔️ | ❔
libarchive| ✔️ | ❔
libevent| ✔️ | ❔
libgit2| ✔️ | ❔
libimobiledevice| ❔ | ❔
libmysql| ✔️ | ❔
libnice| ❔ | ❔
libpq| ✔️ | ❔
librabbitmq| ✔️ | ❔
librdkafka| ✔️ | ❔
librtmp| ❌ | ❌
libsrt| ✔️ | ❔
libssh| ✔️ | ❔
libssh2| ✔️ | ❔
libtorrent| ✔️ | ❔
libu2f-server| ✔️ | ❔
libwebsockets| ✔️ | ❔
libzip| ✔️ | ❔
live555| ❔ | ❔
mongo-c-driver| ✔️ | ❔
mongoose| ✔️ | ❔
mosquitto| ✔️ | ❔
msix| ❔ | ❔
nmap| ✔️ | ❔
opendnp3| ✔️ | ❔
openscap| ❔ | ❔
opusfile| ✔️ | ❔
paho-mqtt| ✔️ | ❔
paho-mqttpp3| ❔ | ❔
podofo| ✔️ | ❔
ppconsul| ✔️ | ❔
proxygen| ✔️ | ❔
python3| ✔️ | ✔️
qpid-proton| ✔️ | ❔
qt5-base| ✔️ | ❔
quickfix| ✔️ | ❔
restbed| ✔️ | ❔
signalrclient| ❔ | ❔
slikenet| ❔ | ❔
thrift| ✔️ | ❔
wampcc| ✔️ | ❔
wangle| ✔️ | ❔
websocketpp| ✔️ | ❔
wt| ✔️ | ✔️
xeus| ✔️ | ✔️
xmlsec| ✔️ | ✔️
yara| ✔️ | ✔️

[JackBoosY]

@dhkatz So, at least we need to support openssl 1.1.

[dhkatz]

@JackBoosY As was mentioned in the original post OpenSSL 1.1.1 is API and ABI compliant with OpenSSL 1.1.0. Any library that has added support for at least 1.1.0 will also support 1.1.1

[JunielKatarn]

@fcharlie Does your port to 1.1.1d build successfully?

[fcharlie]

@JunielKatarn I have closed it because there is better: https://github.com/microsoft/vcpkg/pull/8566

[JackBoosY]

OpenSSl has been updated in #8566, thank you for your contribution, can I close this PR now?

[fcharlie]

@JackBoosY I close this issue now