Homebrew-core: [email protected] Migration [Tracking]

[DomT4]

Borrowing my own comment from another thread:

I don't have a lot of interest in carrying around what is essentially a feature patch (support for 1.1.0) given OpenSSL are supporting 1.0.2 as an LTS branch. Not planning to leap over upstream releases unless absolutely necessary for whatever reason.

Transition is also complicated by potentially overlapping linkage, so for example, if you brew install curl --with-openssl against OpenSSL 1.1.0 but you add --with-libssh2 as well which remains linked against OpenSSL 1.0.2, we're not really sure how predictably that'll behave. Ideally, the plan is:

• Anything standalone (i.e. nothing in brew uses) that supports OpenSSL 1.1.0 can use this new formula as the dependency.
• Anything else needs to be upgraded by tree, so to update curl you'd want libssh2 to be using OpenSSL 1.1.0 first, etc.

We may well change that plan as time passes; If I could I'd have the entire ecosystem using OpenSSL 1.1.0 tomorrow but it really depends on how quickly we can get this done without breaking significant things for users.

Immediate potential deps tree:

brew deps --include-build --include-optional --1 <formula>

Usage or potential usage list:

brew uses <formula> --include-build --include-optional

Generate a build report for addition to the list:

brew gist-logs <formula>

Fair warning: This is going to be a messy list.

• [ ] activemq-cpp - Not used by anything else - Fails to build - https://gist.github.com/anonymous/55392af913363942490f5df2f29fc6ce
• [ ] afflib - Used optionally by bulk_extractor, sleuthkit & autopsy - Fails to build, fixed upstream already, will be in 3.7.9 (?) release.
• [ ] aircrack-ng - Not used by anything else - Fails to build - https://gist.github.com/anonymous/241dee104589d7a37a4ed3861622f0a9
• [ ] alpine - Not used by anything else - Fails to build - https://gist.github.com/anonymous/d8fe6343d990cad18a988efa708fc666
• [ ] amap - Not used by anything else - Builds/runs fine, seemingly, once appdefs files are installed.
• [ ] ansible - Not used by anything else - Fails to build, pyOpenSSL - https://gist.github.com/anonymous/d691b6d13da3a8ee264b4f84c77ef3f0
• [ ] apib - Not used by anything else - Fails to build - https://gist.github.com/anonymous/845ffcb3f795d69c7662e0528bc4e2c2
• [ ] apr-util - Used by subversion, apib - Builds fine, no idea how to test properly.
• [ ] arangodb - Not used by anything else - Fails to build - https://gist.github.com/anonymous/060fa5ca54329bca6713ffc98b6261de
• [ ] asio - Used by cpp-netlib - Fails to build in a big way - https://gist.github.com/anonymous/47f8889e59d9194fd12f875540e7ab11
• [ ] axel - Not used by anything else - Fails to build, fixed upstream already, should be in 2.12 (?).
• [ ] bacula-fd - Not used by anything else - Fails to build - https://gist.github.com/anonymous/8780cbc5dbf28e25d2db7429ab689a74
• [ ] bbftp-client - Not used by anything else - Fails to build - https://gist.github.com/anonymous/1bbe2a9e25e38605fb6657066aaef693
• [ ] bibtexconv - Not used by anything else - Builds/runs fine (Only needs MD5 crypto stuff).
• [ ] bigloo - Not used by anything else - Fails to build - https://gist.github.com/anonymous/161b8539d919e2984c669df34e62c5a0
• [ ] bind - Used by dnsperf - Fails to build - https://gist.github.com/anonymous/68c8cf065be6d48a408126b1ccd864e4
• [ ] bip - Not used by anything else - Fails to build - https://gist.github.com/anonymous/4216da32770ca385e32e5c6a69a65ca0
• [ ] bitchx - Not used by anything else - Fails to build - https://gist.github.com/anonymous/eb52052681e870c52a8438d8dfc1eb3c
• [ ] botan - Used by monotone, qca & softhsm - Fails to build - https://gist.github.com/anonymous/2dd98429848c11f28898517e6196e6c9
• [ ] bro - Not used by anything else - Fails to build, looks unrelated - https://gist.github.com/anonymous/a7e496d762267fff39dd2292c33a5a15
• [ ] btpd - Not used by anything else - Builds fine, looks like only uses SHA1 from Crypto.
• [ ] bulk_extractor - Not used by anything else - Fails to build - https://gist.github.com/anonymous/8dc86676f8ef80008a9c21fa78d644da
• [ ] burp - Not used by anything else - Looks like it fails silently - https://gist.github.com/anonymous/8f9e2429168ceb4a81ee598e2a301948
• [ ] cadaver
• [ ] center-im
• [ ] certbot
• [ ] cfengine
• [ ] charm-tools
• [ ] cherokee
• [ ] clamav
• [ ] conan
• [ ] cpp-netlib
• [ ] cppcms
• [ ] cpprestsdk
• [ ] csup
• [ ] ctorrent
• [ ] ctunnel
• [ ] curl
• [ ] cvsync
• [ ] davix
• [ ] dcmtk
• [ ] dovecot
• [ ] duo_unix
• [ ] duplicity
• [ ] efl
• [ ] ejabberd
• [ ] ekg2
• [ ] elinks
• [ ] engine_pkcs11
• [ ] epic5
• [ ] eralchemy
• [ ] erlang
• [ ] ettercap
• [ ] exim
• [ ] fastd
• [ ] fetchmail
• [ ] ffmpeg
• [ ] folly
• [ ] fossil
• [ ] fq
• [ ] frag_find
• [ ] freeradius-server
• [ ] freeswitch
• [ ] freetds
• [ ] ftimes
• [ ] gammu
• [ ] gearman
• [ ] getdns
• [ ] getxbook
• [ ] git-crypt
• [ ] gkrellm
• [ ] globus-toolkit
• [ ] gloox
• [ ] gpac
• [ ] groonga
• [ ] gsoap
• [ ] gst-plugins-bad
• [ ] gtmess
• [ ] gwenhywfar
• [ ] h2o
• [ ] haproxy
• [ ] hashpump
• [ ] http_load
• [ ] httperf
• [ ] httping
• [ ] httrack
• [ ] hydra
• [ ] icecast
• [ ] ike-scan
• [ ] imap-uw
• [ ] imapfilter
• [ ] innotop
• [ ] inspircd
• [ ] ipmitool
• [ ] ipmiutil
• [ ] ircd-hybrid
• [ ] ircii
• [ ] irods
• [ ] irssi
• [ ] isync
• [ ] john-jumbo
• [ ] knot
• [ ] kore
• [ ] lastpass-cli
• [ ] ld64
• [ ] ldapvi
• [ ] ldid
• [ ] ldns
• [ ] lftp
• [ ] libcapn
• [ ] libcouchbase
• [ ] libdap
• [ ] libevent
• [ ] libewf
• [ ] libexosip
• [ ] libfreefare
• [ ] libgda
• [ ] libimobiledevice
• [ ] liblacewing
• [ ] libmowgli
• [ ] liboauth
• [ ] libopendkim
• [ ] libopkele
• [ ] libp11
• [ ] librdkafka
• [ ] libre
• [ ] libslax
• [ ] libssh
• [ ] libssh2
• [ ] libstrophe
• [ ] libtins
• [ ] libtorrent-rasterbar
• [ ] libu2f-server
• [ ] libwebsockets
• [ ] libxmlsec1
• [ ] libzdb
• [ ] lighttpd
• [ ] links
• [ ] lrdf
• [ ] luvit
• [ ] lynx
• [ ] makepkg
• [ ] mariadb-connector-c
• [ ] mariadb
• [ ] md5sha1sum
• [ ] megatools
• [ ] midnight-commander
• [ ] mit-scheme
• [ ] mitmproxy
• [ ] mktorrent
• [ ] monetdb
• [ ] mongo-c
• [ ] mongodb
• [ ] mongoose
• [ ] monit
• [ ] monkeysphere
• [ ] mosquitto
• [ ] mpop
• [ ] mpw
• [ ] msmtp
• [ ] mutt
• [ ] mycli
• [ ] mydumper
• [ ] mysql-cluster
• [ ] mysql-connector-c++
• [ ] mysql
• [ ] mytop
• [ ] nagios-plugins
• [ ] ncrack
• [ ] neko
• [ ] neon
• [ ] net-snmp
• [ ] nghttp2
• [ ] nginx
• [ ] ngircd
• [ ] nmap
• [ ] node-build
• [ ] node
• [ ] nrpe
• [ ] nsd
• [ ] nut
• [ ] nzbget
• [ ] onepass
• [ ] ooniprobe
• [ ] opensaml
• [ ] opensc
• [ ] openvpn
• [ ] ophcrack
• [ ] opusfile
• [ ] osc
• [ ] osslsigncode
• [ ] passenger
• [ ] pdftoedn
• [ ] pdns
• [ ] pdnsrec
• [ ] percona-server
• [ ] percona-toolkit
• [ ] percona-xtrabackup
• [ ] pev
• [ ] pgcli
• [ ] phantomjs
• [ ] pincaster
• [ ] pjproject
• [ ] pkcs11-helper
• [ ] poco
• [ ] podofo
• [ ] postgres-xc
• [ ] postgresql
• [ ] pound
• [ ] profanity
• [ ] proxytunnel
• [ ] psqlodbc
• [ ] pulledpork
• [ ] pulseaudio
• [ ] pure-ftpd
• [ ] pwntools
• [ ] pwsafe
• [ ] pyenv
• [ ] pypy
• [ ] pypy3
• [ ] python
• [ ] python3
• [ ] qca
• [ ] qt
• [ ] rabbitmq-c
• [ ] radare2
• [ ] rethinkdb
• [ ] rtags
• [ ] rtmpdump
• [ ] ruby-build
• [ ] ruby
• [ ] rust
• [ ] saltstack
• [ ] sane-backends
• [ ] sblim-sfcc
• [ ] scamper
• [ ] scrypt
• [ ] sdhash
• [ ] sgfutils
• [ ] shadowsocks-libev
• [ ] shairport-sync
• [ ] shairport
• [ ] siege
• [ ] silc-client
• [ ] sipp
• [ ] sipsak
• [ ] skipfish
• [ ] slowhttptest
• [ ] slrn
• [ ] sngrep
• [ ] snort
• [ ] snownews
• [ ] socat
• [ ] sofia-sip
• [ ] sops
• [ ] spdylay
• [ ] spiped
• [ ] sqlcipher
• [ ] squid
• [ ] srtp
• [ ] ssldump
• [ ] sslscan
• [ ] sslsplit
• [ ] sstp-client
• [ ] strongswan
• [ ] stunnel
• [ ] stuntman
• [ ] subversion
• [ ] swi-prolog
• [ ] sylpheed
• [ ] sysbench
• [ ] szl
• [ ] tarantool
• [ ] tarsnap
• [ ] tcpflow
• [ ] testssl
• [ ] thc-pptp-bruter
• [ ] thrift
• [ ] tinc
• [ ] tiny-fugue
• [ ] tlsdate
• [ ] tn5250
• [ ] tomcat-native
• [ ] tor
• [ ] trafficserver
• [ ] u-boot-tools
• [ ] unbound
• [ ] unshield
• [ ] upscaledb
• [ ] urweb
• [ ] uwsgi
• [ ] virtuoso
• [ ] voms
• [ ] vsftpd
• [ ] w3m
• [ ] web100clt
• [ ] webfs
• [ ] wget
• [ ] wimlib
• [ ] wrk
• [ ] x11vnc
• [ ] xar-mackyle
• [ ] xaric
• [ ] xml-security-c
• [ ] xml-tooling-c
• [ ] xrootd
• [ ] yara
• [ ] yubico-piv-tool
• [ ] zbackup
• [ ] znc - Fails to build but some upstream activity https://github.com/jimloco/Csocket/pull/64 https://github.com/znc/znc/issues/1310 https://github.com/znc/znc/pull/1311

[alex]

Patches have landed in Python upstream, next releases on all branches will support 1.1.0.

[JLHwung]

@DomT4 How could one generate logs like this? Are there any commands or scripts? If so we can crowdsourcing the build logs on other formulas multiplying other platforms.

[DomT4]

@JLHwung brew gist-logs <formula>.

[JLHwung]

@DomT4 How could one contribute on this list?
For example

• [ ] cadaver Not used by anything else, builds fine, no idea how to test properly.

[DomT4]

Maintainers have direct access. Everyone else will need to leave a comment & wait for a maintainer to incorporate it into the master list. Generally the test process is something like:

• Change any mention of depends_on "openssl" or Formula["openssl"] in formula to [email protected].
• brew install -s <formula>
• brew test -v <formula> (If formula defines a test).
• brew gist-logs <formula>
• Dump comment here with the formula name, say build failed/succeeded, provide gist-logs.

[JLHwung]

@DomT4 don't forget to mention that use
brew uses <formula> --include-build --include-optional
to detect whether this formula is dependent of other formulas.

It is also worthy noting that some formula depends on OpenSSL optionally, so one needs go through the option to make formula building against [email protected]

Here is a list of c initials.

• [ ] cadaver Not used by anything else, builds fine, no idea how to test properly. https://gist.github.com/anonymous/d74839b0aaf7a0d995ac0a0f92c86f09
• [ ] center-im Not used by anything else, builds fine, test passed
  https://gist.github.com/anonymous/e93d431c89f47b9c91620d0ac474d30d
• [ ] certbot Not used by anything else, builds failed, may be related with Python upstream
  https://gist.github.com/JLHwung/ec8cb634aa0e74dceee076e86a180055
• [ ] cfengine Not used by anything else, builds failed
  https://gist.github.com/JLHwung/dbd4d57be2cb1a01d0bcfbf2d5d58561
• [ ] charm-tools Not used by anything else, builds failed, seems same error with certbot
  https://gist.github.com/JLHwung/1d87435c2234bd0d17b59f449e418315
• [ ] cherokee Not used by anything else, builds failed
  https://gist.github.com/79a4dce42d7eab3c7b8b0ffe6f332307
• [ ] clamav Not used by anything else, builds failed on configuration, complained does not found OpenSSL even if [email protected] is installed
  https://gist.github.com/JLHwung/ff173f97a66b089639c4af88604e4a73
• [ ] conan Not used by anything else, builds failed
  https://gist.github.com/b9b2f51f62f0bc60d7875592273caf5d
• [ ] cpp-netlib Used by osquery, builds failed
  https://gist.github.com/289b9eb3e6c97c9cb72d00796149b498
• [ ] cppcms Not used by anything else, builds fine, test passed
  https://gist.github.com/3dba3e7934463ec2848baa7d702d3612
• [ ] cpprestsdk Not used by anything else, builds failed
  https://gist.github.com/a31ee7e2370d0c384b8ec8c03dbde009
• [ ] csup Not used by anything else, builds fine, test passed
  https://gist.github.com/71799fd0b3fe007360587b18b8504b52
• [ ] ctorrent Not used by anything else, builds fine, test passed, no logs
• [ ] ctunnel Not used by anything else, builds failed
  https://gist.github.com/896ab02c880146cc6fec3a239544705a
• [ ] curl Used by freeswitch,lnav,saldl,strongswan,xml-tooling-c,homebrew/php/php54-http,homebrew/php/php56-http,git-ftp,powerman,shibboleth-sp,weechat,homebrew/php/php53-http,homebrew/php/php55-http,homebrew/php/phpbrew
  builds fine, test passed, but we knew libssh2 and nghttp2, etc does not support 1.1 now.
  https://gist.github.com/c85e2a1524e05cafe758bc0b54bae742
• [ ] cvsync Not used by anything else, builds fine, test passed
  https://gist.github.com/7ea3b039da854ec5a7bb3da8f5deeb11

[chdiza]

mutt just officially gained support for building openssl 1.1 in HEAD.

[vszakats]

libssh2 does support [email protected] in HEAD now.

[JLHwung]

@vszakats Thank you so much for reminding people of the progress. I raise pull request #4734 to have libssh2 HEAD depends on [email protected].

[DomT4]

Closing this because of https://github.com/Homebrew/brew/pull/1283. Thank you to those who helped work on it a little so far.

[MikeMcQuaid]

Let's leave it open, others will need to continue this work.

[DomT4]

I was mostly hoping to avoid the notifications. I'll try the unsubscribe notifications button instead.

[MikeMcQuaid]

@DomT4 Thanks!

[MikeMcQuaid]

Based on https://github.com/Homebrew/homebrew-core/issues/6828 I think we should change approach here and stop migrating things to use [email protected] until we can do it all in a large revision PR i.e. all the projects using OpenSSL 1.0 currently work with OpenSSL 1.1. Migrating things one-by-one is going to result in mixed dependency trees and that's not going to be good. CC @tdsmith and generally @Homebrew/maintainers for this.

[gelraen]

Please consider rolling this back for broken packages. For example, ansible has been broken for 3 months now.

[ilovezfs]

@gelraen what issue(s) are you referring to specifically?

[gelraen]

@ilovezfs brew install ansible chokes on installing openssl-1.1. I've worked it around by manually editing ansible.rb locally to use any version of openssl, not 1.1 specifically.

[ilovezfs]

@gelraen that's an issue specific to your system. [email protected] is a non-optional dependency of ansible, and we have pre-built binary bottles for Sierra, El Capitan, and Yosemite, which were all built against 1.1. Please open an issue.

[gelraen]

Sorry, I was indeed too quick to judge. It was broken for me because I have TOP env var set in my shell for completely unrelated reason (see gist for the full error message). Should I open an issue for general env sanitizing or just this specific instance?

[ilovezfs]

@gelraen you can make a note of the situation here: https://github.com/Homebrew/brew/issues/932

[MikeMcQuaid]

Closing as we're not actively migrating these formulae now.