Openssl: Truncated transfers with OpenSSL 1.1.1

[alexh-sauce]

We've run into an issue with a site that isn't working with the latest OpenSSL 1.1.1. It was working on older 1.0.1 versions. The site appears to be running IBM WebSphere.

I've also tested with a manual build of https://github.com/openssl/openssl/commit/9e885a707d604e9528b5491b78fb9c00f41193fc and the latest curl, and that is failing with the same error message.

Any troubleshooting advice would be appreciated. The site is live if you want to test.

Working example with OpenSSL 1.0.1

$ docker run -t -i ubuntu:14.04 bash
root@1d68798f7c9d:/# apt update && apt -y install curl
<...snip...>
root@1d68798f7c9d:/# curl -k https://148.244.57.239/local_pibee/img/Logo_bbva_netcash.png > test.png
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 11305  100 11305    0     0  22903      0 --:--:-- --:--:-- --:--:-- 22931
root@1d68798f7c9d:/# ls -l test.png
-rw-r--r--. 1 root root 11305 Mar 27 22:31 test.png
root@1d68798f7c9d:/# sha1sum test.png
e07a48a91660e6f2f924b7f5071f0fb566940297  test.png
root@1d68798f7c9d:/# dpkg -l openssl
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                                Version                Architecture           Description
+++-===================================-======================-======================-============================================================================
ii  openssl                             1.0.1f-1ubuntu2.27     amd64                  Secure Sockets Layer toolkit - cryptographic utility
root@1d68798f7c9d:/# ldd $(which curl) | grep ssl
        libssl.so.1.0.0 => /lib/x86_64-linux-gnu/libssl.so.1.0.0 (0x00007fcf876e0000)

Failing Example with OpenSSL 1.1.1

$ docker run -t -i ubuntu:18.04 bash
root@1b20824ab9fe:/# apt update && apt -y install curl
root@1b20824ab9fe:/# curl -k https://148.244.57.239/local_pibee/img/Logo_bbva_netcash.png > test.png
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
 70 11305   70  8005    0     0  19716      0 --:--:-- --:--:-- --:--:-- 19716
curl: (56) OpenSSL SSL_read: SSL_ERROR_SYSCALL, errno 104
root@1b20824ab9fe:/# ls -l test.png
-rw-r--r--. 1 root root 8005 Mar 27 22:34 test.png
root@1b20824ab9fe:/# sha1sum test.png
ec21cc61c7665e49a1e90bf20f16f4d86c452acf  test.png
root@1b20824ab9fe:/# dpkg -l openssl
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                                Version                Architecture           Description
+++-===================================-======================-======================-============================================================================
ii  openssl                             1.1.1-1ubuntu2.1~18.04 amd64                  Secure Sockets Layer toolkit - cryptographic utility
root@1b20824ab9fe:/# ldd $(which curl) | grep ssl
        libssl.so.1.1 => /usr/lib/x86_64-linux-gnu/libssl.so.1.1 (0x00007fa6f7888000)

[kaduk]

I believe that (older versions of?) WebSphere are known to be intolerant to certain TLS extensions that openssl first implemented in 1.1.0. You might try disabling EncryptThenMac (as per https://github.com/openssl/openssl/issues/9360#issuecomment-577946461) and/or ExtendedMasterSecret to know what kind of bug report to make to the origin site.

[alexh-sauce]

Thank you! EncryptThenMac made it work!

[kaduk]

To be clear, this is most definitely a bug in the server and I strongly encourage you to attempt to report it to the server's operator. The TLS encrypt-then-mac extension provides protection against attacks like BEAST (since the IBM WebSphere presumably is not new enough to support AEAD ciphers), and disabling it should not be considered a long-term solution.