Cryptography: Compilation fails on latest version of Alpine due to libressl compatibility issues(?)

[anlutro]

Today, when building a docker image, cryptography 2.2.2 failed to compile, even though it worked fine a few days ago.

My best guess is that this is because libressl2.6-libssl in alpine 3.7 was updated 2 days ago. I also tried with Alpine 3.6 and that broke as well.

I believe this is an issue with Alpine itself, but thought you might want to know.

from alpine:3.7
run apk add --update --no-cache python3 python3-dev gcc musl-dev libffi-dev libressl-dev
run python3 -m venv /venv
run /venv/bin/pip3 install --upgrade pip setuptools
run /venv/bin/pip3 install cryptography==2.2.2

This yields a long list of warnings, but also some errors:

error: 'X509_VERIFY_PARAM_set1_ip_asc' redeclared as different kind of symbol
error: 'X509_VERIFY_PARAM_set1_host' redeclared as different kind of symbol

Note that in this case, we could replace libressl-dev with openssl-dev and everything would probably just work, but we also need to install mariadb-dev which specifically requires libressl.

Alpine bug report: https://bugs.alpinelinux.org/issues/8939

[happyraul]

In case it helps anyone while we wait for a fix, I've gotten a build working by uninstalling libressl-dev, installing openssl-dev, installing cryptography, then reverse the steps before proceeding with other dependencies such as mariadb-dev and/or postgresql-dev, which specifically need libressl.

[m1schka]

@happyraul thx for the workaround. in case you're using docker, this is how it compiles for me:

FROM python:3.6-alpine3.6
[...]
COPY requirements.txt ./
RUN set -e; \
  apk update \
  && apk add --virtual .build-deps gcc python3-dev musl-dev libffi-dev \
  # TODO workaround start
  && apk del libressl-dev \
  && apk add openssl-dev \
  && pip install cryptography==2.2.2 \  
  && apk del openssl-dev \
  && apk add libressl-dev \
  # TODO workaround end
  && apk add postgresql-dev \
  && pip install --no-cache-dir -r requirements.txt \
  && apk del .build-deps

[...]

[gregertw]

I can confirm that I have seen the same thing and that compiling openssl 1.0.2o and then build cryptography works. Here is my Dockerfile:

RUN apk -U add build-base linux-headers libffi-dev mariadb-dev \
    gettext curl gcc musl-dev python3-dev libressl-dev perl
RUN pip install virtualenv
RUN ./build-cryptography.sh
RUN pip install --no-index --find-links=./ cryptography

And the build-cryptography.sh script:

#!/bin/sh
set -e

OPENSSL_VERSION="1.0.2o"
CWD=$(pwd)

virtualenv venv
. venv/bin/activate
pip install -U setuptools
pip install -U wheel pip
curl -O https://www.openssl.org/source/openssl-${OPENSSL_VERSION}.tar.gz
tar xvf openssl-${OPENSSL_VERSION}.tar.gz
cd openssl-${OPENSSL_VERSION}
./config no-shared no-ssl2 no-ssl3 -fPIC --prefix=${CWD}/openssl
make && make install
cd ..
CFLAGS="-I${CWD}/openssl/include" LDFLAGS="-L${CWD}/openssl/lib" pip wheel --no-binary :all: cryptography

[jmaroeder]

Another workaround, if you don't require the latest cryptography, is to just use the apk package:

apk add --no-cache py-cryptography

[cory-klein]

^^ FYI, If you do this, you'll get apk's cryptography package which is cryptography==1.8.1

[seanh]

@m1schka 's solution worked for me too, just added this in the correct place in my Dockerfile (has to be done before apk adding any packages that depend on LibreSSL):

diff --git a/Dockerfile b/Dockerfile
index 50c6e4edf..75344d0f3 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -32,6 +32,30 @@ RUN chown -R hypothesis:hypothesis /var/log/nginx /var/lib/nginx /var/tmp/nginx
 # Copy minimal data to allow installation of dependencies.
 COPY requirements.txt ./

+# BEGIN WORKAROUND
+#
+# TODO: Remove this workaround once https://github.com/pyca/cryptography/issues/4264 is resolved.
+#
+# crypotography (one of the dependencies that would be installed from our
+# requirements.txt file below) currently has a problem building against
+# LibreSSL (LibreSSL is going to be installed as a dependency of some of the
+# apk packages installed below):
+#
+# https://github.com/pyca/cryptography/issues/4264
+#
+# So as a workaround we're going to install cryptography separately now, before LibreSSL gets
+# installed.
+RUN apk update                                  # Needed or `apk add openssl-dev` won't work.
+RUN apk add openssl-dev                         # Build cryptography against OpenSSL, which works,
+                                                # instead of against LibreSSL.
+RUN apk add gcc python-dev musl-dev libffi-dev  # Other dependencies needed to build cryptography.
+RUN pip install --no-cache-dir -U pip
+RUN pip install cryptography==2.1.4
+# Remove OpenSSL again because it conflicts with LibreSSL which is going to be installed as a
+# dependency of some of the apk packages below.
+RUN apk del openssl-dev
+# END WORKAROUND
+
 # Install build deps, build, and then clean up.
 RUN apk add --no-cache --virtual build-deps \
     build-base \

[reaperhulk]

That workaround will actually leave cryptography in a broken state where it can attempt to call functions that it thinks exists in the library and fail at runtime. Your particular workload may not hit that, but be aware. A better workaround would be building cryptography statically against OpenSSL so that it has all the symbols it needs to work (there are instructions on how to do this on cryptography.io, but it is not simple).

4234 may end up addressing this, but at the moment no one has contributed a maintainable approach to adding 2.7.x compatibility so there's no ETA.

[huesforalice]

@reaperhulk that is true and happened for me, but using @gregertw's solution this seems to work.

[reaperhulk]

His fix is indeed safe as a workaround for now.

[seanh]

That workaround will actually leave cryptography in a broken state where it can attempt to call functions that it thinks exists in the library and fail at runtime. Your particular workload may not hit that, but be aware. A better workaround would be building cryptography statically against OpenSSL so that it has all the symbols it needs to work (there are instructions on how to do this on cryptography.io, but it is not simple).

Thanks for pointing that out @reaperhulk . It turns out that we only use cryptography for HKDF key derivation, so we ended up replacing it with python-hkdf instead: https://github.com/hypothesis/h/pull/5056

[anlutro]

I ended up just switching away from Alpine altogether. Their mysql/postgres packages linking specifically against libressl, and them backporting a libressl 2.7 change into 2.6 and 2.5 left a very bad impression on me.

[m1schka]

it seems they've reverted the commit, at least on alpine 3.7 (not yet on 3.6): https://git.alpinelinux.org/cgit/aports/commit/?id=1b2b08e28cbf2539b23cd4fc487cf00caaf19163

just need to wait for the new package

edit: now also on 3.6: https://git.alpinelinux.org/cgit/aports/commit/?id=fa3c527048bbf73082279e893c45b645e037cf9b

[oharlem]

Feels related: https://github.com/gliderlabs/docker-alpine/issues/410

[rahulwa]

I guess this is fixed now. I tried to install it on alpine3.6 with pip and it worked.

[23doors]

I have a different experience. Tried fresh image of 3.7 and python3. My understanding is that we need to wait for libressl-2.6.4-r2 to be pushed. As you can see here: https://pkgs.alpinelinux.org/package/edge/main/x86/libressl 2.6.4-r1 is still the latest version available (with build time pre patch).

[f1nality]

Still not working.. m1schka's solution worked.

[alex]

resolved by #4270

[m1schka]

can confirm, not fixed under 3.7 because no new built yet (https://pkgs.alpinelinux.org/package/v3.7/main/x86/libressl) but fixed under 3.6 (https://pkgs.alpinelinux.org/package/v3.6/main/x86/libressl), where they apparently released the fix.

@alex is there a possibility to release version 2.2.3 after the fixes since it affects so many people?

[reaperhulk]

We're going to do a 2.3 soon, although the fix we did (supporting LibreSSL 2.7) won't help alpine because they backported a feature from 2.7 into 2.6. We can't detect that in the C preprocessor so it will continue to fail :(

[m1schka]

@reaperhulk makes sense.

FYI: they also released the package for 3.7 now (libressl version 2.6.4-r2). compiles for me again under 3.7.

[run2]

@happyraul thx for the workaround. in case you're using docker, this is how it compiles for me:

FROM python:3.6-alpine3.6
[...]
COPY requirements.txt ./
RUN set -e; \
  apk update \
  && apk add --virtual .build-deps gcc python3-dev musl-dev libffi-dev \
  # TODO workaround start
  && apk del libressl-dev \
  && apk add openssl-dev \
  && pip install cryptography==2.2.2 \  
  && apk del openssl-dev \
  && apk add libressl-dev \
  # TODO workaround end
  && apk add postgresql-dev \
  && pip install --no-cache-dir -r requirements.txt \
  && apk del .build-deps

[...]

This was really helpful. Thanks

[nabinkhadka]

All workaround comments are really helpful, but what is the actual issue and is it going to get fixed?

[alex]

This issue was fixed in 2018, as my comment from when this was closed indicates.