Synapse: Missing python dependencies

[mvgorcum]

Description

Synapse v0.23.1 fails to start in Docker without the python packages enum34 and ipaddress installed.

Steps to reproduce

• Build Synapse docker (no errors here)
• Try to run the docker container
• Container starts fine, but synapse doesn't start, throwing the following error:
  2017-10-21 19:15:33,810 INFO exited: matrix (exit status 1; not expected)
2017-10-21 19:15:35,813 INFO spawned: 'matrix' with pid 36
ERROR:root:Can't import 'OpenSSL' which is part of 'pyopenssl>=0.14'
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/synapse/python_dependencies.py", line 107, in check_requirements module = __import__(module_name)
File "/usr/local/lib/python2.7/dist-packages/OpenSSL/__init__.py", line 8, in <module> from OpenSSL import crypto, SSL File "/usr/local/lib/python2.7/dist-packages/OpenSSL/crypto.py", line 12, in <module> from cryptography import x509
File "/usr/local/lib/python2.7/dist-packages/cryptography/x509/__init__.py", line 7, in <module> from cryptography.x509 import certificate_transparency
File "/usr/local/lib/python2.7/dist-packages/cryptography/x509/certificate_transparency.py", line 8, in <module>from enum import Enum
ImportError: No module named enum
Missing Requirement: Can't import 'OpenSSL' which is part of 'pyopenssl>=0.14'
To install run:
pip install --upgrade --force "pyopenssl>=0.14"
• This can be fixed my running pip install --upgrade --force "pyopenssl>=0.14" inside of the docker container. This reinstalls a few different python packages, but only 2 are new: enum34 and ipaddress

Describe how what happens differs from what you expected:

The dockerfile tells pip to install all the dependencies for synapse, and it builds without error, but synapse doesn't run without installing the 2 packages mentioned.

Version information

• Homeserver: On a homeserver (in docker)

• Version info version 0.23.1

• Install method: Inside docker container using git clone --branch master --depth 1 https://github.com/matrix-org/synapse.git and pip install --upgrade --process-dependency-links

• Platform:
• Inside docker based on debian jessie. The error occurs both on a debian 9.2 and ubuntu 16.04 host.

I think this could be fixed by adding enum34 and ipaddress to python_dependencies.py, but if this problem only occurs when running in docker it can also be fixed by explicitly installing both packages in the dockerfile.

[edit] Note that when running synapse with enum34 installed, but not ipaddress the following error is thrown:
2017-10-22 00:33:31,230 INFO exited: matrix (exit status 1; not expected)
2017-10-22 00:33:34,234 INFO spawned: 'matrix' with pid 43
ERROR:root:Can't import 'OpenSSL' which is part of 'pyopenssl>=0.14'
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/synapse/python_dependencies.py", line 107, in check_requirements module = __import__(module_name)
File "/usr/local/lib/python2.7/dist-packages/OpenSSL/__init__.py", line 8, in <module> from OpenSSL import crypto, SSL
File "/usr/local/lib/python2.7/dist-packages/OpenSSL/crypto.py", line 12, in <module> from cryptography import x509
File "/usr/local/lib/python2.7/dist-packages/cryptography/x509/__init__.py", line 8, in <module> from cryptography.x509.base import (
File "/usr/local/lib/python2.7/dist-packages/cryptography/x509/base.py", line 16, in <module> from cryptography.x509.extensions import Extension, ExtensionType
File "/usr/local/lib/python2.7/dist-packages/cryptography/x509/extensions.py", line 10, in <module>
import ipaddress
ImportError: No module named ipaddress
Missing Requirement: Can't import 'OpenSSL' which is part of 'pyopenssl>=0.14'
To install run:
pip install --upgrade --force "pyopenssl>=0.14"

[ivilata]

I've also encountered this error after installing Synapse 0.25.1 and 0.26.0 to a virtualenv. I use the following commands:

git checkout v0.26.0
virtualenv ~/env/py/matrix-0.26.0
. ~/env/py/matrix-0.26.0/bin/activate
pip install --upgrade setuptools
pip install .

When trying to run Synapse with --generate-config for testing, I get the pyopenssl error. I need to run this for Synapse to function:

pip install enum34
pip install ipaddress

[richvdh]

The problem here seems to be a problem with old versions of pip.

cryptography depends on enum34 and ipaddress, but both are conditional on the python version (since they are only needed for python < 3.4). It looks like the rather ancient version of pip in jessie doesn't understand the conditional requirement, and doesn't install those packages at all.

One solution is to pip install --upgrade pip before installing anything else, but be aware that we're dropping support for jessie anyway.