Synapse: Synapse 1.12.2 Docker image is broken

[alexrashed]

Description

The docker images of v1.12.2 (matrixdotorg/synapse:v1.12.2[-py3]) are broken.
They fail due to missing python requirements:

ERROR:root:Needed pillow>=4.3.0,<7.1.0, got Pillow==7.1.0
Missing Requirements: 'pillow>=4.3.0,<7.1.0'
To install run:
    pip install --upgrade --force 'pillow>=4.3.0,<7.1.0'

Since docker tags can be overwritten, here's the digest for the failing images:
sha256:d37b7f73e2fe915fdefda8f1dd46f288fcf7452553945ca3de3d5051c8dc57c5
(matrixdotorg/synapse:v1.12.2 is the same as matrixdotorg/synapse:v1.12.2-py3)

Steps to reproduce

• Start Synapse using the following docker image: matrixdotorg/synapse:v1.12.2
• You

Version information

• Version: 1.12.2
• Install method: Docker
• Platform: Docker amd64

[clokep]

Thanks for the report. This is being handled in #7208 / #7210.

[richvdh]

I don't really understand why this has happened. The docker image should have been built with a pillow versions that matched the constraint in the dependencies file.

For now I've removed the 1.12.2 images from dockerhub.

[richvdh]

It seems to be due to some odd behaviour with pip install --prefix and pkg_resources.

To improve caching of the build layers, the Dockerfile first does pip install --prefix=/install pillow, and later installs synapse. This means that we end up with two sets of metadata for Pillow in /usr/local/lib:

/ # ls -d /usr/local/lib/python3.7/site-packages/Pillow*
/usr/local/lib/python3.7/site-packages/Pillow-7.0.0.dist-info  /usr/local/lib/python3.7/site-packages/Pillow-7.1.1.dist-info

There is only actually one installed version of pillow, and it is the right one:

/ # cat /usr/local/lib/python3.7/site-packages/PIL/_version.py
# Master version for Pillow
__version__ = "7.0.0"

/ # python3 -c 'import PIL; print(PIL.__version__)'
7.0.0

However, pkg_resources things otherwise:

>>> from pkg_resources import Requirement
>>> r = Requirement.parse('Pillow')
>>> r
Requirement.parse('Pillow')
>>> get_provider(r)
Pillow 7.1.1 (/usr/local/lib/python3.7/site-packages)

I guess we could work around this by not pre-installing Pillow (or installing the right version), but tbh I think #7212 is a better solution.

[richvdh]

fixed for 1.12.3 by #7212