Uwsgi: openssl 1.1 / uwsgi / psycopg2 - no ciphers error

[cp2587]

Hello,

I have recently upgraded my openssl library to 1.1.0 and since then i get the following error when trying to connect to a database through psycopg2 behind a uwsgi / gevent worker:
(psycopg2.OperationalError) could not create SSL context: library has no ciphers

When i am not using uwsgi (i develop using flask framework), it works fine even with gevent activated. This lead me to think there is some kind of compatibility issue between uwsgi & openssl 1.1.0. Are you aware of any such issue ?

[xrmx]

Are you sure everything is linked to the proper ssl library?

[cp2587]

Not sure how to check this but i get the error both on a debian distribution and my local machine (archlinux) that have only one openssl package installed

[xrmx]

Are psycopg2 / libpq compiled against openssl 1.1? Also check the python you have built the uwsgi plugin for.

[cp2587]

The problem arises from a fresh debian 9 install. I installed uwsgi in the following manner

sudo apt-get install git libpq-dev python-dev libxml2-dev libxslt1-dev build-essential libssl-dev libffi-dev
sudo pip install uwsgi psycopg2

Not sure there is any build going on.

[harel]

Having the same problem - debian 9 (upgrade), python 3.5, uwsgi, psycopg, Django. Postgres uses SSL for the connection (defined in the django config).
Note that django's (extension) shell_plus works fine and connects. Only via uwsgi this error occurs.
Also this worked fine before the python3/debian 9 upgrade

[harel]

Is there anything going on with this issue? Is it a case of rebuilding all the stack from source?

[MatthewLM]

I'm getting the exact same issue. It does not occur when I access the database using psycopg2 outside of uwsgi. It only occurs when running a python WSGI application using uwsgi.

[MatthewLM]

Version 2.0.14 appears to be working OK. The problem is with 2.0.15.

[L226]

I am getting a similar error with v2.0.15. My service (python 2.7, Flask, psycopg2) runs on AWS ECS and needs to connect to a Redshift RDS. Redshift is moving to ACM certs and during the course of this update I started getting OperationalError: could not get home directory to locate root certificate file. Either provide the file or change sslmode to disable server certificate verification. messages during connection attempts. My other ECS services not running uwsgi have no issues with the new ACM configuration.

UPDATE: Downgrading to v2.0.14 did not resolve this issue.

[jleclanche]

Confirming the issue, Debian Stretch / Openssl / uwsgi 2.0.15 / Python 3.5//3.6 / postgresql client 10.1 // redshift. Talk about a heisenbug, I was tearing my hair out.

[axolx]

We're having the same problem migration our Django apps to Stretch. Our version of Stretch has uwsgi 2.0.14 and our Django apps use postgres through psycopg.

[xrmx]

Does this reproduce the issue for you? If not could you please modify it to do so? Do you need a valid connection?

import psycopg2

def application(env, start_response):
    start_response('200 OK', [('Content-Type','text/html')])
    try:
       conn = psycopg2.connect("dbname=foo")
    except Exception as e:
       print(e)
    return [b"Hello World"]

You can run it with:

./bin/uwsgi --wsgi-file foo.py  --http :8000

[jleclanche]

@xrmx it appears a valid connection is needed (with an open port; a socket needs to be created), but it doesn't get far enough to need to know about postgres:

import psycopg2

def application(env, start_response):
        start_response('200 OK', [('Content-Type','text/html')])
        try:
                conn = psycopg2.connect(host="localhost", port=22, sslmode="verify-full")
        except Exception as e:
                print(e)
        return [b"Hello World"]

sslmode needs to be set to verify-full, verify-ca, require or prefer. With prefer, it will first attempt to use ssl, crash, then attempt plaintext auth (and fail since 22 is ssh). Example:

% ./bin/uwsgi --wsgi-file app.py --http-socket '[::1]:8001'                                                                                                           ⇄
*** Starting uWSGI 2.0.15 (64bit) on [Sat Nov 25 13:29:18 2017] ***
compiled with version: 6.3.0 20170516 on 17 November 2017 16:02:11
os: Linux-4.9.0-4-amd64 #1 SMP Debian 4.9.51-1 (2017-09-28)
...
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI worker 1 (and the only) (pid: 16571, cores: 1)
could not create SSL context: library has no ciphers
expected authentication request from server, but received S

[pid: 16571|app: 0|req: 1/1] 0.0.0.0 () {24 vars in 256 bytes} [Sat Nov 25 13:29:19 2017] GET / => generated 11 bytes in 6 msecs (HTTP/1.1 200) 1 headers in 44 bytes (1 switches on core 0)

this is where it fails to create the ssl context:

could not create SSL context: library has no ciphers

and here it attempts to plaintext auth:

expected authentication request from server, but received S

[KensoDev]

We dealt with this very issue all morning.
Downgrading to 2.0.14 did not help in our case.

We are not using SSL in uwsgi as it is behind an Nginx and a load balancer, what solved the issue for us was installing libssl-dev after installing uwsgi.

For reference, here is what the docker container build looks like

FROM python:slim-stretch


ARG EXTRA_INDEX_URL
ENV EXTRA_INDEX_URL ${EXTRA_INDEX_URL}

ENV CORE_PACKAGES locales
ENV BUILD_PACKAGES build-essential libffi-dev libpcre3-dev libpq-dev

RUN apt-get update && \
    apt-get install -y --no-install-recommends ${CORE_PACKAGES} ${BUILD_PACKAGES} && \
    pip install --upgrade uwsgi && \
    apt-get install -y --no-install-recommends libssl-dev && \
    apt-get autoremove -y && \
    rm -rf /var/lib/apt/lists/*

[dcfranca]

I was facing a similar issue, what did the trick for me was downgrading Python3.6 to Python3.5

[xrmx]

A workaround would be to install psycopg2 as source and not as wheel (pip install --no-binary). Or rebuild uwsgi without ssl support if you don't use it.
See #1590 for more details

[Jin-Xu]

Ah, installing from source did the job, thanks.

[justinmayer]

As @xrmx noted, one workaround is to ensure the relevant line in requirements.txt specifies source-only installation. For example:

psycopg2==2.7.4 --no-binary :all:

[HalisCz]

Is there any solution or other workaround available?

[xrmx]

@HalisCz You can read the last 3 comments above yours

[HalisCz]

@xrmx I had actually, but so far there are only downgrades and workarounds, and all of them are few months old. Therefore, I was just checking that there is still no available "clean" solution.

[victorbordo]

Does anyone know how to specify the source-only installation in a Pipfile? I can't find docs on how to do this.

I modified requirements.txt file to include psycopg2==2.7.4 --no-binary :all: and then ran pipenv install -r requirements.txt to generate a new Pipfile.

The resulting line is psycopg2-binary = "==2.7.6.1" which doesn't include --no-binary :all:. Any ideas?

[hbielenia]

Important: the solution posted above, changing the relevant requirements.txt line to psycopg2==2.7.4 --no-binary :all:, affects all your requirements! If you only want to force source installation for psycopg2, the line should be psycopg2==2.7.4 --no-binary psycopg2.

Futhermore, from psycopg2 2.8, this will no longer be needed as the package will default to source installation, a move caused precisely by this issue: http://initd.org/psycopg/articles/2018/02/08/psycopg-274-released/

[thornycrackers]

I was having this issue with psycopg2-binary==2.7.4 but I read through https://github.com/unbit/uwsgi/issues/1651 and was able to solve my issue with this by upgrading to psycopg2-binary==2.8.3.