Warehouse: psycopg2 warning

[brainwane]

When I make initdb or make tests, one error I see in the logs is:

/opt/warehouse/lib/python3.6/site-packages/psycopg2/__init__.py:144: UserWarning: The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use "pip install psycopg2-binary" instead. For details see: http://initd.org/psycopg/docs/install.html#binary-install-from-pypi.

[ewdurbin]

this is rather prescriptive, @dstufft thoughts on if we should just upgrade to psycopg2-binary now? https://pypi.org/project/psycopg2-binary/

[di]

Kind of interesting why they're doing this, more details:

• https://www.postgresql.org/message-id/CA%2Bmi_8bd6kJHLTGkuyHSnqcgDrJ1uHgQWvXCKQFD3tPQBUa2Bw%40mail.gmail.com
• http://initd.org/psycopg/articles/2018/02/08/psycopg-274-released/

Looks like the -binary release still offers a source distribution, so switching to it should be equivalent.

Along these lines, we could probably use a "how to add new dependencies to warehouse" guide somewhere...

[jleclanche]

I do not recommend switching to psycopg2-binary. It breaks any dependency which depends on psycopg2, such as sqlalchemy's postgres bindings, even though they both provide the import.

Context: https://github.com/psycopg/psycopg2/issues/674

[di]

@jleclanche I'm not really sure what you mean by "breaks", if they provide the same import and (presumably) the same functionality, what's broken?

[dstufft]

The dependency metadata, the system will see "I require a project named psycopg2 of at least version X" installed, and that won't be (because psycopg2-binary will be installed instead). It'll be compatible at the import level, but not at the packaging metadata level.

[di]

That's what I figured. At least for us, this wouldn't break anything (we're not even installing sqlalchemy with the postgres extra, which is what adds the pyscopg2 dependency).

[Cykooz]

But the psycopg2-binary has not resolved problem with libssl.

Imagine, what if near future all python libraries will be depends from psycopg2-binary instead of psycopg2. What will this situation be different from when there was only one version of psycopg2?

[drunkwcodes]

If uploader can determine the default dowloading format on the server-side when people downloading with pip install, It will solve this issue IMHO.

Pip (or clients) can't determine which version is recommended by itself sometimes.

Here is a plan for precedence logic of downloading format:

pip_options_specified or uploader_recommended or wheel_preferred

ref: https://github.com/pypa/wheel/issues/215

[di]

As of version 2.8, the psycopg2 package no longer publishes built distributions for manylinux.

We'll probably want to switch to psycopg2-binary to avoid building from source.

[dstufft]

Is building from source a big deal for us? We already build a number of other things from source IIRC?

[drunkwcodes]

To bootstrap in an uncertain circumstance?

[dstufft]

This is solved now due to https://github.com/pypa/warehouse/pull/5696.