Psycopg2: install via Python Wheels

[psycoteam]

Originally submitted by: rcorzogutierrez

We can install via Wheels

[dvarrazzo]

Please explain better.

[thedrow]

Wheels allow you to provide a binary distribution of the package with all dependencies already compiled.
Providing wheels will save a lot of installation time and will not require a working compiler.

[dvarrazzo]

@thedrow It takes an OSX platform to compile the OSX wheel and a Win platform to compile a Win wheel I guess?

[thedrow]

Yes. Platform specific wheels for Linux are not permitted though.

[dvarrazzo]

@thedrow are you buying me the hardware to create these packages?

[thedrow]

You can do that with Travis.
It's a matter of allowing to deploy with travis on tags and using both the linux and osx machines.
You can build Windows Wheels using appveyor.

[dvarrazzo]

I have personally no motivation into building the library for platforms I don't use and can't vouch for the quality of the result. What I can do is to provide good quality source code: if building for some platforms is a mess it is a problem of the people who have chosen to use them, not mine.

So, no: wheels are not happening without a significant financial contribution to the project, because for the people I care installing psycopg from source is already piece of cake.

[thedrow]

piece of cake yes? faster? not really.
If anyone else contributes it, we will need your collaboration anyway.

[bmathieu33]

Actually currently if I want to build a wheel (or a binary egg) that's just not possible (python setup.py bdist_(wheel|egg)). This issue is really about giving that possibility.

You, the maintainer, could continue to upload sdist to pypi as you do now. No, you won't have to build and provide binaries on pypi, don't worry about that.

But for people that want to build their own binaries (for use on a private devpi server for example) that would be very useful. Direct benefits: don't have to recompile the same piece of code again and again. Just on CI servers that's additional time not used to actually test; and millions of cpu cycles wasted.

As far as I understand the blocking point is that setup.py is not using setuptools.

Unfortunately the current 'bdist' command is not enough, the resulting archive cannot be used on devpi for example.

So, if we are able to do "python setup.py bdist_wheel", I believe it would be enough.

[rcorzogutierrez]

I could compile for python- weels the version published in the repository pypi with the below command

pip wheel --wheel-dir = / temp / weelRepo psycopg2

and this file was the result
https://mega.co.nz/#!GQRDCJrS!WKUFYNMNn1Fohk4rhk2_FelNms3TETAQFhSUGB6Wrx0
To test, create a vitual environment and just type the command

pip install psycopg2-2.6-CP27-none-win32.whl

and work perfect for me
If you are interesting to the community, share it by uploading it to the repo pypi
regards

[dvarrazzo]

@rcorzogutierrez let me think: I should download an executable file from mega.co.nz and suggest all the world to use it? I see just some _very tiny_ security problems in this solution...

Sorry if I am not "interesting to the community" but the community I'm pretty sure can do without a security threat like that.

[bmathieu33]

Knowing that you can create a wheel with pip wheel is the real news for me, and it's enough to upload my own generated package on my own devpi server (as a linux user, that's the only way I can have pre-built binaries for my environments).
So supporting python setup.py bdist_wheel would be only a tiny improvement.

[rcorzogutierrez]

The idea is to not download the executable file from mega, the best way is to upload to pypi repo (on this subject my previous comment said "share it by uploading it to the repo pypi") only proposed the idea and I shared my results, my intension is that they could assess, including this new form of distribution, but anyway I understand your questions about security
Many maintainers who have incorporated this way, as well http://pythonwheels.com/ describes this site.
I agree with comments @ bmathieu33.
Regards

[rcorzogutierrez]

Apparently there are people who are alternately occupied by this theme
http://www.lfd.uci.edu/~gohlke/pythonlibs/#psycopg

[blueyed]

Please note that wheels may not work in general in this case, depending on the Python configure options (or minor version changes from 2.7.8 to 2.7.9):

Error loading psycopg2 module: .../pyenv-2.7.9/lib/python2.7/site-packages/psycopg2/_psycopg.so: undefined symbol: PyUnicodeUCS2_AsUTF8String

The wheel was (probably) created by the system Python on Ubuntu (2.7.8), which has (IIRC) some non-default configure options that cause this (--enable-unicode=ucs4 ?).

I guess that's an issue with wheels in general, which should indicate this as a flag in the name probably?!

[ashald]

What I'm curios about is that psycopg requires PostgreSQL headers to compile. Or it doesn't matter what version of PostgreSQL you compile against? I mean, 8, 9 etc.

[dvarrazzo]

The headers are only required for the libpq functions signature, it's not required for any server-side structure. The version doesn't matter up to a point: the newer the better. With some old versions of libpq there could be some missing feature but any currently supported PostgreSQL version should do.

[rcorzogutierrez]

this issue can be closed, in my opinion already is resolved. Thanks to all the team psycopg2, excellent work

[dvarrazzo]

Solved, at least for Windows by #368.

[mr-rodgers]

This is failing again for Python 3.5 (no cp35 wheel on PyPI)

[blueyed]

@te-je
That seems to be fixed in 2.6.2: https://github.com/psycopg/psycopg2/pull/370.