psycopg2 2.8 install failed

[ddahan]

After updating my packages using pipenv (via pipenv update command), it tries to update to psycopg2 2.8 and it fails. Here are the error logs:

[pipenv.exceptions.InstallError]:   File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pipenv/core.py", line 1874, in do_install
[pipenv.exceptions.InstallError]:       keep_outdated=keep_outdated
[pipenv.exceptions.InstallError]:   File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pipenv/core.py", line 1253, in do_init
[pipenv.exceptions.InstallError]:       pypi_mirror=pypi_mirror,
[pipenv.exceptions.InstallError]:   File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pipenv/core.py", line 859, in do_install_dependencies
[pipenv.exceptions.InstallError]:       retry_list, procs, failed_deps_queue, requirements_dir, **install_kwargs
[pipenv.exceptions.InstallError]:   File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pipenv/core.py", line 763, in batch_install
[pipenv.exceptions.InstallError]:       _cleanup_procs(procs, not blocking, failed_deps_queue, retry=retry)
[pipenv.exceptions.InstallError]:   File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pipenv/core.py", line 681, in _cleanup_procs
[pipenv.exceptions.InstallError]:       raise exceptions.InstallError(c.dep.name, extra=err_lines)
[pipenv.exceptions.InstallError]: ['Collecting psycopg2==2.8 (from -r /var/folders/ts/pyvl9xds6sj_wwpcmbkn68jr0000gn/T/pipenv-st367xed-requirements/pipenv-pfzfw1b2-requirement.txt (line 1))', '  Using cached https://files.pythonhosted.org/packages/c7/ca/75236b17f1b951950ffc55d657c5aa408d3d0327a1b6c4c0f7cb16ef7e7b/psycopg2-2.8.tar.gz', '    Complete output from command python setup.py egg_info:', '    running egg_info', '    creating pip-egg-info/psycopg2.egg-info', '    writing pip-egg-info/psycopg2.egg-info/PKG-INFO', '    writing dependency_links to pip-egg-info/psycopg2.egg-info/dependency_links.txt', '    writing top-level names to pip-egg-info/psycopg2.egg-info/top_level.txt', "    writing manifest file 'pip-egg-info/psycopg2.egg-info/SOURCES.txt'", '    ', '    Error: pg_config executable not found.', '    ', '    pg_config is required to build psycopg2 from source.  Please add the directory', '    containing pg_config to the $PATH or specify the full executable path with the', '    option:', '    ', '        python setup.py build_ext --pg-config /path/to/pg_config build ...', '    ', "    or with the pg_config option in 'setup.cfg'.", '    ', '    If you prefer to avoid building psycopg2 from source, please install the PyPI', "    'psycopg2-binary' package instead.", '    ', "    For further information please check the 'doc/src/install.rst' file (also at", '    <http://initd.org/psycopg/docs/install.html>).', '    ', '    ', '    ----------------------------------------']
[pipenv.exceptions.InstallError]: ['Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/ts/pyvl9xds6sj_wwpcmbkn68jr0000gn/T/pip-install-x1y4z2jb/psycopg2/']
ERROR: ERROR: Package installation failed...

[dvarrazzo]

The error log explains what to do, as well as the documentation.

[planewryter]

Attempting to install WRDS v3.0.6 in PyCharm_EDU (v2018.3) on macOS v10.14.4 to support an assignment given to my students.

• This install worked without incident about 3 weeks ago.

Today (on a student's machine), the WRDS install failed, with the diagnostic messages (below).

IMPORTANT: WRDS has a dependency on psycopg which CANNOT be resolved by installing psycopg2-binary.

With full appreciation for the demands on a developer who so generously contributes their time and talents by making psycopg available to all...please accept this posting as a request for a prompt fix to this problem. Some of my students are 'dead in the water' and the due date (and end of the term) are both rapidly approaching.

Thank you very much for considering this request.

Collecting psycopg2
  Using cached https://files.pythonhosted.org/packages/c7/ca/75236b17f1b951950ffc55d657c5aa408d3d0327a1b6c4c0f7cb16ef7e7b/psycopg2-2.8.tar.gz
    Complete output from command python setup.py egg_info:
    running egg_info
    creating pip-egg-info/psycopg2.egg-info
    writing pip-egg-info/psycopg2.egg-info/PKG-INFO
    writing dependency_links to pip-egg-info/psycopg2.egg-info/dependency_links.txt
    writing top-level names to pip-egg-info/psycopg2.egg-info/top_level.txt
    writing manifest file 'pip-egg-info/psycopg2.egg-info/SOURCES.txt'

    Error: pg_config executable not found.

    pg_config is required to build psycopg2 from source.  Please add the directory
    containing pg_config to the $PATH or specify the full executable path with the
    option:

        python setup.py build_ext --pg-config /path/to/pg_config build ...

    or with the pg_config option in 'setup.cfg'.

    If you prefer to avoid building psycopg2 from source, please install the PyPI
    'psycopg2-binary' package instead.

    For further information please check the 'doc/src/install.rst' file (also at
    <http://initd.org/psycopg/docs/install.html>).


    ----------------------------------------

Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/px/bd4dk2ss62b_px0cmk3zd5rc0000gn/T/pycharm-packaging/psycopg2/

[dvarrazzo]

You are referring to https://pypi.org/project/wrds/, which is listed on PyPI as a beta project and doesn't provide a bug tracker or a source code repository.

Taking a look at their setup.py file of that project, they list "psycopg2" as a dependency without specifying a version. You can remove "psycopg2" from their setup.py and install psycopg2-binary if you want to use that. Or fix that setup.py specifying "psycopg2<2.8". Or install libpq-dev packages, as documented, and install the wrds package naturally. Or run pip install psycopg2<2.8 from command line so that the dependency is already met.

You can fix your installation problem in many ways. There is nothing to fix in psycopg.

[planewryter]

Thank you for the response and the suggestions. Appreciated. I will attempt to apply them via PyCharm_EDU.

Because (now) several students have encountered the problem noted above, would you be so kind as to help my understanding what happened?

• Why? Because I'm going to get asked...and I can't explain what's happening.

To that end, what is the meaning of the error message: Error: pg_config executable not found, and why is the 2.8 package encountering this problem when prior versions did not?

Many thanks for all!

[ddahan]

why is the 2.8 package encountering this problem when prior versions did not?

I was thinking of the same question: is pg_config executable a new requirement that comes with 2.8?
If the answer is yes, how can you ensure that this executable is available on environments you don't have full control on, like Heroku platform for example?
(I actually did not test on Heroku yet, but I expect to have the same error than in local environement)

[ulope]

See #674 for the background

[dvarrazzo]

@ddahan the answer is no: no new requirement. The requirement has always been there, if you had installed the package from source. Before 2.8 you were, by default, installing a binary package. Such package has been deprecated for one year and on import was generating a warning that you have decide to ignore. The warning was saying that if you want the binary package you have to ask for it explicitly, it won't be installed by default.

You have ignored the warning, you have ignored the explanation message on failure on 2.8 installation, and you have opened a bug instead of trying to read and solved a widely documented issue. Explained that this is a bug tracker (which means: for our bugs, not for your bugs or other projects bugs) and that you should read docs before opening a bug, which is documented as well but of course you haven't read in first place, silly me, you have added a now deleted passive-aggressive comment.

I am pissed off enough, thank you.

You are expecting a level of hand-holding that is not available here. Furthermore, supporting platforms that are out of control, by definition, is out of control, I couldn't care less. and is your problem, or their problem, hardly mine.

Someone has kindly provided you all the pointers about the changes in 2.8. I don't think I have anything else to add.