Psycopg2: Failed building wheel and fatal error: Pyhon.h

Created on 17 Apr 2019 · 13Comments · Source: psycopg/psycopg2

I'm trying to install the latest version of psycopg2 (2.8.x) with pipenv install psycopg2, and following what to do suggested by the errors doesn't change anything. What am I missing here?

The errors I'm seeing are...

./psycopg/psycopg.h:34:10: fatal error: Python.h: No such file or directory
   #include <Python.h>
            ^~~~~~~~~~

AND

[pipenv.exceptions.InstallError]: ['Failed building wheel for psycopg2',

I am developing on Ubuntu 18.04.2 LTS with a fresh install with an up to date system.
I have python3-dev, python3-testresources, build-essentials, libpq-dev, postgresql-server-dev-all
I have python 3.7.1 installed and am using pipenv
pip is 19.0.3
wheel is 0.33.1
setuptools is 41.0.0
> Tried http://initd.org/psycopg/articles/2018/02/08/psycopg-274-released/
> Tried http://initd.org/psycopg/articles/2019/04/04/psycopg-28-released/
> Tried https://www.postgresql.org/message-id/CA%2Bmi_8bd6kJHLTGkuyHSnqcgDrJ1uHgQWvXCKQFD3tPQBUa2Bw%40mail.gmail.com
> Tried github.com/psycopg/psycopg2/issues/674

Here is the error:

(fwdev) wazd@burger:~/Projects/fwdev$ pipenv install psycopg2-binary
Installing psycopg2-binary…
Adding psycopg2-binary to Pipfile's [packages]…
✔ Installation Succeeded 
Pipfile.lock (26a31f) out of date, updating to (12885b)…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
✔ Success! 
Updated Pipfile.lock (26a31f)!
Installing dependencies from Pipfile.lock (26a31f)…
An error occurred while installing psycopg2==2.8.2 --hash=sha256:00cfecb3f3db6eb76dcc763e71777da56d12b6d61db6a2c6ccbbb0bff5421f8f --hash=sha256:076501fc24ae13b2609ba2303d88d4db79072562f0b8cc87ec1667dedff99dc1 --hash=sha256:4e2b34e4c0ddfeddf770d7df93e269700b080a4d2ec514fec668d71895f56782 --hash=sha256:5cacf21b6f813c239f100ef78a4132056f93a5940219ec25d2ef833cbeb05588 --hash=sha256:61f58e9ecb9e4dc7e30be56b562f8fc10ae3addcfcef51b588eed10a5a66100d --hash=sha256:8954ff6e47247bdd134db602fcadfc21662835bd92ce0760f3842eacfeb6e0f3 --hash=sha256:b6e8c854cdc623028e558a409b06ea2f16d13438335941c7765d0a42b5bedd33 --hash=sha256:baca21c0f7344576346e260454d0007313ccca8c170684707a63946b27a56c8f --hash=sha256:bb1735378770fb95dbe392d29e71405d45c8bdcfa064f916504833a92ab03c55 --hash=sha256:de3d3c46c1ee18f996db42d1eb44cf1565cc9e38fb1dbd9b773ff6b3fa8035d7 --hash=sha256:dee885602bb200bdcb1d30f6da6c7bb207360bc786d0a364fe1540dd14af0bab! Will try again.
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 11/11 — 00:00:09
Installing initially failed dependencies…
pipenv.exceptions.InstallError]:   File "/usr/local/lib/python3.7/dist-packages/pipenv/core.py", line 1992, in do_install
[pipenv.exceptions.InstallError]:       skip_lock=skip_lock,
[pipenv.exceptions.InstallError]:   File "/usr/local/lib/python3.7/dist-packages/pipenv/core.py", line 1253, in do_init
[pipenv.exceptions.InstallError]:       pypi_mirror=pypi_mirror,
[pipenv.exceptions.InstallError]:   File "/usr/local/lib/python3.7/dist-packages/pipenv/core.py", line 859, in do_install_dependencies
[pipenv.exceptions.InstallError]:       retry_list, procs, failed_deps_queue, requirements_dir, **install_kwargs
[pipenv.exceptions.InstallError]:   File "/usr/local/lib/python3.7/dist-packages/pipenv/core.py", line 763, in batch_install
[pipenv.exceptions.InstallError]:       _cleanup_procs(procs, not blocking, failed_deps_queue, retry=retry)
[pipenv.exceptions.InstallError]:   File "/usr/local/lib/python3.7/dist-packages/pipenv/core.py", line 681, in _cleanup_procs
[pipenv.exceptions.InstallError]:       raise exceptions.InstallError(c.dep.name, extra=err_lines)
[pipenv.exceptions.InstallError]: ['Collecting psycopg2==2.8.2 ( ... error: command 'x86_64-linux-gnu-gcc' failed with exit status 1", '    ', '    ----------------------------------------']
[pipenv.exceptions.InstallError]: ['Failed building wheel for psycopg2', 'Command "/home/wazd/.local/share/virtualenvs/fwdev-TAVasR-s/bin/python3 -u -c "import setuptools, tokenize;__file__=\'/tmp/pip-install-utxwtgbp/psycopg2/setup.py\';f=getattr(tokenize, \'open\', open)(__file__);code=f.read().replace(\'\\r\\n\', \'\\n\');f.close();exec(compile(code, __file__, \'exec\'))" install --record /tmp/pip-record-e87i6en_/install-record.txt --single-version-externally-managed --compile --install-headers /home/wazd/.local/share/virtualenvs/fwdev-TAVasR-s/include/site/python3.7/psycopg2" failed with error code 1 in /tmp/pip-install-utxwtgbp/psycopg2/']

AND from pip3 install psycopg2:

(fwdev) wazd@burger:~/Projects/fwdev$ pip3 install psycopg2
Collecting psycopg2
  Using cached https://files.pythonhosted.org/packages/23/7e/93c325482c328619870b6cd09370f6dbe1148283daca65115cd63642e60f/psycopg2-2.8.2.tar.gz
Building wheels for collected packages: psycopg2
  Building wheel for psycopg2 (setup.py) ... error
  Complete output from command /home/wazd/.local/share/virtualenvs/fwdev-TAVasR-s/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-zw76s0g_/psycopg2/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/pip-wheel-s_gi9bl6 --python-tag cp37:
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-3.7
  creating build/lib.linux-x86_64-3.7/psycopg2
  copying lib/__init__.py -> build/lib.linux-x86_64-3.7/psycopg2
  copying lib/sql.py -> build/lib.linux-x86_64-3.7/psycopg2
  copying lib/extensions.py -> build/lib.linux-x86_64-3.7/psycopg2
  copying lib/errorcodes.py -> build/lib.linux-x86_64-3.7/psycopg2
  copying lib/compat.py -> build/lib.linux-x86_64-3.7/psycopg2
  copying lib/tz.py -> build/lib.linux-x86_64-3.7/psycopg2
  copying lib/_json.py -> build/lib.linux-x86_64-3.7/psycopg2
  copying lib/_ipaddress.py -> build/lib.linux-x86_64-3.7/psycopg2
  copying lib/_range.py -> build/lib.linux-x86_64-3.7/psycopg2
  copying lib/_lru_cache.py -> build/lib.linux-x86_64-3.7/psycopg2
  copying lib/extras.py -> build/lib.linux-x86_64-3.7/psycopg2
  copying lib/pool.py -> build/lib.linux-x86_64-3.7/psycopg2
  copying lib/errors.py -> build/lib.linux-x86_64-3.7/psycopg2
  warning: build_py: byte-compiling is disabled, skipping.

  running build_ext
  building 'psycopg2._psycopg' extension
  creating build/temp.linux-x86_64-3.7
  creating build/temp.linux-x86_64-3.7/psycopg
  x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DPSYCOPG_VERSION=2.8.2 (dt dec pq3 ext lo64) -DPG_VERSION_NUM=100007 -DHAVE_LO64=1 -I/usr/include/python3.7m -I/home/wazd/.local/share/virtualenvs/fwdev-TAVasR-s/include/python3.7m -I. -I/usr/include/postgresql -I/usr/include/postgresql/10/server -c psycopg/psycopgmodule.c -o build/temp.linux-x86_64-3.7/psycopg/psycopgmodule.o -Wdeclaration-after-statement
  In file included from psycopg/psycopgmodule.c:27:0:
  ./psycopg/psycopg.h:34:10: fatal error: Python.h: No such file or directory
   #include <Python.h>
            ^~~~~~~~~~
  compilation terminated.

  It appears you are missing some prerequisite to build the package from source.

  You may install a binary package by installing 'psycopg2-binary' from PyPI.
  If you want to install psycopg2 from source, please install the packages
  required for the build and try again.

  For further information please check the 'doc/src/install.rst' file (also at
  <http://initd.org/psycopg/docs/install.html>).

  error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

  ----------------------------------------
  Failed building wheel for psycopg2
  Running setup.py clean for psycopg2
Failed to build psycopg2
Installing collected packages: psycopg2
  Running setup.py install for psycopg2 ... error
    Complete output from command /home/wazd/.local/share/virtualenvs/fwdev-TAVasR-s/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-zw76s0g_/psycopg2/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-he_4xq7_/install-record.txt --single-version-externally-managed --compile --install-headers /home/wazd/.local/share/virtualenvs/fwdev-TAVasR-s/include/site/python3.7/psycopg2:
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-3.7
    creating build/lib.linux-x86_64-3.7/psycopg2
    copying lib/__init__.py -> build/lib.linux-x86_64-3.7/psycopg2
    copying lib/sql.py -> build/lib.linux-x86_64-3.7/psycopg2
    copying lib/extensions.py -> build/lib.linux-x86_64-3.7/psycopg2
    copying lib/errorcodes.py -> build/lib.linux-x86_64-3.7/psycopg2
    copying lib/compat.py -> build/lib.linux-x86_64-3.7/psycopg2
    copying lib/tz.py -> build/lib.linux-x86_64-3.7/psycopg2
    copying lib/_json.py -> build/lib.linux-x86_64-3.7/psycopg2
    copying lib/_ipaddress.py -> build/lib.linux-x86_64-3.7/psycopg2
    copying lib/_range.py -> build/lib.linux-x86_64-3.7/psycopg2
    copying lib/_lru_cache.py -> build/lib.linux-x86_64-3.7/psycopg2
    copying lib/extras.py -> build/lib.linux-x86_64-3.7/psycopg2
    copying lib/pool.py -> build/lib.linux-x86_64-3.7/psycopg2
    copying lib/errors.py -> build/lib.linux-x86_64-3.7/psycopg2
    warning: build_py: byte-compiling is disabled, skipping.

    running build_ext
    building 'psycopg2._psycopg' extension
    creating build/temp.linux-x86_64-3.7
    creating build/temp.linux-x86_64-3.7/psycopg
    x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DPSYCOPG_VERSION=2.8.2 (dt dec pq3 ext lo64) -DPG_VERSION_NUM=100007 -DHAVE_LO64=1 -I/usr/include/python3.7m -I/home/wazd/.local/share/virtualenvs/fwdev-TAVasR-s/include/python3.7m -I. -I/usr/include/postgresql -I/usr/include/postgresql/10/server -c psycopg/psycopgmodule.c -o build/temp.linux-x86_64-3.7/psycopg/psycopgmodule.o -Wdeclaration-after-statement
    In file included from psycopg/psycopgmodule.c:27:0:
    ./psycopg/psycopg.h:34:10: fatal error: Python.h: No such file or directory
     #include <Python.h>
              ^~~~~~~~~~
    compilation terminated.

    It appears you are missing some prerequisite to build the package from source.

    You may install a binary package by installing 'psycopg2-binary' from PyPI.
    If you want to install psycopg2 from source, please install the packages
    required for the build and try again.

    For further information please check the 'doc/src/install.rst' file (also at
    <http://initd.org/psycopg/docs/install.html>).

    error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

    ----------------------------------------
Command "/home/wazd/.local/share/virtualenvs/fwdev-TAVasR-s/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-zw76s0g_/psycopg2/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-he_4xq7_/install-record.txt --single-version-externally-managed --compile --install-headers /home/wazd/.local/share/virtualenvs/fwdev-TAVasR-s/include/site/python3.7/psycopg2" failed with error code 1 in /tmp/pip-install-zw76s0g_/psycopg2/

Here is my Piplock:

(fwdev) wazd@burger:~/Projects/fwdev$ pipenv graph --reverse
click==7.0
  - Flask==1.0.2 [requires: click>=5.1]
    - Flask-SQLAlchemy==2.3.2 [requires: Flask>=0.10]
itsdangerous==1.1.0
  - Flask==1.0.2 [requires: itsdangerous>=0.24]
    - Flask-SQLAlchemy==2.3.2 [requires: Flask>=0.10]
MarkupSafe==1.1.1
  - Jinja2==2.10.1 [requires: MarkupSafe>=0.23]
    - Flask==1.0.2 [requires: Jinja2>=2.10]
      - Flask-SQLAlchemy==2.3.2 [requires: Flask>=0.10]

Here is output from pip freeze:

(fwdev) wazd@burger:~/Projects/fwdev$ pip freeze
Click==7.0
Flask==1.0.2
Flask-SQLAlchemy==2.3.2
itsdangerous==1.1.0
Jinja2==2.10.1
MarkupSafe==1.1.1
psycopg2-binary==2.8.2
SQLAlchemy==1.3.3
Werkzeug==0.15.2

One thing that does work is...
pip install psycopg2-binary

Source

wazdat

Most helpful comment

It works now. How?

sudo apt install python3.7-dev

FYI, I did a fresh install from xubuntu to ubuntu 18.04 LTS thinking that was the problem. I'm pretty sure running the install above fixed it.

What didn't work:

sudo apt install python-dev python3-dev

Cheers!

wazdat on 19 Apr 2019

👍38 🎉16 ❤1

All 13 comments

http://initd.org/psycopg/docs/faq.html#problems-compiling-and-installing-psycopg2

dvarrazzo on 18 Apr 2019

I followed that before posting the issue. Maybe you might catch something I can't? I have python-dev, python3-dev, libpq-dev installed and up to date.

pg_config works too.

wazdat on 18 Apr 2019

You either don't have python-dev, despite what you think, or it's broken. It's your system's problem, not a bug in psycopg2.

dvarrazzo on 18 Apr 2019

👎34

It works now. How?

sudo apt install python3.7-dev

FYI, I did a fresh install from xubuntu to ubuntu 18.04 LTS thinking that was the problem. I'm pretty sure running the install above fixed it.

What didn't work:

sudo apt install python-dev python3-dev

Cheers!

wazdat on 19 Apr 2019

👍38 🎉16 ❤1

Many thanks @wazdat , installing python3.7-dev also solved my issue (I had the same problems to install psycopg2 as described before on Ubuntu 16.04 LTS)

lincal2309 on 16 May 2019

👍13

Indeed, installing python3.7-dev will enable wheel building for psycopg2.

bogdanta on 19 Jul 2019

👍10 🚀1

I built a docker image that had this problem on AWS. installing python3.7-dev did not work. This did
I put the following in a shell script and ran it when the container started.

pip uninstall -y psycopg2-binary
rm -r psycopg2
rm -r psycopg2_binary-2.8.4.dist-info
pip install psycopg2-binary

jackstine on 12 Jan 2020

Thanks jsckstine. I was facing issue installing psycopg2 on EC2.
pip install psycopg2 was failing.
pip install psycopg2-binary did work. Thanks again.

bharat-rb-panchal on 9 Feb 2020

For some that are having the issue because of missing libraries - make sure you have the following libraries installed:

libpq-dev
python3-dev
gcc

Example:
sudo apt install libpq-dev python3-dev gcc

belthaZornv on 5 Mar 2020

👍23 ❤3 🎉1

@belthaZornv This helped me to fix this building, thank you so much.

omonimus1 on 13 Apr 2020

Hi all.
I see this error and this didn't fix with install libpq-dev python3-dev gcc.

aliir74 on 24 Jun 2020

thanks @belthaZornv your solution worked for me after many hours of searching

shols232 on 22 Jul 2020

Seriously? Hours?

From the psycopg2 install pages: https://www.psycopg.org/docs/install.html

To install it from sources you will need:

[...] The Python header files. They are usually installed in a package such as python-dev or python3-dev. A message such as error: Python.h: No such file or directory is an indication that the Python headers are missing.

From the psycopg2 FAQ: https://www.psycopg.org/docs/faq.html#problems-compiling-and-installing-psycopg2

Q: I can’t compile psycopg2: the compiler says error: Python.h: No such file or directory. What am I missing?

A: You need to install a Python development package: it is usually called python-dev or python3-dev according to your Python version.

The first hits on google are:

A Stackoverflow page whose accepted answer is: sudo apt-get install python3-dev
This ticket
The install page of psycopg2

I am curious: what does people look for, so that this would be easy to find? Maybe we can fix something?

dvarrazzo on 22 Jul 2020

Was this page helpful?

0 / 5 - 0 ratings

Related issues

Incorrect psycopg2.tz.FixedOffsetTimezone after pickling datetimes with different timezones

psycoteam · 4Comments

execute_batch(), execute_values() give wrong rowcount

yamagoya · 5Comments

Psycopg2-Binary not on PyPi for macOS and Windows (Python 3.7)

AlexanderNeilson · 3Comments

Error install psycopg2 on Mac

vitorpontual · 5Comments

RealDictRow.items() behave inconsistently on duplicate row names

ProgVal · 6Comments