Transformers: pip install error: "regex_3/_regex.c:48:10: fatal error: Python.h: No such file or directory"

[seyuboglu]

When pip installing pytorch-pretrained-bert on ubuntu and getting the following error:

    regex_3/_regex.c:48:10: fatal error: Python.h: No such file or directory
     #include "Python.h"
              ^~~~~~~~~~
    compilation terminated.
    error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
    ----------------------------------------
ERROR: Command errored out with exit status 1: /home/.../env/bin/python3.7 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-pogvvuk9/regex/setup.py'"'"'; __file__='"'"'/tmp/pip-install-pogvvuk9/regex/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-19j41i0r/install-record.txt --single-version-externally-managed --compile --install-headers /home/.../env/include/site/python3.7/regex Check the logs for full command output.

My python version is Python 3.7.4 (default, Jul 9 2019, 15:11:16) [GCC 7.4.0] on linux.

I'm able to pip install other packages without problems.
Has anyone run into a similar issue?

[stefan-it]

Hi @seyuboglu I think you need to install the Python Dev package on your distribution. For Ubuntu >= 18.04 this should be possible with apt install python3.7-dev :)

[seyuboglu]

Thank you @stefan-it! That did the trick. Any idea why this is necessary to install pytorch-transformers in particular?

[antleypk]

@stefan-it do you know how to make this happen on on amazon linux2, I have tried a bunch of things am am getting same error. I think i already installed a python3-development package with yum. but when i tried python3.7-dev it said no package located.

[stefan-it]

@antleypk Could you try to use yum install python3-devel instead?

[antleypk]

Thanks @stefan-it

I solved it last week and should have updated thread.

To anyone else that may come here; this solution is for "amazon linux2"

my current setup.sh scripts looks like this:

 sudo yum install python3.x86_64 -y
 sudo yum install python3-devel.x86_64 -y

Since getting adding the bottom line I have been able to install every package that I tried to install.

I built the answer based on this solution:
https://stackoverflow.com/questions/43047284/how-to-install-python3-devel-on-red-hat-7

and found this solution based on this original post.