Pandas: Build Error, pip is trying to build a wheel using "pep517"

[ShaharNaveh]

I get an error while following the contributing guide on creating a python eviroment using pip.
when I run the command:

python -m pip install -e . --no-build-isolation

I get an error that says:

error: invalid command 'bdist_wheel'

To repreduce this issue:

1. Install Arch linux, September release(and upgrade to the latest version)
2. Install :
   
   
   
   1. python3-venv
   
   
   2. python-pip
   
   
3. Follow the contributing guide
   
   
   
   1. git clone https://github.com/your-user-name/pandas.git pandas-yourname
   
   
   2. cd pandas-yourname
   
   
   3. git remote add upstream https://github.com/pandas-dev/pandas.git
   
   
   4. python3 -m venv ~/virtualenvs/pandas-dev
   
   
   5. . ~/virtualenvs/pandas-dev/bin/activate
   
   
   6. python -m pip install -r requirements-dev.txt
   
   
   7. python setup.py build_ext --inplace -j 4
   
   
   8. python -m pip install -e . --no-build-isolation
   
   

on line 8 I get an error.

P.S
This is my second time opening an issue(ever), please let me know if what else I should mention/include/change in order to be more productive.

[jbrockmendel]

have you tried pip3 install wheel?

[ShaharNaveh]

Thank you, very much.

[jorisvandenbossche]

@MomIsBestFriend What version of pip do you have?
I am a bit surprised that doing an editable install would try to build a wheel

[ShaharNaveh]

the output of:

source my/venv/path/bin/activate
pip3 --version

Gives me:
pip 19.2.3 from my/venv/path/.../python3.7/.../pip (python3.7)

[jorisvandenbossche]

Can you show the verbose output of pip? Something like python -m pip install -e . --no-build-isolation -vv

[ShaharNaveh]

The output was too large for one screenshot, so I took two(hope it helps.
Also I have included the output below in text, In case something is wrong with the pictures.

(venv_pandas) [bummy@null pandas-MomIsBestFriend]$ python -m pip install -e . --no-build-isolation -vv
Created temporary directory: /tmp/pip-ephem-wheel-cache-kv25t34x
Created temporary directory: /tmp/pip-req-tracker-k_3e9zc5
Created requirements tracker '/tmp/pip-req-tracker-k_3e9zc5'
Created temporary directory: /tmp/pip-install-3tan77je
Obtaining file:///home/bummy/Desktop/Github_projects/pandas/pandas-MomIsBestFriend
  Added file:///home/bummy/Desktop/Github_projects/pandas/pandas-MomIsBestFriend to build tracker '/tmp/pip-req-tracker-k_3e9zc5'
    Running command /home/bummy/Desktop/Github_projects/pandas/venv_pandas/bin/python /home/bummy/Desktop/Github_projects/pandas/venv_pandas/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/tmpwmxkcc2h
    running dist_info
    writing /home/bummy/Desktop/Github_projects/pandas/pandas-MomIsBestFriend/pip-wheel-metadata/pandas.egg-info/PKG-INFO
    writing dependency_links to /home/bummy/Desktop/Github_projects/pandas/pandas-MomIsBestFriend/pip-wheel-metadata/pandas.egg-info/dependency_links.txt
    writing entry points to /home/bummy/Desktop/Github_projects/pandas/pandas-MomIsBestFriend/pip-wheel-metadata/pandas.egg-info/entry_points.txt
    writing requirements to /home/bummy/Desktop/Github_projects/pandas/pandas-MomIsBestFriend/pip-wheel-metadata/pandas.egg-info/requires.txt
    writing top-level names to /home/bummy/Desktop/Github_projects/pandas/pandas-MomIsBestFriend/pip-wheel-metadata/pandas.egg-info/top_level.txt
    reading manifest file '/home/bummy/Desktop/Github_projects/pandas/pandas-MomIsBestFriend/pip-wheel-metadata/pandas.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    no previously-included directories found matching 'doc/build'
    warning: no previously-included files matching '*.pyd' found anywhere in distribution
    warning: no previously-included files matching '*~' found anywhere in distribution
    warning: no previously-included files matching '.DS_Store' found anywhere in distribution
    warning: no previously-included files matching '.git*' found anywhere in distribution
    warning: no previously-included files matching '#*' found anywhere in distribution
    writing manifest file '/home/bummy/Desktop/Github_projects/pandas/pandas-MomIsBestFriend/pip-wheel-metadata/pandas.egg-info/SOURCES.txt'
    creating '/home/bummy/Desktop/Github_projects/pandas/pandas-MomIsBestFriend/pip-wheel-metadata/pandas.dist-info'
    error: invalid command 'bdist_wheel'
    Preparing wheel metadata ... error
Cleaning up...
Removed file:///home/bummy/Desktop/Github_projects/pandas/pandas-MomIsBestFriend from build tracker '/tmp/pip-req-tracker-k_3e9zc5'
Removed build tracker '/tmp/pip-req-tracker-k_3e9zc5'
ERROR: Command errored out with exit status 1: /home/bummy/Desktop/Github_projects/pandas/venv_pandas/bin/python /home/bummy/Desktop/Github_projects/pandas/venv_pandas/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/tmpwmxkcc2h Check the logs for full command output.
Exception information:
Traceback (most recent call last):
  File "/home/bummy/Desktop/Github_projects/pandas/venv_pandas/lib/python3.7/site-packages/pip/_internal/cli/base_command.py", line 188, in main
    status = self.run(options, args)
  File "/home/bummy/Desktop/Github_projects/pandas/venv_pandas/lib/python3.7/site-packages/pip/_internal/commands/install.py", line 345, in run
    resolver.resolve(requirement_set)
  File "/home/bummy/Desktop/Github_projects/pandas/venv_pandas/lib/python3.7/site-packages/pip/_internal/legacy_resolve.py", line 196, in resolve
    self._resolve_one(requirement_set, req)
  File "/home/bummy/Desktop/Github_projects/pandas/venv_pandas/lib/python3.7/site-packages/pip/_internal/legacy_resolve.py", line 359, in _resolve_one
    abstract_dist = self._get_abstract_dist_for(req_to_install)
  File "/home/bummy/Desktop/Github_projects/pandas/venv_pandas/lib/python3.7/site-packages/pip/_internal/legacy_resolve.py", line 291, in _get_abstract_dist_for
    req, self.require_hashes, self.use_user_site, self.finder,
  File "/home/bummy/Desktop/Github_projects/pandas/venv_pandas/lib/python3.7/site-packages/pip/_internal/operations/prepare.py", line 250, in prepare_editable_requirement
    finder, self.build_isolation,
  File "/home/bummy/Desktop/Github_projects/pandas/venv_pandas/lib/python3.7/site-packages/pip/_internal/distributions/source.py", line 79, in prepare_distribution_metadata
    self.req.prepare_metadata()
  File "/home/bummy/Desktop/Github_projects/pandas/venv_pandas/lib/python3.7/site-packages/pip/_internal/req/req_install.py", line 549, in prepare_metadata
    self.prepare_pep517_metadata()
  File "/home/bummy/Desktop/Github_projects/pandas/venv_pandas/lib/python3.7/site-packages/pip/_internal/req/req_install.py", line 594, in prepare_pep517_metadata
    metadata_dir
  File "/home/bummy/Desktop/Github_projects/pandas/venv_pandas/lib/python3.7/site-packages/pip/_vendor/pep517/wrappers.py", line 86, in prepare_metadata_for_build_wheel
    'config_settings': config_settings,
  File "/home/bummy/Desktop/Github_projects/pandas/venv_pandas/lib/python3.7/site-packages/pip/_vendor/pep517/wrappers.py", line 155, in _call_hook
    extra_environ={'PEP517_BUILD_BACKEND': build_backend}
  File "/home/bummy/Desktop/Github_projects/pandas/venv_pandas/lib/python3.7/site-packages/pip/_internal/req/req_install.py", line 532, in runner
    spinner=spinner
  File "/home/bummy/Desktop/Github_projects/pandas/venv_pandas/lib/python3.7/site-packages/pip/_internal/utils/misc.py", line 931, in call_subprocess
    raise InstallationError(exc_msg)
pip._internal.exceptions.InstallationError: Command errored out with exit status 1: /home/bummy/Desktop/Github_projects/pandas/venv_pandas/bin/python /home/bummy/Desktop/Github_projects/pandas/venv_pandas/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/tmpwmxkcc2h Check the logs for full command output.
(venv_pandas) [bummy@null pandas-MomIsBestFriend]$ 

[jorisvandenbossche]

Can you try python -m pip install -e . --no-build-isolation --no-use-pep517 -vv ?

For some reason it still tries to use the new PEP517 build process, which it should not do for editable installs.

[ShaharNaveh]

@jorisvandenbossche it Worked!

Command output:

(venv_pandas) [bummy@null pandas-MomIsBestFriend]$ python -m pip install -e . --no-build-isolation --no-use-pep517 -vv
Created temporary directory: /tmp/pip-ephem-wheel-cache-mh92k48d
Created temporary directory: /tmp/pip-req-tracker-d9adfqsw
Created requirements tracker '/tmp/pip-req-tracker-d9adfqsw'
Created temporary directory: /tmp/pip-install-opfwm5mp
Obtaining file:///home/bummy/Desktop/Github_projects/pandas/pandas-MomIsBestFriend
  Added file:///home/bummy/Desktop/Github_projects/pandas/pandas-MomIsBestFriend to build tracker '/tmp/pip-req-tracker-d9adfqsw'
    Running setup.py (path:/home/bummy/Desktop/Github_projects/pandas/pandas-MomIsBestFriend/setup.py) egg_info for package from file:///home/bummy/Desktop/Github_projects/pandas/pandas-MomIsBestFriend
    Running command python setup.py egg_info
    running egg_info
    writing pandas.egg-info/PKG-INFO
    writing dependency_links to pandas.egg-info/dependency_links.txt
    writing entry points to pandas.egg-info/entry_points.txt
    writing requirements to pandas.egg-info/requires.txt
    writing top-level names to pandas.egg-info/top_level.txt
    reading manifest file 'pandas.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    no previously-included directories found matching 'doc/build'
    warning: no previously-included files matching '*.pyd' found anywhere in distribution
    warning: no previously-included files matching '*~' found anywhere in distribution
    warning: no previously-included files matching '.DS_Store' found anywhere in distribution
    warning: no previously-included files matching '.git*' found anywhere in distribution
    warning: no previously-included files matching '#*' found anywhere in distribution
    writing manifest file 'pandas.egg-info/SOURCES.txt'
  Source in /home/bummy/Desktop/Github_projects/pandas/pandas-MomIsBestFriend has version 0.26.0.dev0+417.gd5fa16bcd.dirty, which satisfies requirement pandas==0.26.0.dev0+417.gd5fa16bcd.dirty from file:///home/bummy/Desktop/Github_projects/pandas/pandas-MomIsBestFriend
  Removed pandas==0.26.0.dev0+417.gd5fa16bcd.dirty from file:///home/bummy/Desktop/Github_projects/pandas/pandas-MomIsBestFriend from build tracker '/tmp/pip-req-tracker-d9adfqsw'
Requirement already satisfied: python-dateutil>=2.6.1 in /home/bummy/Desktop/Github_projects/pandas/venv_pandas/lib/python3.7/site-packages (from pandas==0.26.0.dev0+417.gd5fa16bcd.dirty) (2.8.0)
Requirement already satisfied: pytz>=2017.2 in /home/bummy/Desktop/Github_projects/pandas/venv_pandas/lib/python3.7/site-packages (from pandas==0.26.0.dev0+417.gd5fa16bcd.dirty) (2019.2)
Requirement already satisfied: numpy>=1.13.3 in /home/bummy/Desktop/Github_projects/pandas/venv_pandas/lib/python3.7/site-packages (from pandas==0.26.0.dev0+417.gd5fa16bcd.dirty) (1.17.2)
Requirement already satisfied: six>=1.5 in /home/bummy/Desktop/Github_projects/pandas/venv_pandas/lib/python3.7/site-packages (from python-dateutil>=2.6.1->pandas==0.26.0.dev0+417.gd5fa16bcd.dirty) (1.12.0)
Installing collected packages: pandas
  Found existing installation: pandas 0.26.0.dev0+417.gd5fa16bcd.dirty
    Not uninstalling pandas at /home/bummy/Desktop/Github_projects/pandas/pandas-MomIsBestFriend, outside environment /home/bummy/Desktop/Github_projects/pandas/venv_pandas
    Can't uninstall 'pandas'. No files were found to uninstall.
  Running setup.py develop for pandas
    Running command /home/bummy/Desktop/Github_projects/pandas/venv_pandas/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/bummy/Desktop/Github_projects/pandas/pandas-MomIsBestFriend/setup.py'"'"'; __file__='"'"'/home/bummy/Desktop/Github_projects/pandas/pandas-MomIsBestFriend/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps
    running develop
    running egg_info
    writing pandas.egg-info/PKG-INFO
    writing dependency_links to pandas.egg-info/dependency_links.txt
    writing entry points to pandas.egg-info/entry_points.txt
    writing requirements to pandas.egg-info/requires.txt
    writing top-level names to pandas.egg-info/top_level.txt
    reading manifest file 'pandas.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    no previously-included directories found matching 'doc/build'
    warning: no previously-included files matching '*.pyd' found anywhere in distribution
    warning: no previously-included files matching '*~' found anywhere in distribution
    warning: no previously-included files matching '.DS_Store' found anywhere in distribution
    warning: no previously-included files matching '.git*' found anywhere in distribution
    warning: no previously-included files matching '#*' found anywhere in distribution
    writing manifest file 'pandas.egg-info/SOURCES.txt'
    running build_ext
    Creating /home/bummy/Desktop/Github_projects/pandas/venv_pandas/lib/python3.7/site-packages/pandas.egg-link (link to .)
    Adding pandas 0.26.0.dev0+417.gd5fa16bcd.dirty to easy-install.pth file

    Installed /home/bummy/Desktop/Github_projects/pandas/pandas-MomIsBestFriend
Successfully installed pandas
Cleaning up...
Removed build tracker '/tmp/pip-req-tracker-d9adfqsw'
(venv_pandas) [bummy@null pandas-MomIsBestFriend]$ 

Proof:

(venv_pandas) [bummy@null pandas-MomIsBestFriend]$ python3
Python 3.7.4 (default, Jul 16 2019, 07:12:58) 
[GCC 9.1.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pandas as pd
>>> print(pd)
<module 'pandas' from '/home/bummy/Desktop/Github_projects/pandas/pandas-MomIsBestFriend/pandas/__init__.py'>
>>> 

[gbaychev]

Hello, yesterday when trying to build pandas under Windows I got the same error:

Is there any problem if we simply include the wheel package in requirements-dev.txt? I assume that many of you have the wheel package preinstalled in some form, but if someone tries to build pandas on a clean machine, he is going to have a bad time.

[jorisvandenbossche]

The thing that is should not build a wheel for an inplace (editable) build (so we shouldn't need to add 'wheel' to the requirements). For some reason it is trying to use PEP517 build interface.
As mentioned above, you can do python -m pip install -e . --no-build-isolation --no-use-pep517 to fix this (but again, I don't fully understand why in your case this extra --no-use-pep517 flag is needed)

[gbaychev]

Ah, I see. Sorry @jorisvandenbossche , I think I did not read very carefully what you've written before, now I understand, my bad.

[ShaharNaveh]

@gbaychev ATM the command python -m pip install -e . --no-build-isolation still doesn't work on my computer.

Can we add the argument --no-use-pep517 to the line python -m pip install -e . in the subsection of "Unix/Mac OS" under "creating virtual enviroment using pip" ?

P.S
Every time I'm building this package from source(I try it once in a while), I find myself look to this issue's comments and look for jorisvandenbossche's comment about the extra argument.

It works every time.

[TomAugspurger]

Yeah, we can add that flag to our dev docs.