Pipenv: Can't bootstrap development environment

[JacobHayes]

This is as much an issue report as a potential question/documentation request if I'm just setting up the project wrong for contributing. I'm trying to fix pypa/pipenv#2504, but unable to install the development environment after cloning the repo. I didn't see anything in particular for how to set up the local development environment, so assumed it just required pipenv install --dev.

Issue description

Cloning the repo and trying to pipenv install --dev fails because it tries to use a mix of the repo code and the global pipenv code. For example, version 2018.7.1 has pipenv.pew, but that's removed on master, so (interestingly) passively fails to import. It continues on to hard fail later due to location being None in pipenv.core.which when passed to os.path.join.

Also of note: just running pipenv install completes without any modifications, but doesn't actually create the virtualenv even thought it says it is (there are no non-dev packages). I'm not sure if skipping an empty virtualenv is desired behavior or not.

If I rename the pipenv directory to src/pipenv and update setup.py (probably incomplete setup.py.patch), it is able to complete the installation.

Expected result

The global pipenv code to be used to install the repo, and then after that the local development version (as specified in the Pipfile dev section).

Actual result

[I] ~/s/g/p/pipenv (master|✔) $ which pipenv
/usr/local/bin/pipenv
[I] ~/s/g/p/pipenv (master|✔) $ pipenv --version
pipenv, version 2018.7.1
[I] ~/s/g/p/pipenv (master|✔) $ pipenv install --dev --verbose
Creating a virtualenv for this project...
Pipfile: /Users/jacobhayes/src/github.com/pypa/pipenv/Pipfile
Using /usr/local/bin/python3 (3.6.5) to create virtualenv...
⠋/usr/local/bin/python3: No module named pipenv.pew

Virtualenv location:
Installing -e ....
⠙Installing '-e .'
⠸/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/setuptools/dist.py:517: UserWarning: Module attr was already imported from /usr/local/lib/python3.6/site-packages/pipenv/vendor/attr/__init__.py, but /Users/jacobhayes/src/github.com/pypa/pipenv/.eggs/attrs-18.1.0-py3.6.egg is being added to sys.path
  pkg_resources.working_set.add(dist, replace=True)
Traceback (most recent call last):
  File "/usr/local/bin/pipenv", line 11, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pipenv/cli.py", line 435, in install
    selective_upgrade=selective_upgrade,
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pipenv/core.py", line 1933, in do_install
    pypi_mirror=pypi_mirror,
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pipenv/core.py", line 1429, in pip_install
    quoted_pip = which_pip(allow_global=allow_global)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pipenv/core.py", line 1488, in which_pip
    return which("pip")
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pipenv/core.py", line 110, in which
    p = os.path.join(location, "bin", command)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/posixpath.py", line 80, in join
    a = os.fspath(a)
TypeError: expected str, bytes or os.PathLike object, not NoneType

Steps to replicate

pip3 install pipenv==v2018.07.01
git clone [email protected]:pypa/pipenv.git
cd pipenv
pipenv install --dev

---

Here is the support output when run outside of the repo/virtualenv.

$ pipenv --support

Pipenv version: '2018.7.1'

Pipenv location: '/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pipenv'

Python location: '/usr/local/bin/python3'

Other Python installations in PATH:

• 2.7: /usr/local/bin/python2.7
• 2.7: /usr/local/bin/python2.7
• 2.7: /usr/bin/python2.7
• 3.6: /usr/local/bin/python3.6m

• 3.6: /usr/local/bin/python3.6

• 2.7.15: /usr/local/bin/python

• 2.7.10: /usr/bin/python
• 2.7.15: /usr/local/bin/python2
• 3.6.5: /usr/local/bin/python3

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.6.5',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '17.6.0',
 'platform_system': 'Darwin',
 'platform_version': 'Darwin Kernel Version 17.6.0: Tue May  8 15:22:16 PDT '
                     '2018; root:xnu-4570.61.1~1/RELEASE_X86_64',
 'python_full_version': '3.6.5',
 'python_version': '3.6',
 'sys_platform': 'darwin'}

System environment variables:

• Apple_PubSub_Socket_Render
• COLORFGBG
• COLORTERM
• COPYFILE_DISABLE
• EDITOR
• GOPATH
• HOME
• ITERM_PROFILE
• ITERM_SESSION_ID
• LANG
• LOGNAME
• PAGER
• PATH
• PWD
• SECURITYSESSIONID
• SHELL
• SHLVL
• SSH_AUTH_SOCK
• TERM
• TERM_PROGRAM
• TERM_PROGRAM_VERSION
• TERM_SESSION_ID
• TMPDIR
• TMUX
• TMUX_PANE
• TMUX_PLUGIN_MANAGER_PATH
• USER
• XPC_FLAGS
• XPC_SERVICE_NAME
• __CF_USER_TEXT_ENCODING
• PYTHONDONTWRITEBYTECODE
• PIP_PYTHON_PATH

Pipenv–specific environment variables:

Debug–specific environment variables:

• PATH: /Users/jacobhayes/bin:/Users/jacobhayes/bin:/usr/local/Caskroom/google-cloud-sdk/latest/google-cloud-sdk/bin:/Users/jacobhayes/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin
• SHELL: /usr/local/bin/fish
• EDITOR: vim
• LANG: en_US.UTF-8
• PWD: /Users/jacobhayes/src/github.com/pypa

---

[JacobHayes]

Side note: It seems which was updated recently to be more defensive, but I think its guard check on ln109 is backwards (ex: both location = None and location = '' would skip the error) and should be under the if not allow_global block.

[uranusjr]

It seems you are stuck between two versions of Pipenv. We removed pipenv.pew after 2018.7.1. My guess your pipenv command got confused choosing which pipenv library to load. It should use the global one, but is stuck with the one in your current directory. It might be a solution to use /usr/local/bin/python3 -m pipenv instead.

Another solution is to cd into one of the directories when you initialise the project. This would also avoid Python finding pipenv in the current directory.

@techalchemy I’m thinking maybe it’d be a good idea to change the project structure to wrap pipenv inside src, to prevent this from happening. Any thoughts?

[JacobHayes]

Yes, that's why I was mentioning pew specifically. I'm guessing there are cases where it's desired to allow the imports to fail passively?

Using python3 -m pipenv fixed things, thanks for the suggestion.

I'm happy to submit the src dir move and the setup.py patch I included above as a PR if that's the path the project decides to take.

[techalchemy]

I’m in favor but I am fairly confident Kenneth was opposed. We can talk and work it out

[techalchemy]

FWIW I hate all code that relates to location checks.

[javabrett]

I get this also when trying to use pipenv within the _pipenv_ project itself. python3 -m pipenv install is an effective workaround.

 pipenv install

Creating a virtualenv for this project...
Pipfile: /Users/bsrandal/git/pipenv/Pipfile
Using /Library/Frameworks/Python.framework/Versions/3.6/bin/python3 (3.6.3) to create virtualenv...
⠋/Library/Frameworks/Python.framework/Versions/3.6/bin/python3: No module named pipenv.pew

[techalchemy]

While I do still agree that a src based structure is desirable (we haven't implemented this yet), this specific problem shouldn't still be an issue in the latest versions. Is it?

[livyfu]

While I do still agree that a src based structure is desirable (we haven't implemented this yet), this specific problem shouldn't still be an issue in the latest versions. Is it?

No, it seems not an issue anymore. At least not for me. In the repo root folder:

$ which pipenv
/usr/local/bin/pipenv
$ pipenv install --dev
Creating a virtualenv for this project…
Pipfile: /Users/liweifu/Developer/pipenv/Pipfile
Using /usr/local/Cellar/pipenv/2018.11.26_2/libexec/bin/python3.7 (3.7.3) to create virtualenv…
...
...
...
$ pipenv shell
Launching subshell in virtual environment…
 . /Users/liweifu/.local/share/virtualenvs/pipenv-qffrfQp0/bin/activate
$ which pipenv
/Users/liweifu/.local/share/virtualenvs/pipenv-qffrfQp0/bin/pipenv

My system global pipenv was installed by brew.