Pytest: pipenv => ImportError: No module named 'pathlib2'

[kevindawson]

Q is this a pytest or pipenv issue?

• As pathlib — Object-oriented filesystem paths
  
  
  
  • New in version 3.4.
  
  

python 3.4.9 Fail
python 3.5.6 Fail
python 3.6.6 Pass
python 3.7.0 Pass

3.4.9

(pylsusb-MXKIMXpf) kevin@dual:~/git/pylsusb$ python -m pytest
Traceback (most recent call last):
  File "/home/kevin/.pyenv/versions/3.4.9/lib/python3.4/runpy.py", line 170, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/kevin/.pyenv/versions/3.4.9/lib/python3.4/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/kevin/.local/share/virtualenvs/pylsusb-MXKIMXpf/lib/python3.4/site-packages/pytest.py", line 9, in <module>
    from _pytest.config import main, UsageError, cmdline, hookspec, hookimpl
  File "/home/kevin/.local/share/virtualenvs/pylsusb-MXKIMXpf/lib/python3.4/site-packages/_pytest/config/__init__.py", line 18, in <module>
    import _pytest._code
  File "/home/kevin/.local/share/virtualenvs/pylsusb-MXKIMXpf/lib/python3.4/site-packages/_pytest/_code/__init__.py", line 3, in <module>
    from .code import Code  # noqa
  File "/home/kevin/.local/share/virtualenvs/pylsusb-MXKIMXpf/lib/python3.4/site-packages/_pytest/_code/code.py", line 11, in <module>
    from _pytest.compat import _PY2, _PY3, PY35, safe_str
  File "/home/kevin/.local/share/virtualenvs/pylsusb-MXKIMXpf/lib/python3.4/site-packages/_pytest/compat.py", line 47, in <module>
    from pathlib2 import Path
ImportError: No module named 'pathlib2'

3.5.6

(pylsusb-MXKIMXpf) kevin@dual:~/git/pylsusb$ python -m pytest
Traceback (most recent call last):
  File "/home/kevin/.pyenv/versions/3.5.6/lib/python3.5/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/kevin/.pyenv/versions/3.5.6/lib/python3.5/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/kevin/.local/share/virtualenvs/pylsusb-MXKIMXpf/lib/python3.5/site-packages/pytest.py", line 9, in <module>
    from _pytest.config import main, UsageError, cmdline, hookspec, hookimpl
  File "/home/kevin/.local/share/virtualenvs/pylsusb-MXKIMXpf/lib/python3.5/site-packages/_pytest/config/__init__.py", line 18, in <module>
    import _pytest._code
  File "/home/kevin/.local/share/virtualenvs/pylsusb-MXKIMXpf/lib/python3.5/site-packages/_pytest/_code/__init__.py", line 3, in <module>
    from .code import Code  # noqa
  File "/home/kevin/.local/share/virtualenvs/pylsusb-MXKIMXpf/lib/python3.5/site-packages/_pytest/_code/code.py", line 11, in <module>
    from _pytest.compat import _PY2, _PY3, PY35, safe_str
  File "/home/kevin/.local/share/virtualenvs/pylsusb-MXKIMXpf/lib/python3.5/site-packages/_pytest/compat.py", line 47, in <module>
    from pathlib2 import Path
ImportError: No module named 'pathlib2'

[kevindawson]

Closing - due to a pipenv shell issue

• exit
• pipenv shell

working now!

the poor old dear got confused just like me;)

[Overdrivr]

I can reproduce this issue in a CI environment, by starting from base image docker:3.5-slim and running
a simple

pip install pipenv
pipenv install pytest
pipenv run pytest

Traceback (most recent call last):
  File "/root/.local/share/virtualenvs/analog-T5TIkbnv/bin/pytest", line 7, in <module>
    from pytest import main
  File "/root/.local/share/virtualenvs/analog-T5TIkbnv/lib/python3.5/site-packages/pytest.py", line 7, in <module>
    from _pytest.assertion import register_assert_rewrite
  File "/root/.local/share/virtualenvs/analog-T5TIkbnv/lib/python3.5/site-packages/_pytest/assertion/__init__.py", line 12, in <module>
    from _pytest.assertion import rewrite
  File "/root/.local/share/virtualenvs/analog-T5TIkbnv/lib/python3.5/site-packages/_pytest/assertion/rewrite.py", line 24, in <module>
    from _pytest.pathlib import fnmatch_ex
  File "/root/.local/share/virtualenvs/analog-T5TIkbnv/lib/python3.5/site-packages/_pytest/pathlib.py", line 26, in <module>
    from pathlib2 import Path, PurePath
ImportError: No module named 'pathlib2'

This is solved by running pipenv install pathlib2. My theory is that in stripped-down python versions (such as the one in docker image python:3.5-slim), pathlib2 is not present by default and therefore it looks like should be added as an install dependency by pytest.

@asottile or any repo-member, ok to reopen ?

[blueyed]

@Overdrivr
Try to reproduce it without pipenv.
The dep is there:
https://github.com/pytest-dev/pytest/blob/a4c426b1a891a22ae1b63d0a0fa2dcdf690e69db/setup.py#L16

[asottile]

@Overdrivr I still can't reproduce even with docker (I assume you meant python:3.5-slim above, I've substituted that tag):

$ docker run --rm -ti python:3.5-slim bash -euxc 'mkdir /wat && cd /wat && pip install pipenv && pipenv install pytest && pipenv run pytest'
+ mkdir /wat
+ cd /wat
+ pip install pipenv
Collecting pipenv
  Downloading https://files.pythonhosted.org/packages/13/b4/3ffa55f77161cff9a5220f162670f7c5eb00df52e00939e203f601b0f579/pipenv-2018.11.26-py3-none-any.whl (5.2MB)
    100% |████████████████████████████████| 5.2MB 7.8MB/s 
Collecting virtualenv (from pipenv)
  Downloading https://files.pythonhosted.org/packages/6a/d1/e0d142ce7b8a5c76adbfad01d853bca84c7c0240e35577498e20bc2ade7d/virtualenv-16.2.0-py2.py3-none-any.whl (1.9MB)
    100% |████████████████████████████████| 1.9MB 15.7MB/s 
Requirement already satisfied: setuptools>=36.2.1 in /usr/local/lib/python3.5/site-packages (from pipenv) (40.6.3)
Collecting certifi (from pipenv)
  Downloading https://files.pythonhosted.org/packages/9f/e0/accfc1b56b57e9750eba272e24c4dddeac86852c2bebd1236674d7887e8a/certifi-2018.11.29-py2.py3-none-any.whl (154kB)
    100% |████████████████████████████████| 163kB 15.1MB/s 
Collecting virtualenv-clone>=0.2.5 (from pipenv)
  Downloading https://files.pythonhosted.org/packages/16/9d/6419a4f0fe4350db7fdc01e9d22e949779b6f2d2650e4884aa8aededc5ae/virtualenv_clone-0.4.0-py2.py3-none-any.whl
Requirement already satisfied: pip>=9.0.1 in /usr/local/lib/python3.5/site-packages (from pipenv) (18.1)
Installing collected packages: virtualenv, certifi, virtualenv-clone, pipenv
Successfully installed certifi-2018.11.29 pipenv-2018.11.26 virtualenv-16.2.0 virtualenv-clone-0.4.0
+ pipenv install pytest
Creating a virtualenv for this project…
Pipfile: /wat/Pipfile
Using /usr/local/bin/python (3.5.6) to create virtualenv…
⠹ Creating virtual environment...Already using interpreter /usr/local/bin/python
Using base prefix '/usr/local'
New python executable in /root/.local/share/virtualenvs/wat-cJjBwrMR/bin/python
Installing setuptools, pip, wheel...
done.

✔ Successfully created virtual environment! 
Virtualenv location: /root/.local/share/virtualenvs/wat-cJjBwrMR
Creating a Pipfile for this project…
Installing pytest…
Adding pytest to Pipfile's [packages]…
✔ Installation Succeeded 
Pipfile.lock not found, creating…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
✔ Success! 
Updated Pipfile.lock (2cdf6f)!
Installing dependencies from Pipfile.lock (2cdf6f)…
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 8/8 — 00:00:01
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.
+ pipenv run pytest
============================= test session starts ==============================
platform linux -- Python 3.5.6, pytest-4.1.0, py-1.7.0, pluggy-0.8.0
rootdir: /wat, inifile:
collected 0 items                                                              

========================= no tests ran in 0.00 seconds =========================

[Overdrivr]

Can you add a test and see if it reproduces the issue ? I believe issue was happening during test collection

[asottile]

can you provide an actual reproduction? I have a day job :laughing:

[Overdrivr]

Can't reproduce with docker, only happened in my gitlab CI env. Following command adds a test but no error to report

docker run --rm -ti python:3.5-slim bash -euxc "mkdir /wat && cd /wat && echo -e 'def test_foo():\n\t
pass' >> test_foo.py && pip install pipenv && pipenv install pytest --dev && pipenv run pytest"

So I don't know, it was probably due to something else but I don't have time to investigate further.

[FlorianLudwig]

I can reproduce this on my machine. Running with pipenv 2018.11.26 systemwide install:

# enter empty directory
mkdir /tmp/foo
cd /tmp/foo

# create python 2 pipenv
pipenv --python 2
pipenv install pip  # does not matter which package, any package will do.

python 3 virtualenv does work fine.

[blueyed]

@FlorianLudwig
What is the error then? How is pytest involved there?

I do not recommend using pipenv myself.
(trying your commands made me file https://github.com/pypa/pipenv/issues/3625 though)