Pipenv: Fatal dependency resolution issue with editable dependencies

[jkp]

Checkout the code here and run test.sh with the latest version of pipenv to repro. Dependencies fail to resolve. If you revert to 2018.5.18 it works.

Additional info: moving the same sub-dependencies into the Pipfile directly also allows pipenv to resolve things properly (see the previous commit in the tree).

[andreeagherman]

We had a similar issue:

We are using a private editable repo declared like:
{git = "[email protected]:compnay/library-name.git@version", editable=true}

Tried to change it to:
{ref = "version" git= "[email protected]/company/library-name", editable=true}

Pipenv still added ssh:// in front of it and then failed with an InstallationError:

pipenv.patched.notpip._internal.exceptions.InstallationError: Command "git clone -q ssh://[email protected]/company/library-name

Reverting to 2018.5.18 fixed the issue.

[techalchemy]

{git = "[email protected]:compnay/library-name.git@version", editable=true} => ` {git = "[email protected]:compnay/library-name.git", editable = true, ref = "version"}

There is supposed to be an SSH in front of he URI here, If you're not sure how to build a pipfile entry by hand please use the CLI to add your entry.

@jkp your pipfile says:

"lib1" = {editable = true, path = "lib1"}
"lib2" = {editable = true, path = "lib2"}
"22dede5" = {path = "./lib1", editable = true}
"2a0dbdb" = {path = "./lib2", editable = true}

You are installing the same package 2 times each. Is there a reason why?

For the record, I cloned your repository and did have a resolution issue. I then did as you suggested and put the dependencies from the sub-packages directly into the Pipfile:

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
notebook = "<5.5.0"
tensorflow = "*"

[requires]
python_version = "3.6"

Which caused the exact same resolution issue as I was already experiencing:

Virtualenv location: /home/hawk/.virtualenvs/pipenv-bugs-qLoyEUKf
Locking [dev-packages] dependencies...
Locking [packages] dependencies...

Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
  You can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
  Hint: try $ pipenv lock --pre if it is a pre-release dependency.
Could not find a version that matches html5lib!=1.0b1,!=1.0b2,!=1.0b3,!=1.0b4,!=1.0b5,!=1.0b6,!=1.0b7,!=1.0b8,==0.9999999,>=0.99999999pre
Tried: 1.0-reupload, 0.2, 0.9, 0.10, 0.11, 0.11.1, 0.90, 0.95, 0.99, 0.999, 0.9999, 0.99999, 0.999999, 0.9999999, 0.99999999, 0.999999999, 0.999999999, 1.0.1, 1.0.1
Skipped pre-versions: 1.0b1, 1.0b2, 1.0b3, 1.0b5, 1.0b6, 1.0b7, 1.0b8, 1.0b9, 1.0b10, 1.0b10
There are incompatible versions in the resolved dependencies.

So it looks to me like there is a legitimate conflict. I double checked, and the version of tensorboard this installs does pin a lower version of html5lib than it's dependency bleach installs. Newer versions of tensorboard don't pin html5lib at all.

[jkp]

The double entries are an accident - was throwing the example together in haste :)

Strange you got the conflict...I did not...see the first commit in the repo. How is that possible? It sort of implies there is a bug in the other direction...

Also, why did this work with the older version of pipenv? Was there a bug in the old dependency resolution code that has been fixed?

[jkp]

I believe this issue should be reopened - I've paired down the test-case even more now - it is reproducible on two different machines for me.

Please checkout this commit and run test.sh . The test script passes with pipenv==2018.5.18 and fails with pipenv==2018.6.25.

[tcwalther]

@jkp I just ran this on my own machine, and can confirm that it passes with pipenv==2018.5.18 and fails with pipenv==2018.6.25.

[Diggsey]

@techalchemy that's not correct - I can confirm that pipenv==2018.6.25 also broke our pipfile, and not only that but there is no solution that works under the new version of pipenv for private repos.

Adding an ssh:// in front does not work. We've had to roll back to pipenv==2018.5.18.

This is a recurring problem - pipenv regularly breaks installations from private repos on upgrades. Please add tests that cover this case before releasing new versions.

[techalchemy]

I don’t need to execute a script that runs pipenv install. @Diggsey if you have a problem please go open your own issue and if you would like code to be changed or added you are welcome to contribute. We cover many edge cases and do a lot of transformations on user input. Sometimes we don’t handle your specific one.

Simple solution: if you depend on pipenv, fix it. We are volunteers and none of us gets paid to do this. Follow the code of conduct, engage with us respectfully, and we will be happy to triage your issue. In the meantime be patient and keep your specific bugs in their own reports (and yours is already fixed in master within 2 days of release, not sure what else you want), and don’t come to the issue tracker just to tell us our stuff is bad.

[tcwalther]

@techalchemy I'd love to keep this discussion focused on the bug report. I don't know whether there is a history with previous bug reports that were filed in a non-constructive way, and if so, I really understand your pain. It is clear that everyone on this project is a volunteer, and you are perfectly right, people coming to this issue tracker shouldn't just complain about things being broken.

At the same time, there are various ways to commit to a project. Making a pull request with a working solution to a problem is definitely the strongest way to commit, but the barrier of entry for outsiders for a significantly complicated project such as pipenv is very high. Another way to contribute is to provide a simple test case to narrow down the problem, which the author of this bug, @jkp, has done.

You may find his test.sh script unnecessary; I personally very much enjoyed it - I used his example code to verify that the bug can be reproduced on my machine as well, and then commented in this thread to confirm that I can reproduce it. I have seen enough cases where a reported bug could not be reproduced on a different machine because of one person's weird local machine configuration that I've come to value independent verifications of a bug a lot.

If all of this sounds reasonable to you, I'd be very grateful if you reopened the ticket. Please don't hesitate to ask for further information or other test cases if it helps in narrowing down the issue - I'm hopeful that this way, we can split the workload in debugging this, and ultimately help on improving what is already a fantastic package manager for Python.

[jkp]

Hi again (@techalchemy ?)

I appreciate that I have not fixed the issue but I did manage to add an integration test that reproduces the issue within the pipenv codebase. The problem seems to be related to the version constraints on html5lib in the bleach library. Since pipenv can install and lock bleach==1.5.0 without the first command being run first, my expectation is that pipenv should be able to build a working dependency tree with the code shown - I would expect it to essentially uninstall the first version (2.1.3) and replace it with 1.5.0 (which it does) and build a dependency tree from there. It seems to fail somewhere on that journey. See the attached log:

jkp at Jamies-MacBook-Pro in ~/Development/src/pipenv (master●●)
$ PIPENV_PYTHON=2.7 pipenv run time pytest -v -n auto tests -k test_multiple_editable_dependencies_bleach_bug
Loading .env environment variables...
============================================================ test session starts =============================================================
platform darwin -- Python 2.7.10, pytest-3.6.1, py-1.5.3, pluggy-0.6.0 -- /Users/jkp/.local/share/virtualenvs/pipenv-9udIZ22Y-2.7/bin/python
cachedir: .pytest_cache
rootdir: /Users/jkp/Development/src/pipenv, inifile:
plugins: xdist-1.22.2, tap-2.2, forked-0.2, flaky-3.4.0, pypi-0.1.1
[gw0] darwin Python 2.7.10 cwd: /Users/jkp/Development/src/pipenv
[gw1] darwin Python 2.7.10 cwd: /Users/jkp/Development/src/pipenv
[gw2] darwin Python 2.7.10 cwd: /Users/jkp/Development/src/pipenv
[gw3] darwin Python 2.7.10 cwd: /Users/jkp/Development/src/pipenv
[gw0] Python 2.7.10 (default, Oct  6 2017, 22:29:07)  -- [GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.31)]
[gw1] Python 2.7.10 (default, Oct  6 2017, 22:29:07)  -- [GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.31)]
[gw2] Python 2.7.10 (default, Oct  6 2017, 22:29:07)  -- [GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.31)]
[gw3] Python 2.7.10 (default, Oct  6 2017, 22:29:07)  -- [GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.31)]
gw0 [1] / gw1 [1] / gw2 [1] / gw3 [1]
scheduling tests via LoadScheduling

tests/integration/test_install_twists.py::test_multiple_editable_dependencies_bleach_bug
[gw1] [100%] FAILED tests/integration/test_install_twists.py::test_multiple_editable_dependencies_bleach_bug

================================================================== FAILURES ==================================================================
_______________________________________________ test_multiple_editable_dependencies_bleach_bug _______________________________________________
[gw1] darwin -- Python 2.7.10 /Users/jkp/.local/share/virtualenvs/pipenv-9udIZ22Y-2.7/bin/python

PipenvInstance = <class 'tests.integration.conftest._PipenvInstance'>, pypi = <pytest_pypi.serve.Server object at 0x109691290>

    @pytest.mark.install
    @pytest.mark.local
    def test_multiple_editable_dependencies_bleach_bug(PipenvInstance, pypi):
        """Test-case for the issue reported here:
           https://github.com/pypa/pipenv/issues/2446
        """
        with PipenvInstance(pypi=pypi, chdir=True) as p:
            project = Project()
            testlib1 = os.path.join(p.path, 'testlib1')
            os.mkdir(testlib1)
            with open(os.path.join(testlib1, 'setup.py'), 'w') as fh:
                contents = """
    from setuptools import setup, find_packages
    setup(
    name='testpipenv',
    version='0.1',
    packages=[],
    install_requires=['bleach'],
    )
                """.strip()
                fh.write(contents)
            c = p.pipenv('install -e {0}'.format(testlib1))
            assert c.return_code == 0
            testlib2 = os.path.join(p.path, 'testlib2')
            os.mkdir(testlib2)
            with open(os.path.join(testlib2, 'setup.py'), 'w') as fh:
                contents = """
    from setuptools import setup, find_packages
    setup(
    name='testpipenv2',
    version='0.1',
    packages=[],
    install_requires=['bleach==1.5.0'],
    )
                """.strip()
                fh.write(contents)
            c = p.pipenv('install -e {0}'.format(testlib2))
>           assert c.return_code == 0
E           AssertionError: assert 1 == 0
E            +  where 1 = <Command 'pipenv install -e /private/var/folders/5y/l66h1wy9759dsshc7hmnw_bh0000gn/T/pipenv-GOWGDi-project/testlib2'>.return_code

tests/integration/test_install_twists.py:99: AssertionError
------------------------------------------------------------ Captured stdout call ------------------------------------------------------------
$ pipenv install -e /private/var/folders/5y/l66h1wy9759dsshc7hmnw_bh0000gn/T/pipenv-GOWGDi-project/testlib1
Installing -e /private/var/folders/5y/l66h1wy9759dsshc7hmnw_bh0000gn/T/pipenv-GOWGDi-project/testlib1...
Looking in indexes: http://127.0.0.1:52491/simple
Obtaining file:///private/var/folders/5y/l66h1wy9759dsshc7hmnw_bh0000gn/T/pipenv-GOWGDi-project/testlib1
Collecting bleach (from testpipenv==0.1)
  Downloading http://127.0.0.1:52491/bleach/bleach-2.1.3.tar.gz (60kB)
Collecting six (from bleach->testpipenv==0.1)
  Downloading http://127.0.0.1:52491/six/six-1.11.0-py2.py3-none-any.whl
Collecting html5lib!=1.0b1,!=1.0b2,!=1.0b3,!=1.0b4,!=1.0b5,!=1.0b6,!=1.0b7,!=1.0b8,>=0.99999999pre (from bleach->testpipenv==0.1)
  Downloading http://127.0.0.1:52491/html5lib/html5lib-1.0.1-py2.py3-none-any.whl (117kB)
Collecting webencodings (from html5lib!=1.0b1,!=1.0b2,!=1.0b3,!=1.0b4,!=1.0b5,!=1.0b6,!=1.0b7,!=1.0b8,>=0.99999999pre->bleach->testpipenv==0.1)
  Downloading http://127.0.0.1:52491/webencodings/webencodings-0.5.1.tar.gz
Building wheels for collected packages: bleach, webencodings
  Running setup.py bdist_wheel for bleach: started
  Running setup.py bdist_wheel for bleach: finished with status 'done'
  Stored in directory: /var/folders/5y/l66h1wy9759dsshc7hmnw_bh0000gn/T/pipenv-vMBaim-cache/wheels/e7/c7/12/e1794adac770ec317b5b826e7b136fdbaf6b32f2c802a8c704
  Running setup.py bdist_wheel for webencodings: started
  Running setup.py bdist_wheel for webencodings: finished with status 'done'
  Stored in directory: /var/folders/5y/l66h1wy9759dsshc7hmnw_bh0000gn/T/pipenv-vMBaim-cache/wheels/e8/ee/7e/1d8527ddaf96643b5bf0b2fd8d77c88d98a61f2ad2d8a3ac98
Successfully built bleach webencodings
Installing collected packages: six, webencodings, html5lib, bleach, testpipenv
  Running setup.py develop for testpipenv
Successfully installed bleach-2.1.3 html5lib-1.0.1 six-1.11.0 testpipenv webencodings-0.5.1

Adding -e /private/var/folders/5y/l66h1wy9759dsshc7hmnw_bh0000gn/T/pipenv-GOWGDi-project/testlib1 to Pipfile's [packages]...
Installing dependencies from Pipfile.lock (c510c1)...
To activate this project's virtualenv, run pipenv shell.
Alternativaly, run a command inside the virtualenv with pipenv run.

Creating a virtualenv for this project...
Pipfile: /private/var/folders/5y/l66h1wy9759dsshc7hmnw_bh0000gn/T/pipenv-GOWGDi-project/Pipfile
Using /Users/jkp/.local/share/virtualenvs/pipenv-9udIZ22Y-2.7/bin/python2.7 (2.7.10) to create virtualenv...
Running virtualenv with interpreter /Users/jkp/.local/share/virtualenvs/pipenv-9udIZ22Y-2.7/bin/python2.7
Using real prefix '/System/Library/Frameworks/Python.framework/Versions/2.7'
New python executable in /private/var/folders/5y/l66h1wy9759dsshc7hmnw_bh0000gn/T/pipenv-GOWGDi-project/.venv/bin/python2.7
Also creating executable in /private/var/folders/5y/l66h1wy9759dsshc7hmnw_bh0000gn/T/pipenv-GOWGDi-project/.venv/bin/python
Installing setuptools, pip, wheel...done.

Virtualenv location: /private/var/folders/5y/l66h1wy9759dsshc7hmnw_bh0000gn/T/pipenv-GOWGDi-project/.venv
Creating a Pipfile for this project...
Pipfile.lock not found, creating...
Locking [dev-packages] dependencies...
Locking [packages] dependencies...
Updated Pipfile.lock (c510c1)!

$ pipenv install -e /private/var/folders/5y/l66h1wy9759dsshc7hmnw_bh0000gn/T/pipenv-GOWGDi-project/testlib2
Installing -e /private/var/folders/5y/l66h1wy9759dsshc7hmnw_bh0000gn/T/pipenv-GOWGDi-project/testlib2...
Looking in indexes: http://127.0.0.1:52491/simple
Obtaining file:///private/var/folders/5y/l66h1wy9759dsshc7hmnw_bh0000gn/T/pipenv-GOWGDi-project/testlib2
Collecting bleach==1.5.0 (from testpipenv2==0.1)
  Downloading http://127.0.0.1:52491/bleach/bleach-1.5.0.tar.gz
Requirement already satisfied: six in ./.venv/lib/python2.7/site-packages (from bleach==1.5.0->testpipenv2==0.1) (1.11.0)
Collecting html5lib!=0.9999,!=0.99999,<0.99999999,>=0.999 (from bleach==1.5.0->testpipenv2==0.1)
  Downloading http://127.0.0.1:52491/html5lib/html5lib-0.9999999.tar.gz (889kB)
Building wheels for collected packages: bleach, html5lib
  Running setup.py bdist_wheel for bleach: started
  Running setup.py bdist_wheel for bleach: finished with status 'done'
  Stored in directory: /var/folders/5y/l66h1wy9759dsshc7hmnw_bh0000gn/T/pipenv-L1o7fk-cache/wheels/83/57/a5/2315c7d86547a795cc3859b4612d3506a5901f324bfeb6a0ff
  Running setup.py bdist_wheel for html5lib: started
  Running setup.py bdist_wheel for html5lib: finished with status 'done'
  Stored in directory: /var/folders/5y/l66h1wy9759dsshc7hmnw_bh0000gn/T/pipenv-L1o7fk-cache/wheels/c7/77/da/3abc5cc8dec74f994d715432510008efde812e6fae18829a23
Successfully built bleach html5lib
Installing collected packages: html5lib, bleach, testpipenv2
  Found existing installation: html5lib 1.0.1
    Uninstalling html5lib-1.0.1:
      Successfully uninstalled html5lib-1.0.1
  Found existing installation: bleach 2.1.3
    Uninstalling bleach-2.1.3:
      Successfully uninstalled bleach-2.1.3
  Running setup.py develop for testpipenv2
Successfully installed bleach-1.5.0 html5lib-0.9999999 testpipenv2

Adding -e /private/var/folders/5y/l66h1wy9759dsshc7hmnw_bh0000gn/T/pipenv-GOWGDi-project/testlib2 to Pipfile's [packages]...

Pipfile.lock (c510c1) out of date, updating to (2875e5)...
Locking [dev-packages] dependencies...
Locking [packages] dependencies...

Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
  You can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
  Hint: try $ pipenv lock --pre if it is a pre-release dependency.
Could not find a version that matches html5lib!=0.9999,!=0.99999,!=1.0b1,!=1.0b2,!=1.0b3,!=1.0b4,!=1.0b5,!=1.0b6,!=1.0b7,!=1.0b8,<0.99999999,>=0.999,>=0.99999999pre from http://127.0.0.1:52491/html5lib/html5lib-1.0.1-py2.py3-none-any.whl (from bleach->testpipenv==0.1->-r /var/folders/5y/l66h1wy9759dsshc7hmnw_bh0000gn/T/pipenv-I8KbtP-requirements/pipenv-YjIVWF-constraints.txt (line 3))
Tried: 0.9999999, 1.0.1
There are incompatible versions in the resolved dependencies.

/Users/jkp/Development/src/pipenv/pipenv/_compat.py:108: ResourceWarning: Implicitly cleaning up <TemporaryDirectory '/var/folders/5y/l66h1wy9759dsshc7hmnw_bh0000gn/T/pipenv-XKUHPu-requirements'>
  warnings.warn(warn_message, ResourceWarning)

------------------------------------------------------------ Captured stderr call ------------------------------------------------------------
127.0.0.1 - - [29/Jun/2018 14:12:18] "GET /simple/bleach/ HTTP/1.1" 200 318
127.0.0.1 - - [29/Jun/2018 14:12:18] "GET /bleach/bleach-2.1.3.tar.gz HTTP/1.1" 200 60141
127.0.0.1 - - [29/Jun/2018 14:12:19] "GET /simple/six/ HTTP/1.1" 200 320
127.0.0.1 - - [29/Jun/2018 14:12:19] "GET /six/six-1.11.0-py2.py3-none-any.whl HTTP/1.1" 200 10702
127.0.0.1 - - [29/Jun/2018 14:12:19] "GET /simple/html5lib/ HTTP/1.1" 200 374
127.0.0.1 - - [29/Jun/2018 14:12:19] "GET /html5lib/html5lib-1.0.1-py2.py3-none-any.whl HTTP/1.1" 200 117321
127.0.0.1 - - [29/Jun/2018 14:12:19] "GET /simple/webencodings/ HTTP/1.1" 200 273
127.0.0.1 - - [29/Jun/2018 14:12:19] "GET /webencodings/webencodings-0.5.1.tar.gz HTTP/1.1" 200 9721
127.0.0.1 - - [29/Jun/2018 14:12:21] "GET /simple/bleach/ HTTP/1.1" 200 318
127.0.0.1 - - [29/Jun/2018 14:12:21] "GET /simple/html5lib/ HTTP/1.1" 200 374
127.0.0.1 - - [29/Jun/2018 14:12:21] "GET /html5lib/html5lib-1.0.1-py2.py3-none-any.whl HTTP/1.1" 200 117321
127.0.0.1 - - [29/Jun/2018 14:12:21] "GET /simple/six/ HTTP/1.1" 200 320
127.0.0.1 - - [29/Jun/2018 14:12:21] "GET /six/six-1.11.0-py2.py3-none-any.whl HTTP/1.1" 200 10702
127.0.0.1 - - [29/Jun/2018 14:12:21] "GET /simple/webencodings/ HTTP/1.1" 200 273
127.0.0.1 - - [29/Jun/2018 14:12:21] "GET /simple/bleach/ HTTP/1.1" 200 318
127.0.0.1 - - [29/Jun/2018 14:12:21] "GET /simple/html5lib/ HTTP/1.1" 200 374
127.0.0.1 - - [29/Jun/2018 14:12:22] "GET /simple/six/ HTTP/1.1" 200 320
127.0.0.1 - - [29/Jun/2018 14:12:22] "GET /simple/webencodings/ HTTP/1.1" 200 273
127.0.0.1 - - [29/Jun/2018 14:12:22] "GET /simple/webencodings/ HTTP/1.1" 200 273
127.0.0.1 - - [29/Jun/2018 14:12:22] "GET /webencodings/webencodings-0.5.1.tar.gz HTTP/1.1" 200 9721
127.0.0.1 - - [29/Jun/2018 14:12:22] "GET /simple/six/ HTTP/1.1" 200 320
127.0.0.1 - - [29/Jun/2018 14:12:22] "GET /simple/bleach/ HTTP/1.1" 200 318
127.0.0.1 - - [29/Jun/2018 14:12:22] "GET /bleach/bleach-2.1.3.tar.gz HTTP/1.1" 200 60141
127.0.0.1 - - [29/Jun/2018 14:12:22] "GET /simple/six/ HTTP/1.1" 200 320
127.0.0.1 - - [29/Jun/2018 14:12:22] "GET /simple/html5lib/ HTTP/1.1" 200 374
127.0.0.1 - - [29/Jun/2018 14:12:22] "GET /simple/webencodings/ HTTP/1.1" 200 273
127.0.0.1 - - [29/Jun/2018 14:12:22] "GET /simple/html5lib/ HTTP/1.1" 200 374
127.0.0.1 - - [29/Jun/2018 14:12:22] "GET /simple/webencodings/ HTTP/1.1" 200 273
127.0.0.1 - - [29/Jun/2018 14:12:22] "GET /simple/six/ HTTP/1.1" 200 320
127.0.0.1 - - [29/Jun/2018 14:12:22] "GET /simple/html5lib/ HTTP/1.1" 200 374
127.0.0.1 - - [29/Jun/2018 14:12:22] "GET /html5lib/html5lib-1.0.1-py2.py3-none-any.whl HTTP/1.1" 200 117321
127.0.0.1 - - [29/Jun/2018 14:12:22] "GET /simple/webencodings/ HTTP/1.1" 200 273
127.0.0.1 - - [29/Jun/2018 14:12:22] "GET /webencodings/webencodings-0.5.1.tar.gz HTTP/1.1" 200 9721
127.0.0.1 - - [29/Jun/2018 14:12:22] "GET /simple/bleach/ HTTP/1.1" 200 318
127.0.0.1 - - [29/Jun/2018 14:12:22] "GET /bleach/bleach-2.1.3.tar.gz HTTP/1.1" 200 60141
127.0.0.1 - - [29/Jun/2018 14:12:22] "GET /simple/six/ HTTP/1.1" 200 320
127.0.0.1 - - [29/Jun/2018 14:12:22] "GET /six/six-1.11.0.tar.gz HTTP/1.1" 200 29860
127.0.0.1 - - [29/Jun/2018 14:12:22] "GET /six/six-1.11.0-py2.py3-none-any.whl HTTP/1.1" 200 10702
127.0.0.1 - - [29/Jun/2018 14:12:25] "GET /simple/bleach/ HTTP/1.1" 200 318
127.0.0.1 - - [29/Jun/2018 14:12:25] "GET /bleach/bleach-1.5.0.tar.gz HTTP/1.1" 200 36502
127.0.0.1 - - [29/Jun/2018 14:12:25] "GET /simple/html5lib/ HTTP/1.1" 200 374
127.0.0.1 - - [29/Jun/2018 14:12:25] "GET /html5lib/html5lib-0.9999999.tar.gz HTTP/1.1" 200 889312
127.0.0.1 - - [29/Jun/2018 14:12:28] "GET /simple/bleach/ HTTP/1.1" 200 318
127.0.0.1 - - [29/Jun/2018 14:12:28] "GET /bleach/bleach-2.1.3.tar.gz HTTP/1.1" 200 60141
127.0.0.1 - - [29/Jun/2018 14:12:29] "GET /simple/six/ HTTP/1.1" 200 320
127.0.0.1 - - [29/Jun/2018 14:12:29] "GET /six/six-1.11.0-py2.py3-none-any.whl HTTP/1.1" 200 10702
127.0.0.1 - - [29/Jun/2018 14:12:29] "GET /simple/html5lib/ HTTP/1.1" 200 374
127.0.0.1 - - [29/Jun/2018 14:12:29] "GET /html5lib/html5lib-1.0.1-py2.py3-none-any.whl HTTP/1.1" 200 117321
127.0.0.1 - - [29/Jun/2018 14:12:29] "GET /simple/webencodings/ HTTP/1.1" 200 273
127.0.0.1 - - [29/Jun/2018 14:12:29] "GET /webencodings/webencodings-0.5.1.tar.gz HTTP/1.1" 200 9721
127.0.0.1 - - [29/Jun/2018 14:12:29] "GET /simple/bleach/ HTTP/1.1" 200 318
127.0.0.1 - - [29/Jun/2018 14:12:29] "GET /simple/html5lib/ HTTP/1.1" 200 374
127.0.0.1 - - [29/Jun/2018 14:12:29] "GET /simple/six/ HTTP/1.1" 200 320
127.0.0.1 - - [29/Jun/2018 14:12:29] "GET /simple/html5lib/ HTTP/1.1" 200 374
127.0.0.1 - - [29/Jun/2018 14:12:30] "GET /simple/bleach/ HTTP/1.1" 200 318
127.0.0.1 - - [29/Jun/2018 14:12:30] "GET /bleach/bleach-2.1.3.tar.gz HTTP/1.1" 200 60141
127.0.0.1 - - [29/Jun/2018 14:12:30] "GET /simple/six/ HTTP/1.1" 200 320
127.0.0.1 - - [29/Jun/2018 14:12:30] "GET /simple/html5lib/ HTTP/1.1" 200 374
127.0.0.1 - - [29/Jun/2018 14:12:30] "GET /simple/webencodings/ HTTP/1.1" 200 273
127.0.0.1 - - [29/Jun/2018 14:12:30] "GET /webencodings/webencodings-0.5.1.tar.gz HTTP/1.1" 200 9721
127.0.0.1 - - [29/Jun/2018 14:12:30] "GET /simple/bleach/ HTTP/1.1" 200 318
127.0.0.1 - - [29/Jun/2018 14:12:30] "GET /simple/html5lib/ HTTP/1.1" 200 374
127.0.0.1 - - [29/Jun/2018 14:12:30] "GET /simple/six/ HTTP/1.1" 200 320
127.0.0.1 - - [29/Jun/2018 14:12:30] "GET /simple/html5lib/ HTTP/1.1" 200 374
===Flaky Test Report===


===End Flaky Test Report===
========================================================= 1 failed in 20.26 seconds ==========================================================
       20.88 real        21.25 user         7.46 sys

Hopefully this will help if someone has the time and knowledge to try to track down this problem.

[jkp]

Hi - I downloaded and tried the latest release and this still seems to be a live issue. Can I help to move it forward in any way?

[mattijohn]

@techalchemy I don't think you are correct that there should be a ssh:// in front of the URI. Looking at the comment in https://github.com/pypa/pipenv/blob/d47b43002a5674a005e1d5d152b687baec590326/pipenv/patched/notpip/_internal/vcs/git.py#L270 it looks like ssh:// doesn't work with GitHub.

Running on my own machine I get the following error:

pipenv.patched.notpip._internal.exceptions.InstallationError: Command "git clone -q ssh://[email protected]:private/repo.git /var/folders/m_/g7w0_d3512zch868x39xpm3r0000gn/T/tmp0mpnu3epsource/repo" failed with error code 128 in None

Executing the git clone command in the terminal results in this error: ssh: Could not resolve hostname github.com:private: nodename nor servname provided, or not known. However, executing the git clone without ssh:// works fine.

I haven't had time to come up with a fix but if nobody else picks it up I'd be happy to contribute later this week.

[uranusjr]

ssh:// totally works with GitHub. The problem is that we are using the wrong format; it should be ssh://[email protected]/private/repo.git instead.

The problem likely lies into how requirementslib converts the URL before passing it into Git. We need to convert the git@host:username/repo.git format into ssh://git@host/username/repo.git so we can correctly parse it with urllib.parse to get relevent URL information. I hope this helps you tracking down the problem

[mattijohn]

Thanks @uranusjr, you're right it's the git@host:username/repo.git format that doesn't work with ssh://. I think the get_url_rev function needs to be changed to change the format if it adds ssh://.

In the meantime users can workaround this in the latest version of pipenv by changing the URI in their Pipfile.

[jkp]

Folks, I think we're discussing two separate issues here. The issue I posted and have created a test-case for doesn't relate to ssh:// urls. Maybe migrate that discussion to a new issue so we can keep this one focussed?

[techalchemy]

Indeed. Not to mention that issue is already fixed and released.

[jkp]

@techalchemy which thing do you say is fixed?

[mattijohn]

@jkp apologies, I saw the second message in this thread matched my issue and assumed your issue was the same thing.

@techalchemy the problem with prefixing URI's containing : instead of / isn't fixed in the latest release: I'm using version 2018.7.1, freshly installed this morning. I think a new issue needs to be created for the URI format bug. I'm happy to contribute a pull request in the next day or two if nobody else gets round to it.

[techalchemy]

The ssh:// issue. I just saw that you have a test case in here — I haven’t looked at that yet but I’ll reopen now and take a look in a bit

[techalchemy]

@mattijohn if you have a problem and it isn’t about the specific thing being discussed please go fill out the issue template in a new issue with a reproducible test case.

[tcwalther]

@techalchemy This issue still persists with the current version 51cbc2e2f76a7c60f879a608fd96e95faca35f2e. It can be reproduced using @jkp's example code that he linked to in the bug description.

It would be great if you could re-open this issue.

[lovepocky]

this issue still exists in version 2018.11.26

[festum]

The issue still there with version 2018.11.26 cmd pipenv install -e git+ssh://[email protected]:7999/org/repo.git#egg=repo.