Pipenv: Can not read from ssh repos

Created on 26 Dec 2018 · 4Comments · Source: pypa/pipenv

Hey,

I have some problem with installing from ssh-repos. I first thought this was a problem with gitlab, but I have the same problem on github.

Issue description

pipenv install -e "[email protected]:Corvan/upyls.git#egg=upyls" fails. Does not matter if I use the ssh:// way of defining the repo and it fails with any other git repo over ssh for me. Tried this as well in a completely new (empty) pipenv folder.

I have loaded my ssh-key via ssh-add, and I can successfully run git clone [email protected]:Corvan/upyls.git

Expected result

Installing the package works

Actual result

18-12-26 14:31:12 lars@Denephor:/home/lars/Workspace/pipenvtest  $ pipenv install -e "[email protected]:Corvan/upyls.git#egg=upyls"
Installing -e [email protected]:Corvan/upyls.git#egg=upyls...
Adding upyls to Pipfile's [packages]...
✔ Installation Succeeded
Pipfile.lock (494624) out of date, updating to (a65489)...
Locking [dev-packages] dependencies...
Locking [packages] dependencies...
⠙ Pinning VCS Packages...ssh: connect to host github.com:Corvan port 22: Connection refused
fatal: Konnte nicht vom Remote-Repository lesen. (Could not read from remote repository)

Bitte stellen Sie sicher, dass die korrekten Zugriffsberechtigungen bestehen
und das Repository existiert. (Please make sure you have the proper rights to access the repository and make sure it exists)
Traceback (most recent call last):
  File "/usr/local/bin/pipenv", line 11, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python2.7/dist-packages/pipenv/vendor/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/pipenv/vendor/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python2.7/dist-packages/pipenv/vendor/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python2.7/dist-packages/pipenv/vendor/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python2.7/dist-packages/pipenv/vendor/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/pipenv/vendor/click/decorators.py", line 64, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/pipenv/vendor/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/pipenv/vendor/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/pipenv/cli/command.py", line 254, in install
    editable_packages=state.installstate.editables,
  File "/usr/local/lib/python2.7/dist-packages/pipenv/core.py", line 1992, in do_install
    skip_lock=skip_lock,
  File "/usr/local/lib/python2.7/dist-packages/pipenv/core.py", line 1221, in do_init
    pypi_mirror=pypi_mirror,
  File "/usr/local/lib/python2.7/dist-packages/pipenv/core.py", line 1068, in do_lock
    lockfile=lockfile
  File "/usr/local/lib/python2.7/dist-packages/pipenv/utils.py", line 626, in venv_resolve_deps
    dev=dev,
  File "/usr/local/lib/python2.7/dist-packages/pipenv/utils.py", line 1311, in get_vcs_deps
    with locked_repository(requirement) as repo:
  File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "/usr/local/lib/python2.7/dist-packages/pipenv/utils.py", line 1447, in locked_repository
    with requirement.req.locked_vcs_repo(src_dir=src_dir) as repo:
  File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "/usr/local/lib/python2.7/dist-packages/pipenv/vendor/requirementslib/models/requirements.py", line 840, in locked_vcs_repo
    vcsrepo = self.get_vcs_repo(src_dir=src_dir)
  File "/usr/local/lib/python2.7/dist-packages/pipenv/vendor/requirementslib/models/requirements.py", line 802, in get_vcs_repo
    vcsrepo.obtain()
  File "/usr/local/lib/python2.7/dist-packages/pipenv/vendor/requirementslib/models/vcs.py", line 37, in obtain
    self.repo_instance.obtain(self.checkout_directory)
  File "/usr/local/lib/python2.7/dist-packages/pipenv/patched/notpip/_internal/vcs/__init__.py", line 332, in obtain
    self.fetch_new(dest, url, rev_options)
  File "/usr/local/lib/python2.7/dist-packages/pipenv/patched/notpip/_internal/vcs/git.py", line 198, in fetch_new
    self.run_command(['clone', '-q', url, dest])
  File "/usr/local/lib/python2.7/dist-packages/pipenv/patched/notpip/_internal/vcs/__init__.py", line 456, in run_command
    spinner=spinner)
  File "/usr/local/lib/python2.7/dist-packages/pipenv/patched/notpip/_internal/utils/misc.py", line 705, in call_subprocess
    % (command_desc, proc.returncode, cwd))
pipenv.patched.notpip._internal.exceptions.InstallationError: Command "git clone -q ssh://[email protected]:Corvan/upyls.git /tmp/requirementslibIhzu9R/upyls" failed with error code 128 in None

Steps to replicate

Create a new empty folder and run pipenv install -e "[email protected]:Corvan/upyls.git#egg=upyls"

$ pipenv --support

Pipenv version: '2018.11.26'

Pipenv location: '/usr/local/lib/python2.7/dist-packages/pipenv'

Python location: '/usr/bin/python'

Python installations found:

3.7.1: /usr/bin/python3.7
3.6.7: /usr/bin/python3.6
3.6.7: /usr/bin/python3.6m
3.7.1: /usr/bin/python3.7m
2.7.15: /usr/bin/python2

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '0',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '4.18.0-13-generic',
 'platform_system': 'Linux',
 'platform_version': '#14-Ubuntu SMP Wed Dec 5 09:04:24 UTC 2018',
 'python_full_version': '2.7.15+',
 'python_version': '2.7',
 'sys_platform': 'linux2'}

System environment variables:

USERNAME
VIRTUAL_ENV_DISABLE_PROMPT
QT_QPA_PLATFORMTHEME
LESS
LC_CTYPE
WINDOWPATH
XDG_CURRENT_DESKTOP
XDG_SESSION_TYPE
LOGNAME
USER
PATH
XDG_VTNR
HOME
ZSH
KDE_SESSION_VERSION
KDE_FULL_SESSION
LANG
PROFILEHOME
TERM
SHELL
COLORFGBG
XAUTHORITY
LANGUAGE
SESSION_MANAGER
SHLVL
XDG_SESSION_ID
MANDATORY_PATH
KONSOLE_PROFILE_NAME
SHELL_SESSION_ID
DISPLAY
QT_ACCESSIBILITY
WINDOWID
SSH_AGENT_PID
XDG_RUNTIME_DIR
LIBVIRT_DEFAULT_URI
GPG_AGENT_INFO
KICAD_PATH
KONSOLE_DBUS_SESSION
QT_AUTO_SCREEN_SCALE_FACTOR
GTK2_MODULES
XDG_SESSION_DESKTOP
SSH_AUTH_SOCK
COLORTERM
PYTHONPATH
KDE_SESSION_UID
KONSOLE_DBUS_SERVICE
GDMSESSION
PIP_SHIMS_BASE_MODULE
GTK2_RC_FILES
PIP_PYTHON_PATH
PYTHONDONTWRITEBYTECODE
GS_LIB
DBUS_SESSION_BUS_ADDRESS
_
KIGITHUB
DESKTOP_SESSION
LSCOLORS
XDG_CONFIG_DIRS
DEFAULTS_PATH
XCURSOR_THEME
PIP_DISABLE_PIP_VERSION_CHECK
OLDPWD
GTK_MODULES
XDG_DATA_DIRS
PWD
PYTHONFINDER_IGNORE_UNSUPPORTED
GTK_RC_FILES
XCURSOR_SIZE
LS_COLORS
PAGER
XDG_SEAT

Pipenv–specific environment variables:

Debug–specific environment variables:

PATH: /home/lars/bin:/home/lars/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
SHELL: /usr/bin/zsh
LANG: de_DE.UTF-8
PWD: /home/lars/Workspace/pipenvtest

Contents of Pipfile ('/home/lars/Workspace/pipenvtest/Pipfile'):

[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[dev-packages]

[packages]
upyls = {git = "[email protected]:Corvan/upyls.git",editable = true}

[requires]
python_version = "3.7"

Contents of Pipfile.lock ('/home/lars/Workspace/pipenvtest/Pipfile.lock'):

{
    "_meta": {
        "hash": {
            "sha256": "7e7ef69da7248742e869378f8421880cf8f0017f96d94d086813baa518a65489"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_version": "3.7"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {},
    "develop": {}
}

Type

Source

Corvan

Most helpful comment

I found the solution:
it must be
pipenv install -e "[email protected]/Corvan/upyls.git#egg=upyls"
It must be written as absolute path with a / not with a colon, maybe it might help, adding a note here to the documentation.

Corvan on 29 Dec 2018

👍10 ❤5

All 4 comments

git clone -q ssh://[email protected]:Corvan/upyls.git
ssh: Could not resolve hostname github.com:Corvan: nodename nor servname provided, or not known

frostming on 27 Dec 2018

yes, I have them same with ssh:// but not without:
git clone [email protected]:Corvan/upyls.git

Corvan on 29 Dec 2018

👍10 ❤5

the mentioned solution is a workaround, it will not work going forward.

The actual solution is to use the full ssh url: git+ssh://[email protected]/Corvan/upyls.git#egg=upyls until we release the new version of pipenv with an updated requirementslib release.

Note that for the format you are using here, pip specifically indicates that the colon is the required syntax. From the documentation (note the last example, the only one which matches your syntax).

[-e] git://git.example.com/MyProject#egg=MyProject
[-e] git+http://git.example.com/MyProject#egg=MyProject
[-e] git+https://git.example.com/MyProject#egg=MyProject
[-e] git+ssh://git.example.com/MyProject#egg=MyProject
[-e] git+git://git.example.com/MyProject#egg=MyProject
[-e] git+file:///home/user/projects/MyProject#egg=MyProject
-e [email protected]:MyProject#egg=MyProject

The important distinction here is that the parser reads the element before the + as the VCS indicator and splits it off, then it either parses the scheme as defined by whatever spec defines URI standards, but which includes everything until ://. IF there is no scheme present _and_ the line is prefixed with a VCS prefix, the parser _assumes_ it is an ssh connection and prefixes it for you. Additionally it reformats the URL accordingly.