Pipenv: Lock fails with flit = "*" in dev packages

Created on 29 Mar 2018 · 6Comments · Source: pypa/pipenv

The pipenv lock command fails when I have flit = "*" in my dev-packages section of Pipfile. Tried as well with flit in the packages section, same result.

I understand that it might be because flit is different and may not be already supported by pip or else, so feel free to close if this has nothing to do with pipenv :smiley_cat:

$ python -m pipenv.help output

Pipenv version: '11.9.0'

Pipenv location: '/home/pawamoy/.local/venvs/pipenv/lib/python3.5/site-packages/pipenv'

Python location: '/home/pawamoy/.local/venvs/pipenv/bin/python'

Other Python installations in PATH:

2.7: /home/pawamoy/.pyenv/shims/python2.7
2.7: /home/pawamoy/.pyenv/shims/python2.7
2.7: /home/pawamoy/.pyenv/shims/python2.7
2.7: /usr/bin/python2.7
3.4: /usr/bin/python3.4m
3.4: /usr/bin/python3.4
3.5: /home/pawamoy/.pyenv/shims/python3.5m
3.5: /home/pawamoy/.local/venvs/pipenv/bin/python3.5
3.5: /home/pawamoy/.pyenv/shims/python3.5
3.5: /home/pawamoy/.pyenv/shims/python3.5
3.6: /home/pawamoy/.pyenv/shims/python3.6m
3.6: /home/pawamoy/.pyenv/shims/python3.6
3.6: /home/pawamoy/.pyenv/shims/python3.6
3.7: /home/pawamoy/.pyenv/shims/python3.7m
3.7: /home/pawamoy/.pyenv/shims/python3.7
3.7: /home/pawamoy/.pyenv/shims/python3.7
3.5.5: /home/pawamoy/.local/venvs/pipenv/bin/python
3.5.5: /home/pawamoy/.pyenv/shims/python
3.5.5: /home/pawamoy/.pyenv/shims/python
2.7.9: /usr/bin/python
2.7.9: /home/pawamoy/.pyenv/shims/python2
2.7.9: /home/pawamoy/.pyenv/shims/python2
2.7.9: /usr/bin/python2
3.5.5: /home/pawamoy/.local/venvs/pipenv/bin/python3
3.5.5: /home/pawamoy/.pyenv/shims/python3
3.5.5: /home/pawamoy/.pyenv/shims/python3
3.4.2: /usr/bin/python3

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.5.5',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '3.16.0-5-amd64',
 'platform_system': 'Linux',
 'platform_version': '#1 SMP Debian 3.16.51-3+deb8u1 (2018-01-08)',
 'python_full_version': '3.5.5',
 'python_version': '3.5',
 'sys_platform': 'linux'}

System environment variables:

XDG_SESSION_PATH
GOENV_SHELL
USER
PYTHONDONTWRITEBYTECODE
BASH_FUNC_shellm-include%%
PS4
XAUTHORITY
EDITOR_GUI
SHELLHISTORY_FILE
OLDPWD
BASH_FUNC_shellm-ndef%%
XDG_SESSION_ID
EDITOR
XDG_VTNR
RBENV_SHELL
PS1
BASH_FUNC_shellm-define%%
_
PWD
BASH_FUNC_shellm%%
BASHER_ROOT
BASHER_SHELL
LOGNAME
SHELLM_PROJECT
DESKTOP_SESSION
BASH_FUNC_shellm-find-lib%%
LANG
SSH_AUTH_SOCK
HISTTIMEFORMAT
EDITOR_CLI
HISTSIZE
DISPLAY
PYTHONUNBUFFERED
HISTFILESIZE
WINDOWID
SSH_AGENT_PID
XDG_SEAT
INSTALLED
GPG_AGENT_INFO
SHENV_ROOT
PACKAGE_HOME
XDG_GREETER_DATA_DIR
HOME
SHENV_SHELL
PYDJAMAS
MANPATH
WORKON_HOME
PROJECT_HOME
_SHELLHISTORY_UUID
BASH_FUNC_shellm-exclude%%
ISOLATED
FILEMANAGER_CLI
BUILD_A_SPACE
QT_STYLE_OVERRIDE
SHELLHISTORY_VENV
PIP_PYTHON_PATH
GOPATH
PAPERLESS_PASSPHRASE
BASH_FUNC_shenv%%
DBUS_SESSION_BUS_ADDRESS
XDG_SEAT_PATH
TERM
DATA
SHENV_VERSION
XDG_CURRENT_DESKTOP
VISUAL
GDMSESSION
SHENV_HOOK_PATH
PYENV_SHELL
SHENV_DIR
TERMINATOR_UUID
BASHER_PREFIX
GIT_HOME
PAPERLESS_CONSUMPTION_DIR
SHELL
BASH_FUNC_shellm-find-script%%
SHELLM_PROJECT_PROFILE
SHLVL
XDG_RUNTIME_DIR
KEYCUT_DATA
PATH
BASH_FUNC_man%%
LIBPATH
GIT_PS1_SHOWCOLORHINTS
COLORTERM
VIRTUAL_ENV
FILEMANAGER_GUI
LANGUAGE
LS_COLORS

Pipenv–specific environment variables:

Debug–specific environment variables:

PATH: /home/pawamoy/.local/venvs/pipenv/bin:/home/pawamoy/.basher/cellar/bin:/home/pawamoy/.basher/bin:/home/pawamoy/.local/bin:/home/pawamoy/.shenv/shims:/home/pawamoy/.shenv/bin:/home/pawamoy/.go/bin:/home/pawamoy/.goenv/shims:/home/pawamoy/.goenv/bin:/home/pawamoy/.pyenv/shims:/home/pawamoy/.pyenv/bin:/home/pawamoy/.rbenv/shims:/home/pawamoy/.rbenv/bin:/home/pawamoy/.shenv/versions/bash-4.4.12/bin:/home/pawamoy/.shenv/libexec:/home/pawamoy/.shenv/plugins/shell-build/bin:/home/pawamoy/.basher/cellar/bin:/home/pawamoy/.basher/bin:/home/pawamoy/.local/bin:/home/pawamoy/.shenv/shims:/home/pawamoy/.shenv/bin:/home/pawamoy/.go/bin:/home/pawamoy/.goenv/shims:/home/pawamoy/.goenv/bin:/home/pawamoy/.pyenv/shims:/home/pawamoy/.pyenv/bin:/home/pawamoy/.rbenv/shims:/home/pawamoy/.rbenv/bin:/media/pawamoy/Data/git/pawamoy/shellmpawamoy/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
SHELL: /bin/bash
EDITOR: /usr/bin/vim
LANG: en_US.UTF-8
PWD: /media/pawamoy/Data/git/pawamoy/neopy
VIRTUAL_ENV: /home/pawamoy/.local/venvs/pipenv

Contents of Pipfile ('/media/pawamoy/Data/git/pawamoy/neopy/Pipfile'):

[[source]]
url = "https://pypi.python.org/simple"
verify_ssl = true
name = "pypi"

[requires]
python_version = "3.5"

[dev-packages]
ipython = "*"
pytest = "*"
pytest-sugar = "*"
pypandoc = "*"
flit = "*"

[packages]
"neo4j-driver" = "*"

Expected result

No errors :grin:

Actual result

$ pipenv lock --verbose
Locking [dev-packages] dependencies…
Using pip: -i https://pypi.python.org/simple

                          ROUND 1                           
Current constraints:
  flit

Finding the best candidates:

Traceback (most recent call last):
  File "/home/pawamoy/.local/venvs/pipenv/lib/python3.5/site-packages/pipenv/vendor/pip9/_vendor/packaging/specifiers.py", line 601, in __init__
    parsed.add(Specifier(specifier))
  File "/home/pawamoy/.local/venvs/pipenv/lib/python3.5/site-packages/pipenv/vendor/pip9/_vendor/packaging/specifiers.py", line 85, in __init__
    raise InvalidSpecifier("Invalid specifier: '{0}'".format(spec))
pip9._vendor.packaging.specifiers.InvalidSpecifier: Invalid specifier: '3'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/pawamoy/.local/venvs/pipenv/lib/python3.5/site-packages/pipenv/resolver.py", line 82, in <module>
    main()
  File "/home/pawamoy/.local/venvs/pipenv/lib/python3.5/site-packages/pipenv/resolver.py", line 71, in main
    clear=do_clear,
  File "/home/pawamoy/.local/venvs/pipenv/lib/python3.5/site-packages/pipenv/resolver.py", line 63, in resolve
    verbose=verbose,
  File "/home/pawamoy/.local/venvs/pipenv/lib/python3.5/site-packages/pipenv/utils.py", line 425, in resolve_deps
    pre,
  File "/home/pawamoy/.local/venvs/pipenv/lib/python3.5/site-packages/pipenv/utils.py", line 336, in actually_resolve_reps
    resolved_tree.update(resolver.resolve(max_rounds=PIPENV_MAX_ROUNDS))
  File "/home/pawamoy/.local/venvs/pipenv/lib/python3.5/site-packages/pipenv/patched/piptools/resolver.py", line 102, in resolve
    has_changed, best_matches = self._resolve_one_round()
  File "/home/pawamoy/.local/venvs/pipenv/lib/python3.5/site-packages/pipenv/patched/piptools/resolver.py", line 193, in _resolve_one_round
    best_matches = {self.get_best_match(ireq) for ireq in constraints}
  File "/home/pawamoy/.local/venvs/pipenv/lib/python3.5/site-packages/pipenv/patched/piptools/resolver.py", line 193, in <setcomp>
    best_matches = {self.get_best_match(ireq) for ireq in constraints}
  File "/home/pawamoy/.local/venvs/pipenv/lib/python3.5/site-packages/pipenv/patched/piptools/resolver.py", line 257, in get_best_match
    best_match = self.repository.find_best_match(ireq, prereleases=self.prereleases)
  File "/home/pawamoy/.local/venvs/pipenv/lib/python3.5/site-packages/pipenv/patched/piptools/repositories/pypi.py", line 116, in find_best_match
    all_candidates = self.find_all_candidates(ireq.name)
  File "/home/pawamoy/.local/venvs/pipenv/lib/python3.5/site-packages/pipenv/patched/piptools/repositories/pypi.py", line 101, in find_all_candidates
    candidates = self.finder.find_all_candidates(req_name)
  File "/home/pawamoy/.local/venvs/pipenv/lib/python3.5/site-packages/pipenv/patched/notpip/index.py", line 460, in find_all_candidates
    self._package_versions(page.links, search)
  File "/home/pawamoy/.local/venvs/pipenv/lib/python3.5/site-packages/pipenv/patched/notpip/index.py", line 635, in _package_versions
    v = self._link_package_versions(link, search)
  File "/home/pawamoy/.local/venvs/pipenv/lib/python3.5/site-packages/pipenv/patched/notpip/index.py", line 707, in _link_package_versions
    support_this_python = check_requires_python(link.requires_python)
  File "/home/pawamoy/.local/venvs/pipenv/lib/python3.5/site-packages/pipenv/patched/notpip/utils/packaging.py", line 30, in check_requires_python
    requires_python_specifier = specifiers.SpecifierSet(requires_python)
  File "/home/pawamoy/.local/venvs/pipenv/lib/python3.5/site-packages/pipenv/vendor/pip9/_vendor/packaging/specifiers.py", line 603, in __init__
    parsed.add(LegacySpecifier(specifier))
  File "/home/pawamoy/.local/venvs/pipenv/lib/python3.5/site-packages/pipenv/vendor/pip9/_vendor/packaging/specifiers.py", line 85, in __init__
    raise InvalidSpecifier("Invalid specifier: '{0}'".format(spec))
pip9._vendor.packaging.specifiers.InvalidSpecifier: Invalid specifier: '3'

Steps to replicate

mkdir test_project
cd test_project
pipenv install --dev flit

Source

pawamoy

Most helpful comment

if you are trying to teach us about python specifiers we are pretty familiar with them

I didn't mean to condescend. I was trying to record the information that (1) I learned while trying to dig deeper into the issue and that (2) looked relevant to the scope of the issue (whether it's likely to be just flit, or might affect other packages too). In other projects, I've found it helpful when people leave notes about what they've learned so that other people can pick up where they left off. I'll be more minimal if I interact with this project again.

If you would like to be constructive please open an issue describing your problem with an example

Done: #2343. I was trying to be constructive above, and I'm trying to be constructive in that bug report too. I remain open to feedback about whether I've accomplished that and what I can do to improve, although I would appreciate it if it came from an assumption of good faith.

As far as I know there is nothing about flit that makes it incompatible with pipenv

My understanding of this issue is that it reports an unresolved incompatibility between flit and pipenv, and that it was closed, effectively, as “will not fix”. My question was whether there's a workaround, since I was unable to find one.

I don't know why you are linking 10 year old peps

To explain why the package repository might contain packages with versions such as Requires-Python: 3. This seems like it might bear on the scope of the issue. I considered the upside that more information might be useful; I didn't consider that including it might be offensive.

I'm not super interested in having negative interactions today

Me neither! Or ever! Please give me the benefit of a doubt — I may have written confusingly, but not malevolently, and was surprised to receive a harsh response.

osteele on 11 Jun 2018

👍2

All 6 comments

Well, after a long walk up to the vendor of the vendor of the vendor of pipenv, it turns out that some legacy versions of flit define version specifiers not complying with PEP 440('3' instead of '>=3') in their python_requires field. It makes pip fail to parse the package info.

It is an issue related to the upstream of pipenv. I'm not sure whether flit is the only one and whether we should support it. So let's stop here, I don't think the maintainers of pipenv are likely to patch for it.

frostming on 30 Mar 2018

Alright, I would vote to close as well. Thanks for the investigation!

pawamoy on 30 Mar 2018

Thanks for tracking that down @frostming! Much appreciated!

techalchemy on 31 Mar 2018

Is there a workaround, or is flit permanently incompatible with pipenv?

Note that the current version of flit specifies specifies Requires-Python: >=3. This has been true of all releases as of flit v0.1, released 2016-10-16. Yet the presence of the previous versions of flit in pypi prevents pipenv (but not pip) from installing recent versions of flit (0.10 through the current 1.0). flit = "*", flit = "1.0", and "flit = ">=1.0", in [packages] or [dev-packages], all cause pipenv install to fail with the symptom reported in this bug report.

Also note that version specifiers of the form Requires-Python: 3 were valid from 2005-04-28 (PEP 345) through 2009-06-04 (PEP 386). The “Requires Python” section of PEP 345 gives Requires-Python: 2.5 as an example. Although the offending flit releases were not written within this window, it's likely that other packages had releases during that time, or incorrectly used that version description format even after PEP 386 changed it (especially since PEP 345 doesn't note that it's been superseded). If I understand this issue correctly, pipenv will fail to install those packages too.

osteele on 11 Jun 2018

hi @osteele, I am not really following what you are saying here, but if you are trying to teach us about python specifiers we are pretty familiar with them. If you would like to be constructive please open an issue describing your problem with an example.

Is there a workaround, or is flit permanently incompatible with pipenv?

As far as I know there is nothing about flit that makes it incompatible with pipenv, I don't know why you are linking 10 year old peps, but I'm not super interested in having negative interactions today so if you want to describe a problem or a bug and you are hoping to solve it productively, please open an issue and fill out the issue template

techalchemy on 11 Jun 2018

👎1

if you are trying to teach us about python specifiers we are pretty familiar with them

If you would like to be constructive please open an issue describing your problem with an example

As far as I know there is nothing about flit that makes it incompatible with pipenv

I don't know why you are linking 10 year old peps

I'm not super interested in having negative interactions today

Me neither! Or ever! Please give me the benefit of a doubt — I may have written confusingly, but not malevolently, and was surprised to receive a harsh response.

osteele on 11 Jun 2018

👍2

Was this page helpful?

0 / 5 - 0 ratings