Poetry: Poetry fails when there is something already installed with a bad version

[bdc34]

• [ x] I am on the latest Poetry version.
• [x ] I have searched the issues of this repo and believe that this is not a duplicate.
• [ ] If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: Linux bdc34-Latitude-E7450 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

• Poetry version: Poetry version 1.0.5

• Link of a Gist with the contents of your pyproject.toml file: not relevant

Issue

When I'm using a venv and I already have a package installed with a version that is not a semver, I get an error.

Expected behavior: Just ignore the package?

I get this error:

$ poetry install -v
Using virtualenv: /home/bdc34/.pyenv/versions/3.6.9/envs/submission-core

[ParseVersionError]
Unable to parse "no-git-or-release-version".

Traceback (most recent call last):
  File "/home/bdc34/.pyenv/versions/3.6.9/envs/submission-core/lib/python3.6/site-packages/clikit/console_application.py", line 131, in run
    status_code = command.handle(parsed_args, io)
  File "/home/bdc34/.pyenv/versions/3.6.9/envs/submission-core/lib/python3.6/site-packages/clikit/api/command/command.py", line 120, in handle
    status_code = self._do_handle(args, io)
  File "/home/bdc34/.pyenv/versions/3.6.9/envs/submission-core/lib/python3.6/site-packages/clikit/api/command/command.py", line 171, in _do_handle
    return getattr(handler, handler_method)(args, io, self)
  File "/home/bdc34/.pyenv/versions/3.6.9/envs/submission-core/lib/python3.6/site-packages/cleo/commands/command.py", line 92, in wrap_handle
    return self.handle()
  File "/home/bdc34/.pyenv/versions/3.6.9/envs/submission-core/lib/python3.6/site-packages/poetry/console/commands/install.py", line 48, in handle
    self.io, self.env, self.poetry.package, self.poetry.locker, self.poetry.pool
  File "/home/bdc34/.pyenv/versions/3.6.9/envs/submission-core/lib/python3.6/site-packages/poetry/installation/installer.py", line 55, in __init__
    installed = self._get_installed()
  File "/home/bdc34/.pyenv/versions/3.6.9/envs/submission-core/lib/python3.6/site-packages/poetry/installation/installer.py", line 488, in _get_installed
    return InstalledRepository.load(self._env)
  File "/home/bdc34/.pyenv/versions/3.6.9/envs/submission-core/lib/python3.6/site-packages/poetry/repositories/installed_repository.py", line 28, in load
    package = Package(name, version, version)
  File "/home/bdc34/.pyenv/versions/3.6.9/envs/submission-core/lib/python3.6/site-packages/poetry/packages/package.py", line 46, in __init__
    self._version = Version.parse(version)
  File "/home/bdc34/.pyenv/versions/3.6.9/envs/submission-core/lib/python3.6/site-packages/poetry/semver/version.py", line 206, in parse
    raise ParseVersionError('Unable to parse "{}".'.format(text))

[finswimmer]

Hello @bdc34,

When I'm using a venv and I already have a package installed with a version that is not a semver, I get an error.

Could you describe this a bit more detailed, please? What steps are we need to do, to reproduce your issue?

fin swimmer

[bdc34]

Thanks for your questions.

1. Create and install a package with the version "no-git-or-release-version".
2. Run poetry install -v

Result: error
Expected: poetry installs the projects dependencies. (Maybe poetry needs to assume that this package with a bad version is of a very low version number?) Or maybe poetry should exit with a message here like "There is a package wonkoversion with version 'super.bad.version' that is causing poetry to fail. Please uninstall it."

[SteveClement]

I have a very similar error, but I quite honestly have no clue where "git-540969f" comes from :(

I could just hard reset my entire env but that would take the fun away of figuring out where my issue is :)

I also found issue https://github.com/python-poetry/poetry/issues/637 which looks similar.

$ poetry install -vvv
Using virtualenv: /home/steve/.cache/pypoetry/virtualenvs/safebr-bn3pauzM-py3.7

[ParseVersionError]
Unable to parse "git-540969f".

Traceback (most recent call last):
  File "/home/steve/.poetry/lib/poetry/_vendor/py3.7/clikit/console_application.py", line 131, in run
    status_code = command.handle(parsed_args, io)
  File "/home/steve/.poetry/lib/poetry/_vendor/py3.7/clikit/api/command/command.py", line 120, in handle
    status_code = self._do_handle(args, io)
  File "/home/steve/.poetry/lib/poetry/_vendor/py3.7/clikit/api/command/command.py", line 171, in _do_handle
    return getattr(handler, handler_method)(args, io, self)
  File "/home/steve/.poetry/lib/poetry/_vendor/py3.7/cleo/commands/command.py", line 92, in wrap_handle
    return self.handle()
  File "/home/steve/.poetry/lib/poetry/console/commands/install.py", line 54, in handle
    self.io, self.env, self.poetry.package, self.poetry.locker, self.poetry.pool
  File "/home/steve/.poetry/lib/poetry/installation/installer.py", line 55, in __init__
    installed = self._get_installed()
  File "/home/steve/.poetry/lib/poetry/installation/installer.py", line 488, in _get_installed
    return InstalledRepository.load(self._env)
  File "/home/steve/.poetry/lib/poetry/repositories/installed_repository.py", line 28, in load
    package = Package(name, version, version)
  File "/home/steve/.poetry/lib/poetry/packages/package.py", line 46, in __init__
    self._version = Version.parse(version)
  File "/home/steve/.poetry/lib/poetry/semver/version.py", line 206, in parse
    raise ParseVersionError('Unable to parse "{}".'.format(text))

[randallpittman]

I have this issue with the MATLAB Engine for Python (not available on PyPI--comes with MATLAB and has to be installed manually).

Poetry 1.1.4 BTW.

After installing, pip list shows the version for matlabengineforpython as R2020b.

I do not have matlabengineforpython package included in my pyproject.toml file since it's not pip-installable, but poetry update fails just because it doesn't know how to handle R2020b as a version string.

poetry update -vvv output:

Using virtualenv: /shared/Development/waveradar-repo/radar_proc/.venv

  Stack trace:

  10  /shared/poetry/lib/poetry/_vendor/py3.7/clikit/console_application.py:131 in run
       129│             parsed_args = resolved_command.args
       130│ 
     → 131│             status_code = command.handle(parsed_args, io)
       132│         except KeyboardInterrupt:
       133│             status_code = 1

   9  /shared/poetry/lib/poetry/_vendor/py3.7/clikit/api/command/command.py:120 in handle
       118│     def handle(self, args, io):  # type: (Args, IO) -> int
       119│         try:
     → 120│             status_code = self._do_handle(args, io)
       121│         except KeyboardInterrupt:
       122│             if io.is_debug():

   8  /shared/poetry/lib/poetry/_vendor/py3.7/clikit/api/command/command.py:163 in _do_handle
       161│         if self._dispatcher and self._dispatcher.has_listeners(PRE_HANDLE):
       162│             event = PreHandleEvent(args, io, self)
     → 163│             self._dispatcher.dispatch(PRE_HANDLE, event)
       164│ 
       165│             if event.is_handled():

   7  /shared/poetry/lib/poetry/_vendor/py3.7/clikit/api/event/event_dispatcher.py:22 in dispatch
        20│ 
        21│         if listeners:
     →  22│             self._do_dispatch(listeners, event_name, event)
        23│ 
        24│         return event

   6  /shared/poetry/lib/poetry/_vendor/py3.7/clikit/api/event/event_dispatcher.py:89 in _do_dispatch
        87│                 break
        88│ 
     →  89│             listener(event, event_name, self)
        90│ 
        91│     def _sort_listeners(self, event_name):  # type: (str) -> None

   5  /shared/poetry/lib/poetry/console/config/application_config.py:147 in set_installer
       145│             poetry.locker,
       146│             poetry.pool,
     → 147│             poetry.config,
       148│         )
       149│         installer.use_executor(poetry.config.get("experimental.new-installer", False))

   4  /shared/poetry/lib/poetry/installation/installer.py:65 in __init__
        63│         self._installer = self._get_installer()
        64│         if installed is None:
     →  65│             installed = self._get_installed()
        66│ 
        67│         self._installed_repository = installed

   3  /shared/poetry/lib/poetry/installation/installer.py:561 in _get_installed
       559│ 
       560│     def _get_installed(self):  # type: () -> InstalledRepository
     → 561│         return InstalledRepository.load(self._env)
       562│ 

   2  /shared/poetry/lib/poetry/repositories/installed_repository.py:118 in load
       116│                 path = Path(str(distribution._path))
       117│                 version = distribution.metadata["version"]
     → 118│                 package = Package(name, version, version)
       119│                 package.description = distribution.metadata.get("summary", "")
       120│ 

   1  /shared/poetry/lib/poetry/_vendor/py3.7/poetry/core/packages/package.py:61 in __init__
        59│ 
        60│         if not isinstance(version, Version):
     →  61│             self._version = Version.parse(version)
        62│             self._pretty_version = pretty_version or version
        63│         else:

  ParseVersionError

  Unable to parse "R2020b".

  at /shared/poetry/lib/poetry/_vendor/py3.7/poetry/core/semver/version.py:206 in parse
      202│         except TypeError:
      203│             match = None
      204│ 
      205│         if match is None:
    → 206│             raise ParseVersionError('Unable to parse "{}".'.format(text))
      207│ 
      208│         text = text.rstrip(".")
      209│ 
      210│         major = int(match.group(1))

pip list ouput:

Package               Version
--------------------- -------
matlabengineforpython R2020b
pip                   20.3
setuptools            41.2.0

pyproject.toml file:

[tool.poetry]
name = "foo_bar"
version = "0.1.0"
description = "foobar"
authors = ["me <[email protected]>"]
license = "MIT"

[tool.poetry.dependencies]
python = "^3.8"

[tool.poetry.dev-dependencies]

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

[randallpittman]

I think in my perfect world there would be a way to have some dep. specs that would be strictly for checking only (installed by some other manner) that would allow arbitrary version strings, or even regex versions.

That said, this particular issue is strictly a problem with parsing the currently installed packages.

[thaeber]

The same here for:

Matlab Version R2019b
Poetry 1.0.10

Is there any workaround?

[finswimmer]

Hello everyone,

neither R2019b nor no-git-or-release-version are valid versions according to PEP 440. Even setuptools is giving you a warning about this, when you are trying to create a package with such a version:

UserWarning: The version specified ('R2020b') is an invalid version, this may not work as expected with newer versions of setuptools, pip, and PyPI. Please see PEP 440 for more details.

poetry is already strict about it. Even if you haven't installed such a broken package with poetry, poetry tries to read the version for this package as it must know which packages are already installed in the venv to properly resolve dependencies.

TL;DR: The version identifiers are invalid and must be fixed by the package maintainers.

fin swimmer

[randallpittman]

@finswimmer I respect your decision. I doubt we're ever going to get Mathworks to change that, so I guess we just can't use Poetry in environments that integrate MATLAB and Python. =( All the same I'll open an issue with Mathworks.

[thaeber]

Adhering to a strict versioning scheme is certainly the most sensible approach. @randallpittman Until Mathworks changes its distribution, you can use the following worakaround. Sorry I didn't think of it sooner.

1. Copy all files and sub-directories from the directory matlabroot/extern/engines/python to a location of your choice.
2. Open the file setup.py in an editor and change the following lines:
   
   
   
   • Point the _bin_dir variable in the _generate_arch_file function to the bin folder in the Matlab root directory. The _bin_dir string must end with a path separator.
   
   
   • Change the version string in the setup call to be compatible with semantic versioning.
   
   
3. Install the matlab engine: python .\setup.py build install

[bdc34]

I don't expect to change anyone's minds but =no-git-or-release-version= was specifically chosen for our use case so that it does not conform to PEP 440.

[sinoroc]

Just skimmed the thread, so I might be off...

Is that a case where "arbitrary equality" could help?

[randallpittman]

I did open an issue with Mathworks and they were receptive to the PEP 440 info and said the developers will "consider implementing a fix for it in a future release of MATLAB".

[sinoroc]

@randallpittman Is there a URL link to the Matlab issue that you filed?

[randallpittman]

@sinoroc Sorry, no, it was not in a public forum, more like a form + email exchange. I thought there was a publicly-visible place for submitting issues but I couldn't find it.