Poetry: Poetry 1.1.0 Unable to Resolve Private Packages from pre-1.1.0 poetry.lock

[xM8WVqaG]

• [x] I am on the latest Poetry version.
• [x] I have searched the issues of this repo and believe that this is not a duplicate.
• [x] If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: Ubuntu 18.04 bionic
• Poetry version: Poetry version 1.0.10
• Link of a Gist with the contents of your pyproject.toml file: N/A

Issue

Starting with 1.1.0 of Poetry poetry install is failing for some of our private packages:

$ poetry install --no-interaction --no-ansi
Creating virtualenv project-name-nxCwz_m9-py3.8 in /builds/REDACTED/.cache/poetry/virtualenvs
Installing dependencies from lock file
Warning: The lock file is not up to date with the latest changes in pyproject.toml. You may be getting outdated dependencies. Run update to update them.
Package operations: 85 installs, 0 updates, 0 removals
  • Installing six (1.15.0)
<SNIP>
  • Installing private-package-1 (0.2.1)
  RuntimeError
  Unable to find installation candidates for private-package-1 (0.2.1)
  at /usr/local/lib/python3.8/site-packages/poetry/installation/chooser.py:72 in choose_for
       68│ 
       69│             links.append(link)
       70│ 
       71│         if not links:
    →  72│             raise RuntimeError(
       73│                 "Unable to find installation candidates for {}".format(package)
       74│             )
       75│ 
       76│         # Get the best link
  • Installing flake8-print (3.1.4)
  • Installing hypothesis (5.26.0)
  • Installing isort (4.3.21)
  • Installing jsonschema (3.2.0)
  • Installing mock (4.0.2)
  • Installing mypy (0.780)
  • Installing networkx (2.4)
  • Installing pytest-cov (2.10.1)
  • Installing private-package-2 (0.1.6)
  RuntimeError
  Unable to find installation candidates for private-package-2 (0.1.6)
  at /usr/local/lib/python3.8/site-packages/poetry/installation/chooser.py:72 in choose_for
       68│ 
       69│             links.append(link)
       70│ 
       71│         if not links:
    →  72│             raise RuntimeError(
       73│                 "Unable to find installation candidates for {}".format(package)
       74│             )
       75│ 
       76│         # Get the best link
  • Installing pyyaml (5.3.1)
  • Installing seed-isort-config (2.2.0)
  • Installing sentry-sdk (0.17.3)
  • Installing tabulate (0.8.7)
  • Installing taskipy (1.3.0)
  • Installing vulture (1.6)
  • Installing xlrd (1.2.0)

Note, it returns non-zero but other than the error in the middle of the install it doesn't actually have a specific message.

After playing around a bit with poetry update I discovered that adding type = "legacy" to the poetry.lock for those packages fixes this.

diff --git a/project-name/poetry.lock b/project-name/poetry.lock
index ece9221..83d5f84 100644
--- a/project-name/poetry.lock
+++ b/project-name/poetry.lock
@@ -446,6 +446,7 @@ snappy = ["python-snappy (>=0.5.4,<0.6.0)"]

 [package.source]
 reference = "gitlabprivate"
+type = "legacy"
 url = "https://gitlab.com/api/v4/projects/REDACTED/packages/pypi/simple"

 [[package]]
@@ -921,6 +922,7 @@ python-json-logger = ">=0.1.11,<0.2.0"

 [package.source]
 reference = "gitlabprivate"
+type = "legacy"
 url = "https://gitlab.com/api/v4/projects/REDACTED/packages/pypi/simple"

 [[package]]

Based on this part of the blogpost[1] that suggests 1.1.0 should be able to read previous versions but not vice versa, this is likely unintended.

[1] https://python-poetry.org/blog/announcing-poetry-1-1-0.html#lock-file-format-changes

[dvf]

Can confirm. Legacy users need to run poetry update to add these keys to the older sources.

[lucinvitae]

Our team just hit this issue, which breaks docker builds where the poetry version is not pinned. A short-term workaround that fixed our builds is to pin the poetry version (pip install poetry==1.0.5) prior to running poetry install.

[itai12312]

Our team just hit this issue, which breaks docker builds where the poetry version is not pinned. A short-term workaround that fixed our builds is to pin the poetry version (pip install poetry==1.0.5) prior to running poetry install.

Thanks! I was also struggling with this issue today(remote run fails).

[abn]

@xM8WVqaG poetry lock should resolve your issue, this is because the lock file now correctly adds package source information.

I have also added #3034 to add a feature that adds poetry lock --no-update that will allow users to refresh the lock file without updating pinned dependencies. Feel free to test that branch.

[xM8WVqaG]

@abn Thanks for the follow up to this but I have attempted poetry lock --no-update as suggested using poetry from the master branch and it didn't add the necessary information to poetry.lock to fix the poetry install.

All poetry lock --no-update did to the relevant section of of poetry.lock is:

 [package.source]
-reference = "gitlabprivate"
 url = "https://gitlab.com/api/v4/projects/REDACTED/packages/pypi/simple"
+reference = "gitlabprivate"

It still fails with the same error:

$ poetry install

<snip>

  • Installing private-package-2 (0.1.6): Failed

  RuntimeError

  Unable to find installation candidates for private-package-2 (0.1.6)

  at /REDACTED/venv/lib/python3.8/site-packages/poetry/installation/chooser.py:72 in choose_for
       68│ 
       69│             links.append(link)
       70│ 
       71│         if not links:
    →  72│             raise RuntimeError(
       73│                 "Unable to find installation candidates for {}".format(package)
       74│             )
       75│ 
       76│         # Get the best link

[xM8WVqaG]

I can confirm that MR #3051 (from #3048) fixes the lock --no-update failing to update the source block.

 [package.source]
-reference = "gitlabprivate"
+type = "legacy"
 url = "https://gitlab.com/api/v4/projects/REDACTED/packages/pypi/simple"
+reference = "gitlabprivate"

[ahobsonsayers]

I'm still having this issue on and off. I have tried updating my lock file using:

• poetry lock
• poetry lock --no-update
• poetry update

I can confirm that type = "legacy" exists in poetry.lock, however when I run poetry install with a clean environment i get the error:

$ poetry install

<snip>

  • Installing XXX (1.0.0): Failed

  RuntimeError

  Unable to find installation candidates for XXX (1.0.0)

  at progs/poetry/lib/poetry/installation/chooser.py:72 in choose_for
       68│ 
       69│             links.append(link)
       70│ 
       71│         if not links:
    →  72│             raise RuntimeError(
       73│                 "Unable to find installation candidates for {}".format(package)
       74│             )
       75│ 
       76│         # Get the best link

If i rerun poetry install a second time, it sometimes will work fine. Sometimes a third run is required. However it always fails for the first run, but will pass at some point on subsequent runs. THis is fine on my local machine but is causing havoc on my CI runs and install always fails.

Any ideas as to why this would happen?