Poetry: source resolving issue in pyproject.toml

[avivex1000]

So I have a pyproject.toml where i've defined two different sources for two private repo based dependencies in gitlab like so:

[tool.poetry]
name = "my_service"
version = "0.1.0"
description = ""
authors = [""]

[[tool.poetry.source]]
name = "package_1_source"
url = "https://gitlab.com/api/v4/projects/project_id_1/packages/pypi/simple/"

[[tool.poetry.source]]
name = "package_2_source"
url = "https://gitlab.com/api/v4/projects/project_id_2/packages/pypi/simple"

[tool.poetry.dependencies]
python = "^3.8"
package_1 = "1.0.0"
package_2 = "2.0.0"

And I keep getting the following error:

>>>poetry update
Updating dependencies
Resolving dependencies... (0.4s)

[SolverProblemError]
Because my_service depends on package_1 (1.0.0) which doesn't match any versions, version solving failed.

I've also tried explicitly pointing dependencies to their relevant sources.

[tool.poetry.dependencies]
package_1 = {version="1.0.0", source="package_1_source"}
package_2 = {version="2.0.0", source="package_2_source"}

I've properly set the http token in the poetry config using poetry config http-basic.gitlab __token__ <my_gitlab_token>,
and when using only one source there doesn't seem to be an issue.
Also, when changing the order of source declaration the missing dependencies flip, which makes me assume that source is being somehow overridden by the second declaration instead of being updated with both 😞

Another curious behavior is that even with one dependency and one source, poetry insisted that the source name and URL had some correlation (like sharing the repo name? I'm not sure but here is an example):
this works:

[[tool.poetry.source]]
name = "gitlab"
url = "https://gitlab.com/api/v4/projects/project_id/packages/pypi/simple/"

but this doesn't: (even when I specifically point to It from the dependency section)

[[tool.poetry.source]]
name = "any_other_name"
url = "https://gitlab.com/api/v4/projects/project_id/packages/pypi/simple/"

I'm running the latest poetry 1.0.10 on a mac Catalina-os version 10.15.3

I've gone over the following issue and documentation but couldn't find a solution for this:
https://python-poetry.org/docs/repositories/#install-dependencies-from-a-private-repository
https://github.com/python-poetry/poetry/issues/14
https://github.com/python-poetry/poetry/issues/1547

Any help will be much appreciated
BTW, this is used for local and CI and both fail in the same manner so far...

Thank you!

[abn]

Can you try the pre-release?

[avivex1000]

Thanks for responding!
So locally this seems to have solved the issue, (though the name referencing doesn't work, when I removed the explicit naming and called everything gitlab it worked) using this toml:

[[tool.poetry.source]]
name = "gitlab"
url = "https://gitlab.com/api/v4/projects/project_id_1/packages/pypi/simple/"

[[tool.poetry.source]]
name = "gitlab"
url = "https://gitlab.com/api/v4/projects/project_id_2/packages/pypi/simple"

[tool.poetry.dependencies]
python = "^3.8"
package_1 = "1.0.0"
package_2 = "2.0.0"

once I change the name gitlab it breaks.

On my Ci I'm getting a different error :

• Installing package_1 (1.0.0)
  RuntimeError
  Unable to find installation candidates for package_1-1.0.0
  at ~/.poetry/lib/poetry/installation/chooser.py:71 in choose_for
       67│ 
       68│             links.append(link)
       69│ 
       70│         if not links:
    →  71│             raise RuntimeError(
       72│                 "Unable to find installation candidates for {}".format(package)
       73│             )
       74│ 
       75│         # Get the best link

btw the next log below it seems to show that package_2 was installed with no issues:

• Installing package_2 (2.0.0)
• Installing pytest-asyncio (0.14.0)
• Installing vulture (1.6)
ERROR: Job failed: command terminated with exit code 1

so I guess similar to my original issue.

BTW, when is the next release planned and how stable are the pre-release versions?

[abn]

That does not sound like a good solution :) Do you have gitlab repos I can use to try reproduce the issue?

As for next release, we are planning atleast one more beta release prior to the stable realease of 1.1.0. We want to target 1.1.0 before the end of next month. But all that depends on issues that creep up.

[avivex1000]

Unfortunately, this is all happening on private projects at the company I'm working at so I can't give you access to anything I'm working on :(
Is there any other information I could give to help understand this?

And thanks for the answer regarding the upcoming releases :)

[abn]

@avivex1000 even if you can setup the scenario on 2 sample public packages that would help. I just do not have the time right now to try reproduce the issue so I can debug it.

Note that there are couple of issues in the issue tracker where authentication fails silently. This might be cause of your CI error (ie. credentials not available).

If I were to guess, the relevant code for this might be in https://github.com/python-poetry/poetry/tree/master/poetry/repositories. Particularly, with regards to how packages are searched for in a pool or even how the repository pools are created (ie. discovering source configurations from pyproject.toml). You are welcome to try debug the issue and post any insights.

Feel free to join our discord server too, might be easier debug issues there than over github issue comments. :)

[avivex1000]

So looks like I've missed something a bit silly 😅
when authenticating using poetry config http-basic.gitlab __token__ <my_gitlab_token> i didn't notice gitlab is the source name and not the provider or something...
so I should have done poetry config http-basic.<source_name> __token__ <my_gitlab_token> for each of my sources.
After doing so it seems to work fine :)
Sorry for the false alarm, hope this will help anyone else encountering this...
I'll be closing this issue for now.
Thanks for the help!