Poetry: Can't use private repository in `poetry add`

[ojii]

Poetry version: 0.10.3
Python Version: 3.6.5

(Note: I redacted URL of my private repository)

pyproject.toml:

[tool.poetry]
name = "project"
version = "0.1.0"
description = ""
authors = []

[[tool.poetry.source]]
name = "private"
url = "<url>"

[tool.poetry.dependencies]
python = "^3.6"

[tool.poetry.dev-dependencies]
pytest = "^3.5"

I ran the following two commands:

• poetry config repositories.private <url>
• poetry config http-basic.private <username>

Checking the config files on disk, the values are correct.

When I run poetry search <private package> or poetry add <private package> poetry cannot find the package. Here's the output of poetry -vvv add:

poetry -vvv add <private package>

[ValueError]       
Package [<private package>] not found.  

Exception trace:
 /usr/local/lib/python3.6/site-packages/poetry/_vendor/cleo/application.py in run() at line 94
   status_code = self.do_run(input_, output_)
 /usr/local/lib/python3.6/site-packages/poetry/console/application.py in do_run() at line 87
   return super(Application, self).do_run(i, o)
 /usr/local/lib/python3.6/site-packages/poetry/_vendor/cleo/application.py in do_run() at line 197
   status_code = command.run(input_, output_)
 /usr/local/lib/python3.6/site-packages/poetry/console/commands/command.py in run() at line 72
   return super(BaseCommand, self).run(i, o)
 /usr/local/lib/python3.6/site-packages/poetry/_vendor/cleo/commands/base_command.py in run() at line 146
   status_code = self.execute(input_, output_)
 /usr/local/lib/python3.6/site-packages/poetry/_vendor/cleo/commands/command.py in execute() at line 107
   return self.handle()
 /usr/local/lib/python3.6/site-packages/poetry/console/commands/add.py in handle() at line 65
   packages, allow_prereleases=self.option("allow-prereleases")
 /usr/local/lib/python3.6/site-packages/poetry/console/commands/init.py in _determine_requirements() at line 215
   requirement["name"], allow_prereleases=allow_prereleases
 /usr/local/lib/python3.6/site-packages/poetry/console/commands/init.py in _find_best_version_for_package() at line 245
   name, required_version, allow_prereleases=allow_prereleases
 /usr/local/lib/python3.6/site-packages/poetry/version/version_selector.py in find_best_candidate() at line 28
   package_name, constraint, allow_prereleases=allow_prereleases
 /usr/local/lib/python3.6/site-packages/poetry/repositories/pool.py in find_packages() at line 83
   name, constraint, extras=extras, allow_prereleases=allow_prereleases
 /usr/local/lib/python3.6/site-packages/poetry/repositories/pypi_repository.py in find_packages() at line 90
   info = self.get_package_info(name)
 /usr/local/lib/python3.6/site-packages/poetry/repositories/pypi_repository.py in get_package_info() at line 225
   name, lambda: self._get_package_info(name)
 /usr/local/lib/python3.6/site-packages/poetry/_vendor/cachy/repository.py in remember_forever() at line 174
   val = value(callback)
 /usr/local/lib/python3.6/site-packages/poetry/_vendor/cachy/helpers.py in value() at line 6
   return val()
 /usr/local/lib/python3.6/site-packages/poetry/repositories/pypi_repository.py in <lambda>() at line 225
   name, lambda: self._get_package_info(name)
 /usr/local/lib/python3.6/site-packages/poetry/repositories/pypi_repository.py in _get_package_info() at line 231
   raise ValueError("Package [{}] not found.".format(name))

add [-D|--dev] [--git GIT] [--path PATH] [-E|--extras EXTRAS] [--optional] [--python PYTHON] [--platform PLATFORM] [--allow-prereleases] [--dry-run] [--] <name> (<name>)...

We use pypicloud for our private repository.

[xsduan]

have you tried changing if you're using the /simple/ index (or some other equivalent), and then toggling the trailing slash? I've found that it's pretty finicky about which url goes where, and after running into your problem it just...fixed itself after a few different urls. (currently using /simple with no slash)

It'd be nice to have some attempted diagnostics (like wrong format vs 404/403 or something).

[sdispater]

@ojii You might want to try the latest preview release (0.11.0a4), it should be fixed.

[ojii]

have you tried changing if you're using the /simple/ index (or some other equivalent), and then toggling the trailing slash?

I'm using the exact same URL that works fine with pip3.

@ojii You might want to try the latest preview release (0.11.0a4), it should be fixed.
@sdispater I've upgraded to Poetry 0.11.0-alpha.4 but still have the same issue.

[Gr1N]

Got same issue. Private repository configured step by step like described in https://poetry.eustace.io/docs/repositories/.

Updating dependencies
Resolving dependencies...
   1: fact: my-package is 0.1.0-alpha.0
   1: derived: my-package
   1: fact: my-package depends on my-private (*)
   1: selecting my-package (0.1.0-alpha.0)
   1: derived: my-private (*)
privatepypi: 0 packages found for my-private *
   1: fact: my-private doesn't exist
   1: conflict: my-private doesn't exist
   1: ! my-private (*) is satisfied by my-private (*)
   1: ! which is caused by "my-package depends on my-private (*)"
   1: ! thus: version solving failed
   1: Version solving took 0.440 seconds.
   1: Tried 1 solutions.

[SolverProblemError]
Because my-package depends on my-private (*) which doesn't exist, version solving failed.

Exception trace:
 /Users/ng/.pyenv/versions/3.6.5/lib/python3.6/site-packages/cleo/application.py in run() at line 94
   status_code = self.do_run(input_, output_)
 /Users/ng/.pyenv/versions/3.6.5/lib/python3.6/site-packages/poetry/console/application.py in do_run() at line 88
   return super(Application, self).do_run(i, o)
 /Users/ng/.pyenv/versions/3.6.5/lib/python3.6/site-packages/cleo/application.py in do_run() at line 197
   status_code = command.run(input_, output_)
 /Users/ng/.pyenv/versions/3.6.5/lib/python3.6/site-packages/poetry/console/commands/command.py in run() at line 72
   return super(BaseCommand, self).run(i, o)
 /Users/ng/.pyenv/versions/3.6.5/lib/python3.6/site-packages/cleo/commands/base_command.py in run() at line 146
   status_code = self.execute(input_, output_)
 /Users/ng/.pyenv/versions/3.6.5/lib/python3.6/site-packages/cleo/commands/command.py in execute() at line 107
   return self.handle()
 /Users/ng/.pyenv/versions/3.6.5/lib/python3.6/site-packages/poetry/console/commands/install.py in handle() at line 50
   return installer.run()
 /Users/ng/.pyenv/versions/3.6.5/lib/python3.6/site-packages/poetry/installation/installer.py in run() at line 76
   self._do_install(local_repo)
 /Users/ng/.pyenv/versions/3.6.5/lib/python3.6/site-packages/poetry/installation/installer.py in _do_install() at line 158
   ops = solver.solve(use_latest=self._whitelist)
 /Users/ng/.pyenv/versions/3.6.5/lib/python3.6/site-packages/poetry/puzzle/solver.py in solve() at line 31
   packages, depths = self._solve(use_latest=use_latest)
 /Users/ng/.pyenv/versions/3.6.5/lib/python3.6/site-packages/poetry/puzzle/solver.py in _solve() at line 130
   raise SolverProblemError(e)

install [--no-dev] [--dry-run] [-E|--extras EXTRAS] [--develop DEVELOP]

Interesting fact if in pyproject.toml define source with basic auth then everything work as expected (but I don't want to store credentials in repo).

[[tool.poetry.source]]
name = "privatepypi"
url = "https://username@password:party.jfrog.io/party/api/pypi/pypi/simple/"

As PyPI server we use Artifactory by JFrog.

And poetry version is 0.11.2.

[Gr1N]

So as I understand from code base when poetry tries to access private repository for search or install command authentication credentials which configured and stored in auth.toml not used.

And for example in case of publish command poetry reads auth.toml and uses credentials from it https://github.com/sdispater/poetry/blob/master/poetry/masonry/publishing/publisher.py#L66.

[Gr1N]

@sdispater am I right?

[ojii]

search (and probably install) seem to completely ignore custom repositories.

[Gr1N]

Can it be fixed soon?

[Gr1N]

I've created a PR #307 with fix.

[lepisma]

I am facing a similar issue (not able to add package from a private repository) though its not related to authentication.

• OS version and name: 4.17.5-1-MANJARO
• Poetry version: Poetry 0.11.2
• Link of a Gist with the contents of your pyproject.toml file: toml

Trying to add -vvv gives this:

[IndexError]  
list index out of range    

Exception trace:
 /home/lepisma/.miniconda/lib/python3.6/site-packages/poetry/_vendor/cleo/application.py in run() at line 94
   status_code = self.do_run(input_, output_)
 /home/lepisma/.miniconda/lib/python3.6/site-packages/poetry/console/application.py in do_run() at line 88
   return super(Application, self).do_run(i, o)
 /home/lepisma/.miniconda/lib/python3.6/site-packages/poetry/_vendor/cleo/application.py in do_run() at line 197
   status_code = command.run(input_, output_)
 /home/lepisma/.miniconda/lib/python3.6/site-packages/poetry/console/commands/command.py in run() at line 72
   return super(BaseCommand, self).run(i, o)
 /home/lepisma/.miniconda/lib/python3.6/site-packages/poetry/_vendor/cleo/commands/base_command.py in run() at line 146
   status_code = self.execute(input_, output_)
 /home/lepisma/.miniconda/lib/python3.6/site-packages/poetry/_vendor/cleo/commands/command.py in execute() at line 107
   return self.handle()
 /home/lepisma/.miniconda/lib/python3.6/site-packages/poetry/console/commands/add.py in handle() at line 137
   status = installer.run()
 /home/lepisma/.miniconda/lib/python3.6/site-packages/poetry/installation/installer.py in run() at line 76
   self._do_install(local_repo)
 /home/lepisma/.miniconda/lib/python3.6/site-packages/poetry/installation/installer.py in _do_install() at line 158
   ops = solver.solve(use_latest=self._whitelist)
 /home/lepisma/.miniconda/lib/python3.6/site-packages/poetry/puzzle/solver.py in solve() at line 31
   packages, depths = self._solve(use_latest=use_latest)
 /home/lepisma/.miniconda/lib/python3.6/site-packages/poetry/puzzle/solver.py in _solve() at line 121
   self._package, self._provider, locked=locked, use_latest=use_latest
 /home/lepisma/.miniconda/lib/python3.6/site-packages/poetry/mixology/__init__.py in resolve_version() at line 7
   return solver.solve()
 /home/lepisma/.miniconda/lib/python3.6/site-packages/poetry/mixology/version_solver.py in solve() at line 79
   next = self._choose_package_version()
 /home/lepisma/.miniconda/lib/python3.6/site-packages/poetry/mixology/version_solver.py in _choose_package_version() at line 378
   version = self._provider.complete_package(version)
 /home/lepisma/.miniconda/lib/python3.6/site-packages/poetry/puzzle/provider.py in complete_package() at line 323
   package.name, package.version.text, extras=package.requires_extras
 /home/lepisma/.miniconda/lib/python3.6/site-packages/poetry/repositories/pool.py in package() at line 67
   package = repository.package(name, version, extras=extras)
 /home/lepisma/.miniconda/lib/python3.6/site-packages/poetry/repositories/legacy_repository.py in package() at line 235
   release_info = self.get_release_info(name, version)
 /home/lepisma/.miniconda/lib/python3.6/site-packages/poetry/repositories/pypi_repository.py in get_release_info() at line 239
   "{}:{}".format(name, version), lambda: self._get_release_info(name, version)
 /home/lepisma/.miniconda/lib/python3.6/site-packages/poetry/_vendor/cachy/repository.py in remember_forever() at line 174
   val = value(callback)
 /home/lepisma/.miniconda/lib/python3.6/site-packages/poetry/_vendor/cachy/helpers.py in value() at line 6
   return val()
 /home/lepisma/.miniconda/lib/python3.6/site-packages/poetry/repositories/pypi_repository.py in <lambda>() at line 239
   "{}:{}".format(name, version), lambda: self._get_release_info(name, version)
 /home/lepisma/.miniconda/lib/python3.6/site-packages/poetry/repositories/legacy_repository.py in _get_release_info() at line 297
   default_link = links[0]

add [-D|--dev] [--git GIT] [--path PATH] [-E|--extras EXTRAS] [--optional] [--python PYTHON] [--platform PLATFORM] [--allow-prereleases] [--dry-run] [--] <name> (<name>)...

I tried to look into the problem and found that the IndexError gets thrown when it looks for pytest in my private repo (which is a LegacyRepository) while looping over the pool. I assume when we don't find anything in a repository from the pool, we skip that repo and continue. This skipping is not capturing the IndexError (happening here).

As of now, I am adding a catch block for IndexError in pool.py and its working fine. Though @sdispater might want to take a look.

[Gr1N]

Any news on this? @sdispater?

[greatvovan]

Why is this issue closed?

poetry add still does not work for packages in private repos. The repo is configured:

poetry  config --list
settings.virtualenvs.create = true
settings.virtualenvs.in-project = false
settings.virtualenvs.path = "/Users/xxx/Library/Caches/pypoetry/virtualenvs"
repositories.my.url = "http://pypi.my.domain.net/simple/"

$ poetry add mypackage==1.3.2

[PackageNotFound]  
Package [mypackage] not found.  

See also verbose output.

There is no authentication in my repo.

The path from repositories.my.url works well with pip install.

I was able to overcome this by adding the package manually into pyproject.toml as well as this section:

[[tool.poetry.source]]
name = "my"
url = "http://pypi.my.domain.net/simple/"

Then I deleted poetry.lock and issued poetry install. Now I got it in the 'lock' file, but this seems too cumbersome.

@sdispater should I open a new issue?

P.S. My username, package name and URL are consistently changed into other values.

$ poetry --version
Poetry 0.12.17

[greatvovan]

With version 1.0.0b9 it shows this:

$ poetry add -vvv mypackage==1.3.2
Using virtualenv: /Users/xxx/Library/Caches/pypoetry/virtualenvs/peapi-7kyxBY5Q-py3.8
PyPI: 0 packages found for mypackage 1.3.2

[ValueError]
Could not find a matching version of package mypackage

Traceback (most recent call last):
  File "/Users/xxx/.poetry/lib/poetry/_vendor/py3.8/clikit/console_application.py", line 131, in run
    status_code = command.handle(parsed_args, io)
  File "/Users/xxx/.poetry/lib/poetry/_vendor/py3.8/clikit/api/command/command.py", line 120, in handle
    status_code = self._do_handle(args, io)
  File "/Users/xxx/.poetry/lib/poetry/_vendor/py3.8/clikit/api/command/command.py", line 171, in _do_handle
    return getattr(handler, handler_method)(args, io, self)
  File "/Users/xxx/.poetry/lib/poetry/_vendor/py3.8/cleo/commands/command.py", line 92, in wrap_handle
    return self.handle()
  File "/Users/xxx/.poetry/lib/poetry/console/commands/add.py", line 88, in handle
    requirements = self._determine_requirements(
  File "/Users/xxx/.poetry/lib/poetry/console/commands/init.py", line 314, in _determine_requirements
    name, _ = self._find_best_version_for_package(
  File "/Users/xxx/.poetry/lib/poetry/console/commands/init.py", line 338, in _find_best_version_for_package
    raise ValueError(

If I add [[tool.poetry.source]] in poetry.toml poetry add works. But isn't it supposed to work right away with all the configured repositories?

Edit: I don't think so because:

When you specify a dependency in pyproject.toml, Poetry first take the name of the package that you have requested and searches for it in any repository you have registered using the repositories key. If you have not registered any extra repositories, or it does not find a package with that name in the repositories you have specified, it falls back on PyPI.
(https://python-poetry.org/docs/basic-usage/#version-constraints)

[szafranek]

This issue should be reopened. The problem still happens in poetry 1.0.5.

[qiuwei]

I have the same problem poetry 1.0.5.

[LSaldyt]

EDIT: I've fixed this, by adding /simple/ to the end of my config repo, and now the debug log actually checks my custom repo. A warning about a bad URL would be very helpful, though.

Same issue in 1.0.5. Debug output indicates poetry doesn't even look at the custom repo:

!poetry config --list
cache-dir = "/Users/X/Library/Caches/pypoetry"
repositories.X.url = "https://sonatype.services.X.com/repository/pypi-internal/"
virtualenvs.create = true
virtualenvs.in-project = false
virtualenvs.path = "{cache-dir}/virtualenvs"  # /Users/X/Library/Caches/pypoetry/virtualenvs
!poetry add my-package -vvv
Using virtualenv: /Users/X/Library/Caches/pypoetry/virtualenvs/X-BmgmJgwc-py3.7
PyPI: No packages found for my-package *

[ValueError]
Could not find a matching version of package my-package

Traceback (most recent call last):
  File "/Users/X/Library/Python/3.7/lib/python/site-packages/clikit/console_application.py", line 131, in run
    status_code = command.handle(parsed_args, io)
  File "/Users/X/Library/Python/3.7/lib/python/site-packages/clikit/api/command/command.py", line 120, in handle
    status_code = self._do_handle(args, io)
  File "/Users/X/Library/Python/3.7/lib/python/site-packages/clikit/api/command/command.py", line 171, in _do_handle
    return getattr(handler, handler_method)(args, io, self)
  File "/Users/X/Library/Python/3.7/lib/python/site-packages/cleo/commands/command.py", line 92, in wrap_handle
    return self.handle()
  File "/Users/X/Library/Python/3.7/lib/python/site-packages/poetry/console/commands/add.py", line 89, in handle
    packages, allow_prereleases=self.option('allow-prereleases')
  File "/Users/X/Library/Python/3.7/lib/python/site-packages/poetry/console/commands/init.py", line 302, in _determine_requirements
    requirement['name'], allow_prereleases=allow_prereleases
  File "/Users/X/Library/Python/3.7/lib/python/site-packages/poetry/console/commands/init.py", line 338, in _find_best_version_for_package
    'Could not find a matching version of package {}'.format(name)

!poetry --version
Poetry version 1.0.5

And here's the project config

[tool.poetry]
name = "X"
version = "0.1.0"
description = ""
authors = ["X"]

[[tool.poetry.source]]
name = 'X'
url  = 'https://sonatype.services.X.com/repository/pypi-internal/'
secondary = true

[tool.poetry.dependencies]
python = "^3.7"
Django = "<3.0"

[tool.poetry.dev-dependencies]

[build-system]
requires = ["poetry>=0.12"]
build-backend = "poetry.masonry.api"

The weird part is that I had poetry downloading dependencies from our custom PyPI repository, but once I uploaded a package myself, it doesn't like it.

[kmcminn]

still a source of confusion not mentioned in any docs I could find. @LSaldyt fix worked though.