Poetry: Allow to specify the source when adding a package

[k4nar]

• [x] I have searched the issues of this repo and believe that this is not a duplicate.
• [x] I have searched the documentation and believe that my question is not covered.

Feature Request

Since Poetry 1.0 we can specify the source of package in the pyproject.toml, in order to use another index than PyPI. This is very convenient, but there is no way to specify this when using the poetry add command.

This is especially inconvenient when you're trying to update an existing private dependency with poetry add, as it will erase its source.

My suggestion would be to add a --source option to poetry add, taking the name of a source defined in the pyproject.toml. I don't think adding a package from a source not defined previously should be accepted.

[k4nar]

I'm working on a PR, the solution seems to be very easy in the end.

[ns-admetrics]

It looks like https://github.com/python-poetry/poetry/pull/1912 was merged but then later lost again in https://github.com/python-poetry/poetry/commit/9ab25813f9de1019910549dc92d30bbad427a838. Was this by accident?

In light of GitLab's new feature (git/source repository specific package repositories), this would be a rather useful feature.

[k4nar]

I think it was intentionally delayed for the version 1.1. Apparently the commit is still present in the branch 1.1.0b2 so I'm hopeful :) .

[abn]

This already added for 1.1.0 and available in the current pre-release.

[GuillaumeLegoy]

Hi, it looks like this was never implemented or I'm maybe missing something? We have a use case where a custom package bears the same name as a public PyPi package and specifying the source when running poetry add would be great.

[sinoroc]

@GuillaumeLegoy You probably should use something like _devpi_ and configure it correctly so that private packages are preferred over public ones (_PyPI_). This is the only correct way of dealing with such cases in the end. A lot of background information in this discussion if you are interested.

[GuillaumeLegoy]

Thanks will take a look. However it doesn't answer the above question as it looks like it was implemented but somehow never made it to live?
Also I've experienced with the secondary vs default settings of poetry sources and ran into errors as well. If we could simply specify a private repository as default and PyPi as fallback that would solve it as well (first look into private repo, if not found revert to PyPi).

[sinoroc]

However it doesn't answer the above question as it looks like it was implemented but somehow never made it to live?

I have _poetry 1.1.4_ and I can see the --source flag:

$ poetry add --help
USAGE
  poetry add [-D] [-E <...>] [--optional] [--python <...>] [--platform <...>] [--source <...>] [--allow-
             prereleases] [--dry-run] [--lock] <name1> ... [<nameN>]

ARGUMENTS
  <name>                 The packages to add.

OPTIONS
  -D (--dev)             Add as a development dependency.
  -E (--extras)          Extras to activate for the dependency. (multiple values allowed)
  --optional             Add as an optional dependency.
  --python               Python version for which the dependency must be installed.
  --platform             Platforms for which the dependency must be installed.
  --source               Name of the source to use to install the package.
  --allow-prereleases    Accept prereleases.
  --dry-run              Output the operations but do not execute anything (implicitly enables
                         --verbose).
  --lock                 Do not perform operations (only update the lockfile).

[...]

If we could simply specify a private repository as default and PyPi as fallback that would solve it as well (first look into private repo, if not found revert to PyPi).

In Python packaging world, this is not a real thing as far as I understood. There can not be priorities between indexes. _Poetry_ might offer this, but in general the packaging ecosystem was not built to allow this (for example _pip_ does not allow ordering the indexes), so I would not recommend relying on this too much. See also more details in https://github.com/python-poetry/poetry/issues/3269. From my point of view a well configured private index (_devpi_ or other), is a much better solution that will work with any tool (poetry, pip, etc.).

Also I've experienced with the secondary vs default settings of poetry sources and ran into errors as well.

I don't think I ever seriously experimented with this. Indeed _poetry_ claims to support a single primary index plus multiple secondary indexes (which I assume means that distributions are searched in the primary first, and if it fails in the secondary ones). So if that does not work as expected it might be a _poetry_ bug. Maybe there are already tickets on this, if not feel free to open a bug tracking ticket.

[GuillaumeLegoy]

@sinoroc you are right, I see the tag as well. It's not in the doc yet (might go ahead and submit a PR!).

Thanks for the info, it was really helpful, appreciate it!