Tox: Clarify which part of the indexserver spec is deprecated

[obestwalter]

... or even decide to not deprecate it after all.

This came up on the mailing list.

It seems that this config option is well established, works as expected and is used in many scenarios (especially corporate when using internal pypi repos with different stages or devpi). So we might want to decide to remove the deprecation notice completely, if there are no objections regarding the maintenance of it.

As @fschulze mentioned, at least the single index variant is highly desirable (and I am pretty sure widely used).

I would opt for removing the deprecation notice and keep the full functionality.

[gaborbernat]

I personally would like to remove it completely. I need to think what's a better solution to the problem though. I don't think it's tox problem to solve index server abstractions. People should use devpi for that with some option to pass in the index server via PIP_INDEX_URL env var.

[obestwalter]

Having to pass in something rather than being able to configure it in the tox.ini is not a desirable alternative IMO. Behind corporate firewalls this is a very useful feature to set the index to the internal pypi mirror for everyone rather than having to tell everyone they have to call with some special env var set.

[gaborbernat]

You can use setenv to set that env var in tox.ini. That's what we do at our company and works like a charm.

[fschulze]

I just tested it and I think setenv with PIP_INDEX_URL is a good solution. According to the docs it is used for setup and test run. This just needs to be documented prominently so it's easy to find and there should be a deprecation message pointing to the docs when the option is used in the config. Not sure about the command line option though. It doesn't seem to be deprecated, but also only seems to pass on one index straight to pip.

[fschulze]

One problem I just thought about is that you can't overwrite PIP_INDEX_URL from the command line or via CI environment variables, but that may already be the case right now.

[gaborbernat]

We can change the default pip command to auto pass through pip index URL from outside env var👌For now we do it explicitly overriding the pip install command inside tox.ini at our company this.

[gaborbernat]

Here's what we use:

install_command = pip install -i {env:PIP_INDEX_URL:https://magic.index.server.address/simple} {opts} {packages}

We should probably just make this as the default and drop the entire indexserver thing. This would keep the installer tool agnostic the index server selection. The main reason for this is that index server is a pip thing, conda/dpkg, for example, doesn't have such. It's a clearer separation of concern if we the tool doesn't know about pips internals; and makes it easier to extend to use other packaging tools.

This allows the user to easily insert their own index server (such as a local devpi mirror index), but fallback to some hardcoded sane value on a per project basis.

[obestwalter]

This needs a deprecation warning then and a hint how to solve this differently, plus some docs.

[sinoroc]

The indexserver setting is still the recommended solution in the examples "_using a different default PyPI url_" and "_installing dependencies from multiple PyPI servers_".

Is there a newer, up-to-date example?

Is using the PIP_INDEX_URL environment variable still the recommended alternative?

What can we expect from the upcoming _Tox 4_ on this topic?

[gaborbernat]

Is using the PIP_INDEX_URL environment variable still the recommended alternative?

Yes. tox 4 will also recommend this. This ticket is for someone to fix our documentation to reflect this. PRs are welcome.

[sinoroc]

Thanks.

I will experiment with that. We will see if I find a technique I feel is good enough to be used as an example.

[ziima]

I've replaced the indexserver from the docs in basic case. I'm not sure though, what's the purpose of multiple PyPI servers. Is the problem in question to locate packages which are not available at pypi.org?

[sinoroc]

I'm not sure though, what's the purpose of multiple PyPI servers.

@ziima It should cover the _--extra-index-url_ use cases.

Maybe something like:

[tox]
setenv =
    PIP_INDEX_URL = {env:PIP_INDEX_URL:https://pep503.alpha.dev}
    PIP_EXTRA_INDEX_URL = {env:PIP_EXTRA_INDEX_URL:https://pep503.bravo.dev https://pep503.delta.dev}

[fschulze]

@ziima afaik it was added for devpi etc, so one could test releases before uploading the to PyPI. It might be that multiple indexes were necessary until devpi got inheritance to look at PyPI itself.

[ziima]

OK, that seems to be simple. I was somewhat confused by the case where individual packages were installed from different indexes.

[sinoroc]

OK, that seems to be simple. I was somewhat confused by the case where individual packages were installed from different indexes.

I don't think this part can be recreated without the deprecated indexserver setting (I have not found how).

[ziima]

Me neither, but it seems like a specific behavior of tox only. I haven't found alternatives in other tools, so I believe it's too exotic and can be removed.

[sinoroc]

Me neither, but it seems like a specific behavior of tox only. I haven't found alternatives in other tools, so I believe it's too exotic and can be removed.

@ziima I agree.

On a side note... What might be missing in the Python packaging ecosystem is a requirement notation such as:

Library @ https://pep503.tango.dev/simple/

I believe this would fit quite well into requirements.txt files, or:

[testenv]
deps =
    Library @ https://pep503.tango.dev/

which would translate to:

python -m pip install --index-url 'https://pep503.tango.dev/' Library

[sinoroc]

I opened a discussion on the topic:

• https://discuss.python.org/t/dependency-notation-including-the-index-url/5659

[sinoroc]

As far as I understood, the philosophy is that _indexes_ should be indistinguishable, interchangeable. If 2 projects of the same name exist on 2 indexes, it is assumed that they are the exact same project and that 2 distributions of the same name and version number are the exact same distribution and so it does not matter from which one we fetch. In other words:

Packages are expected to be unique up to name and version, so two wheels with the same package name and version are treated as indistinguishable by pip. This is a deliberate feature of the package metadata, and not likely to change.

-- pfmoore, https://github.com/pypa/pip/issues/5045#issuecomment-369521345

If one needs to circumvent this, they need to put something like _devpi_ or _pydist_ in place.

• In the case of devpi: of particular importance is its "_inheritance_" feature, as far as I understood this would be the key feature that would prevent downloading a dependency from the "_wrong_" index when configured right (not sure how, though).
• For pydist: https://pydist.com/blog/extra-index-url

With that said, and the changes made in #1728, I would say that the topic can be closed.