Conan: Consider defining remotes per package reference

[memsharded]

IMO, it shouldn't be coupled to conanfiles at all.
There is an idea to be able to define them in profiles, could be a possibility. Maybe another is to define an extra file package_origins.txt with such info. Basically it is very similar to the registry.txt file, and there are conan utils in the codebase to do this, shouldn't be very hard to implement.

Related to: https://github.com/conan-io/conan/issues/2645

[sztomi]

What would this mean for a workflow where we switch between multiple remotes containing the same packages? e.g. we have an experimental remote which promotes its builds to stable.

[memsharded]

This feature aims to give flexibility defining remotes, better than just using remote ordering. For your case, it means that you will be able to define if packages are retrieved from "experimental" or "stable" remotes, irrespective of remote ordering, if you provide the right information in the profile or in such a new file. Of course this is a fully backward compatible feature, nothing changes if that information is not provided.

[memsharded]

Furthermore, you can select which packages are retrieved from which remote: so you can mix packages from experimental and packages from stable, just definining something like:

Pkg/0.1@user/channel        experimental/maybe URL?
Other/1.2@user/channel     stable

[sztomi]

Sounds very interesting, especially the part where we can use individual packages from different remotes. /cc @tru

[memsharded]

You can do it right now with conan remote update_ref, it would be just an UX improvement / automation over it.

[solvingj]

I understand the reasoning here. There is a proliferation of remotes since switching to Bintray, and Conan Center is not populated enough to satisfy most needs. Until it's more populous, this will be useful. The determinism it would bring will probably help to address some compliance/governance concerns (I imagine). I would say that I agree with the goal to keep it out of conanfile.py.

Also, remote management does become a challenge in other ecosystems under some rare circumstances, for example in some bigger enterprise where many dev teams have nuget/maven repos, and workflows involve pulling in many different nightly builds in a flexible way. So, there's lots of precedent to work on such a feature.

However, just a note about complexity, when you add the functionality to locking a package to a remotes, you effectively extend the "fully-qualified address" of a package to : zlib/1.0.0@conan/stable@conan-center. It makes the inner @conan/ part of the channel definition feel less meaningful and a bit confusing. Also, now you split the full definition of the "fully-qualified address" in two different places, and everyone in the organization has to share some registry.txt if they want consistency or safety. I think it has non-trivial implications which are not obvious on first glance, so hopefully people won't need to leverage this too much. Still, it's probably still good for cases that really do need it.

[lasote]

I'm not sure about this feature. I don't like the idea of mixing this with profiles. Where this issue came from? It should be used in very special cases. Maybe just an improvement of the conan remove update_ref command to allow wildcards and better visualization could be enough?

[memsharded]

I created the issue after a couple of comments in slack, latest one from @sixten-hilborn IIRC.
The functionality is indeed an extension of conan remote update_ref, the key would be if it is executed automatically when some file is found or if it is command base. I think for people actually having this need, the command is not enough, and they are looking for a file based definition of correspondences Pkg->Remote