Conan: [Feature/Question] Developer local workflow.

[p-groarke]

Hello,
A problem I see coming up with various dependency managers and build systems is local workflow for developers. To a certain extent, cmake's externalproject supports this, as you can point to a local directory for your download source.

I discussed this briefly on slack, and I've tried the current "local flow" from bincrafters. However, that is a completely different use case intended for package creators.

This may already be possible, however I haven't found any information about it.

The user story:

• I am a developer, I don't know anything about conan other than it automagically builds things and actually works (OMG WAOW <3 !). I don't want to learn anything about conan either, because that isn't my job and I don't have time.
• I have an application that uses, lets say 100 internal libraries. Those libraries (lib1 .. lib100) are in constant development. By myself and colleagues.
• While working, I need to add a feature to a lib. I want to do so quickly, test locally and have the least friction possible.
• I will build my software with this local version of the lib until everything is ready, PR sent, review done etc. Then I want to point back to the remote lib at version x.y.z+1.

So, I know this might seem impossible or a huge undertaking. Lets say I'm not expecting this workflow to drop anytime soon ;)

Here is an example implementation to clarify the sort of thing that would be a real game changer for me.

Dream Workflow

In conanfile.txt, I can change a package source quickly and easily. So for example:

[requires]
Bacon/1.0.0@me/stable

I clone Bacon somewhere on my machine, then in conanfile.txt, I add a path overload for the new source:

[requires]
Bacon/1.0.0@me/stable

[source]
Bacon:source_url=~/Documents/code/Bacon

At this point, conan automagically starts "pulling" from my local development clone. When done I remove the source option, increment version and bingo bango, frictionless local development.

This example is simply to illustrate the idea, I'm sure there are many other ways to do this. However, here are some things that wouldn't be appropriate :

• Command-line only : Problematic because you have to remember an enormous command when dealing with 5+ modified libs. Worst on Windows as cmd doesn't have a persistent history. You end up wasting time creating a batch script for your ungodly command.
• Editing conanfile.py, having to work in ~/.conan/, having to run local flow. Basically, anything that requires package creator level knowledge about conan cannot be asked of developers. Deadlines are short and people are busy etc.

I think that sums it up.
Thanks and good day

[memsharded]

Hi @p-groarke

Thanks very much for your detailed feedback. I think this might be exactly what the "conan-project" feature would be implementing. Please have a look at: https://github.com/conan-io/conan/issues/1377

This feature has been put on hold, until the local flow was matured. In short, we thought that the local flow, together with the $ conan export-pkg command, that can be automated, would have been enough. This would put artifacts in the conan local cache, where they could be already consumed from other packages.

If you think the "conan-project" feature is what you are missing, please:

• Close this issue and continue conversation there (link this issue for reference)
• Briefly comment why the $ conan export-pkg is not good enough
• Comment on the idea of having a dedicated file (I think it doesn't belong to conanfiles, or command line), like "conan.project" where you would map packages to local folders (including the possibility of overriding include-dirs, lib-dirs, etc (give them a different layout locally to the one defined in the recipe package_info()

Many thanks!

[p-groarke]

I've posted a follow up on the mentioned issue. Thanks to you as well :D

[sourcedelica]

I have the same use case and am thinking about something similar to your approach.

My current workaround is to have the developer edit CMakeLists.txt and point to MyProject::Bacon instead of CONAN_PKG::Bacon.

For example:

#   set(MYPROJ_DATABASE_COMMON_LIB "MyProj::DatabaseCommon")
    set(MYPROJ_DATABASE_COMMON_LIB "CONAN_PKG::database-common")
#   set(MYPROJ_SERVICE_CLIENT_LIB "MyProj::ServiceClient")
    set(MYPROJ_SERVICE_CLIENT_LIB "CONAN_PKG::service-client")

    target_link_libraries(service-test
            ${MYPROJ_DATABASE_COMMON_LIB}
            ${MYPROJ_SERVICE_CLIENT_LIB}
            CONAN_PKG::catch2 CONAN_PKG::spdlog)

This gets more complicated when the library that you are working on is used in multiple places. For example, if I am working on CoreUtil and that library is used by both DatabaseCommon and ServiceClient, I have to change both DatabaseCommon/CMakeLists.txt and ServiceClient/CMakeLists.txt to use MyProj::CoreUtil instead of CONAN_PKG::CoreUtil. This is error-prone to say the least, and why your conanfile.txt approach is better because it is done in one place and it automatically translates correctly in conanbuildinfo.cmake.

Another thing I'm concerned about is developers accidentally checking in the modified conanfile.txt (or CMakeLists.txt). What I'm thinking about is having a non-version controlled conanfile-override.txt (listed in .gitignore) to contain the overrides.

I'm looking into using a customized CMakeGenerator for this.

[p-groarke]

I've updated the idea in the linked issue. There is a conaninfo.txt file in your build/ directory which would be perfect for this sort of thing ;)