Hello
I apologize if this is not the proper place to ask this question.
This is not strictly an issue but more of a question whether conan supports the workflow I'd like to implement.
Currently, our development workflow is as follows :
We have a huge Visual Studio solution that builds 200+ projects retrieved from source code.
The team usually retrieves this source tree and starts working on features and fixes in one or several of these projects. Most of the time, a single developer leave 90% of the projects untouched during a development story (but may retrieve commits in these projects done by others)
Since the build time is very very slow, I'd like to split this solution and put everything in conan packages, so that developers may retrieve pre-built projects.
However, It's difficult to know in advance whether a developer will need to change a given project.
My question : would it be possible to put these projects in conan packages, containing the source code, the recipe to build and pre-built binaries, while, at the same time , allowing the developers to rebuild the potentially modified source code locally if he/she has made changes ?
Another consequence of this scenario is that, if the developer has made local changes to a project and also retrieves a new conan package for the same project, his source code changes should not be overwritten (it must be done with proper git/svn commands, not a raw copy) and, of course, in that case, the pre-built binaries from the package cannot be used, because of the local changes.
The goal is NOT to let developers publish new packages from their PC. The CI will take care of this, but I'd like them to be able to speed up their build while also testing their changes without having to commit/wait for CI/update.
I want to introduce Conan with the least possible amount of friction : it would be great if I can configure the projects so that the developer who is used to choose "build solution" will benefit from pre-built binaries from the conan packages if they are present, but will build them if needed (i.e. if there are local changes).
Also, currently, this huge solution produces all modules (shared libs and executables) in a single top-level folder. I need to find a way for a conan package installtion to also put the needed DLLs and EXEs (static libs can stay where they are) in this top-level binary folder.
(And I also have the exact same problem for the Linux side, where we have Linux-specific CMakeLists.txt files with a top-level file that includes everything.)
Thank you very much for your advice regarding the suitability of Conan for this and whether my goals are maybe unrealistic.
Benjamin Golinvaux
Yes, this is a good place, and the question is very interesting.
You can see some related issues from latest release: https://github.com/conan-io/conan/issues/1001 and https://github.com/conan-io/conan/issues/1171
So, basically, what we have done in 0.23 is improve and smooth most of the "local" commands, i.e. those who work in the user space/folders, to allow a smoother flow working on active development of packages. With the proposed approach, the subprojects still link with packages that are in the conan local cache, not linking with the other subprojects in user space. When editing a subproject, a user can locally execute conan package
and conan package_files
to directly copy the binaries into the conan local cache, without needing to rebuild them "cleanly", so they are directly available to other subprojects to link with it.
The features are there, and I have an example project and working in the docs. I hope I can publish it soon (we have been busy with other things, like the conan.io => bintray migration of tomorrow). I will ping when they are ready and online.
Furthermore, we are considering to add some "conan-project" concept that would allow after a conan install
linking with other subprojects binaries directly in user folders, so developers can work more smoothly without having to conan package_files
binaries to copy them to the local cache.
So, in short: it is possible as today (still missing the docs and examples), with some added required steps (which could be a minor inconvenience if project is large and time savings are then also big). And we are continuously improving the tool based on feedback, so it is very likely that the "conan-project" feature could be available in a couple of iterations (months).
Thanks very much for your feedback!
Explicitely added the feature: https://github.com/conan-io/conan/issues/1377
Just a quick note to add that Conan was one of the (many) reasons why we are very glad of our choice of Artifactory Pro over other solutions when looking for a repo manager for our company. We're just starting using it and it is great :) Fantastic work.
Thanks very much!!!! :)
Hi @bgolinvaux
I wanted to follow up on this issue, since it remains open since a long time, and I think there are some related new conan features:
These features also have dedicated issues here in github. Could you please update on this issue? Maybe it can be closed and follow up in other issues? Thanks!
Sorry for being.. ahem... a bit late on this. The new features definitely fit the team needs (I left that company and went through some turmoil since that request). THANKS !
Most helpful comment
Just a quick note to add that Conan was one of the (many) reasons why we are very glad of our choice of Artifactory Pro over other solutions when looking for a repo manager for our company. We're just starting using it and it is great :) Fantastic work.