Conan: [Feature] Pass 'workspace_folder' value to the layout files

[jgsogo]

Hi!

I think it will be useful to have the workspace_folder value available in the layout files, it will allow to define all the build_folder of the editable packages that belongs to the workspace inside the same directory, let me show you an example:

• Given a workspace definition:

         editables:
              HelloB/0.1@lasote/stable:
                  path: /path/to/some/place/B
              HelloC/0.1@lasote/stable:
                  path: /mywork/dev/path/HelloC
              HelloA/0.1@lasote/stable:
                  path: /this/is/HelloA
          layout: layout
          workspace_generator: cmake
          root: HelloA/0.1@lasote/stable
  

• The layout file could be

  ...
  [build_folder]
  {{ workspace_folder }}/build/{{ reference.name }}
  

Take into account that this variable will only be available when working with workspaces, so the layout file should add some conditions if it is going to be used for editable packages too.

Some preliminar work can be found here (not even a POC)

[ezaiakinsc]

Please, consider that feature to be implemented in nearest releases! In our build scenarios, source folders and build folders are never related, so after upgrade to 1.13.2, we cannot use workspaces at all - there no such a profile, that can describe relations between unrelated values.

I reported this issue in https://github.com/conan-io/conan/issues/4800 - but is was closed, despite of the description: I explicitly stated we need to build to subfolders of workspace folder, not somewhere nearby sources folder.

Another option would be to allow simplified workspace generation - by invoking "usual-mode" generator for root package into the workspace folder: our custom generators are supporting editable packages, and including them into build as sub-projects automatically.

[jgsogo]

Hi @ezaiakinsc.

I did a quick trial in the past and it wasn't as straightforward as I was expecting. IMHO (and this is conditioned by the POC I'm implementing here https://github.com/conan-io/conan/pull/5314) I also support this functionality and I think it is mandatory for proper implementation of workspaces. Different workspaces may use the same library and they need different directories, even more (and we will disagree on this) I think that the workspace should force the build directory.

Coming back to the current implementation of workspaces I think that this feature is really nice, but not mandatory; so I'm very interested in understanding your use case so we can realize why it is needed. Why can't you use as _a folder nearby sources folders_? Why is it a stopper to use workspaces?

Thanks for the feedback.

[ezaiakinsc]

Hi @jgsogo

In our use case, sources for libraries are checked out from git partially by automatic script into separate area, partially manually into user home directory. Creating build folders for intermediate artifacts in those areas is not an option, so anything-relative-to-sources does not work for us.

Say, on Windows sources could be on drive C:, and workspace on drive D:, I guess there are no such a relative path exists.

Mentioned POC looks promising, idea of locking build artifacts into workspace folder sounds really good! However, I would like to consider couple of things to make it usable for us:
1) Please make layouts optional: as I explained in https://github.com/conan-io/conan/issues/4878, we are using automatized packaging and interconnection for both native code and accompanying Java counterparts (on Android platform) - by using custom generator. As result, in our use case layouts are pure duplication, and has no effect other than build dir placement (which is already set by -if parameter!).
2) Please consider alternative use flow for workspaces as direct replacement of "conan install" command, without "workspace-mode" generator - invocation of our custom "usual-install-mode" generator for root element into the workspace is all we need (OSS workspace generators will not support generation of Gradle scripts for optional Java counterpart of the libraries for android platform anyway).

Maybe it makes sense to introduce "light workspaces" for those, who need exactly same functionality, as "conan editable add && conan install", but not on system-wide level, just for specific build directory? Last revisions of workspaces are going further away from such a naive implementation.

In our case, for Android platform (we are developing cross-platform products), invocation of conan is done by Gradle plugins, multiple times, with different settings (per arch, for example). Build directory is dictated by Gradle, and output must be generated exactly at this path. With "conan install" command and editables that work like a charm - custom generator delivers same CMake targets as editable sources or as compiled libs, and top-level "consumer" project just don't care. Same works for Gradle and Java counterparts of the libraries - they are automatically attached either as source subprojects, or as compiled .aar files from corresponding conan package folder, based on json, generated by conan, all custom generator/Gradle needs for that - path to the root of the package/root of the editable.

[climblinne]

Hi @jgsogo,

for us this would be also an important feature. We are using the TI Code Composer Studio IDE (based on Eclipse), for which we have written a custom CMake Generator (based on the Visual Studio Compiler).
There are some limits when generating a workspace for the Code Composer Studio. One is, that the project files must be inside the workspace build folder.
That I could easily generate with a workspace_folder variable.

[kenfred]

Hi @jgsogo

I too am looking for a way to use the workspace folder in the jinja expression. Right now I find the layout file mechanism to be problematic on how it specifies the build folder: it doesn't allow out-of-source builds unless you blindly do a ../ relative path, and it doesn't allow for multiple build folders for different configs.

I want to issue a command like conan install on my workspace providing an out-of-source directory (not a relative path hardcoded in a layout file) and have it generate all the necessary files in my out-of-source directory. This is how workspaces worked prior to 1.13.2. I'm hoping that having {{ workspace_folder }} would do this for me.