Meson: Using subproject feature when adopting already existing projects

[dzhang-b]

From the documentation, I know subproject feature has a few limitations that make it hard to use.

1. subproject folder cannot be named differently. (we are using submodules name)
2. subproject has to follow specific path layout. root_dir/subprojects/xxx

The existing project has a lot of other small subprojects exists recursively, and a lot of scripts (tools, build CI, etc) are depended on these paths.

In order to use meson right now, I have to manually prepare a ./configure script that configure each meson project recursively, which I am not sure if it is ideal.

Why can't we allow subproject declared in the root meson.build file using whatever path we want?

# root repo
subproject('path..',..)

# under subprojects 
if project.root
  subproject('path..',..)...
else
  // subproject must defined in the root meson.build..
endif

[phillipjohnston]

You can use this argument in the project() declaration:

subproject_dir: 'submodules',

From the reference manual:

subproject_dir specifies the top level directory name that holds Meson subprojects. This is only meant as a compatibility option for existing code bases that house their embedded source code in a custom directory. All new projects should not set this but instead use the default value. It should be noted that this keyword argument is ignored inside subprojects. There can be only one subproject dir and it is set in the top level Meson file.

[phillipjohnston]

I should note that in the project I adapted to meson, the subproject directory is src/external, so it is not necessarily a "top level directory".

[dzhang-b]

@phillipjohnston Thanks!, This definitely a great start for using subprojects in meson.

[xclaesse]

It is a design decision that all subprojects must be in the same directory.

I personally think it is an arbitrary limitation that should be lifted for the case of in-tree subprojects. I think there are 2 types of subprojects: external dependencies that can be downloaded using a wrap file (those should remain into subprojects/), and internal project libraries that can be built standalone (those should be allowed anywhere).

Sadly @jpakkane strongly disagree and I don't really understand why.

[dzhang-b]

@phillipjohnston @xclaesse While using the project option

subproject_dir: 'submodules',

It is kind of unnecessary to limit the path name of subprojects, the limit could be that subproject could only be set at top level meson.build file.

[dzhang-b]

additional new meson functions can be add like this?

# the function can only be used in top level `meson.build` file.
subproject_set('name', 'path')

# the function can check if a subproject is set in the top level project
subproject_get('name', 'path') # the path is relative, meson can check if top level and subproject definition are the same

then a sample use case can be:

if is_subproject() # include as subproject in other projects
   # p = subproject_get('xxxx', path) # report error if no match meson.build
else
  # p = subproject_set('xxxx', path) # include the subproject like as a top level meson.build
endif

[phillipjohnston]

The holy war resumes!

(I agree there is an ontology of subprojects, and that enforcing a single "this is how it shall be" paradigm is ineffective.)

[dzhang-b]

During the process of porting existing projects,

I found that dependencies between subprojects is not allowed. all inter-dependencies must bring up to the master project using subproject.get_variables().

If meson can support subproject.get_variables(), why can not meson support subprojects hierarchy as long as same level subprojects cannot talk to each other?

   /    \
 B        C
          /  \
        D      E

In the above graph, B cannot talk to D or E or C directly for example, every symbol must be resolved at A for B and D/E works together.

Other issue is that existing source expect to use header files from installed directory. while include_directory only specify the source. Is it possible to have include_directories to prepend the install_dir string?

[xclaesse]

B can do subproject(D).get_variable().

[dzhang-b]

@xclaesse can meson do this now?

[xclaesse]

It always could, AFAIK.

[dzhang-b]

@xclaesse It tells me that D.wrap file is not found, but where should I put the wrap file and what content I need to put into it?

[xclaesse]

@dzhang-b wrap files must be in A's subprojects/ directory. They describe how to download code for B, C, D, E. So you'll have A/subprojects/{B,C,D,E} but you cannot have A/subprojects/C/subprojects/{D,E}, which is a limitation we should probably fix.

[xclaesse]

https://mesonbuild.com/Wrap-dependency-system-manual.html

[dzhang-b]

I will close this issue, as I find out that using a package manager like spack is a better way to manage multiple projects/components than using build system.