Meson: subproject installs when project using it installs

[harryhaaren]

As from the IRC log just now:

HarryHaaren | hey all, any advice for when using a library subproject, should "install" be the default for the library? eg: project X uses subproject Y (as above, with install as default) ninja install of project X will also cause subproject Y to be installed globally next time project X has "meson build" run, it picks up the globally installed copy of subproject Y, instead of using the subproject Git method. This causes problems when configuring and installing multiple times.

Current solution is to mark the library "install : value" of the subproject to be false if meson.is_subproject()

Does that sound reasonable? (need to be careful with install_headers() and others too..) confirmed the above works. https://github.com/openAVproductions/openAV-Avtka/blob/master/meson.build#L31 (and #52 to notinstall headers either)

It places the burden of not polluting the system on a subproject - I think it would be cleaner if the "main project" could set to not install any of its subproject dependencies (especially when statically linking this makes sense).

Just noting my woes, and the solution above. Feel free to close this issue if the above solution is the recommended one.

[liugang]

It'd better to add a meson commond options to control this behavior. install library and exectuable of a subproject is useful at most time.

For example , I want to cross compile a package to a target device, I also need install all dependency libraries.

another example, when build a package run in docker, usually, the system run in docker is minimum, so it'd better ship all dependency library with you package.

[harryhaaren]

In this case, the "main project" has a dependency on the subproject. The subproject explicitly does not need to be available on the system; Meson will download the subproject from Git, compile it and use it.

In my opinion, with the above case (dependency not available system wide, so Meson grabs subproject to resolve, and uses locally), it does not make sense to install the subproject along with the main project, when ninja install is run later.

Does that explain my logic better?

[harryhaaren]

Ok; i think I need to be more specific: when doing shared libraries, they must be installed too. Static linking however, it is probably not required. Hmm. Yet another complication. I don't know the right answer.

[liugang]

if subprojects provide pkg-config file, you can workaround like this:

meson builddir_tmp --prefix=/tmp/rootfs_tmp;
 ninja -C builddir_tmp install

export PKG_CONFIG_PATH=/tmp/rootfs_tmp/lib/x86_64-linux-gnu/pkgconfig
 meson builddir  --prefix=/tmp/rootfs;
 ninja -C builddir install

[jpakkane]

In the mean time projects can ask whether they are being built as subprojects with meson.is_subproject() and adjust their install settings accordingly.

[nirbheek]

This would need a new command like meson install

[nioncode]

We now have meson install, but how does this help here? Maybe we can add an install kwarg to subproject to control if targets from it should be installed?

[dcbaker]

I think we actually already can know if something from a subproject needs to be installed and install it accordingly, if the target is used by the main project and is a shared library install it, otherwise don't. the only really tricky bit is data, since we can't know if the library needs to use that data at runtime.

[dpirch]

Maybe meson should just not install subprojects by default, and instead set default_library to static for subprojects so they _don't need_ to be installed. (There could be a way to override this like @nioncode suggested, in case a library cannot be build as a static library, or requires installed data.)

Having any project install random (possibly unstable) versions of shared libraries for system-wide use could cause various compatibility and security issues. Also, since subprojects are also _uninstalled_ when the parent project is uninstalled, uninstalling a project that has a shared library subproject will unexpectedly break other programs and libraries on the system that were installed later and use the same shared library.

[falk-werner]

I would like to have some control, which subproject is installed and which is not. I believe there is a large variety of use cases and an implicit logic might not fit.

The current behavior, that all subprojects are installed by default, is tricky. Let's think of an external test framework that is used as a subproject via some wrap file. The test framework, which is used to write unit tests only, will be installed on the system, when the project is installed. Is suppose, that is not what's intendet mostly.
_(Fortunately, goolgetest is not a native meson project and the wrap available at WrapDB does not install anything.)_

Otherwise, I can imagine to create a project that provides a set of libraries. Subprojects may be used for structuring purposes. The Boost library is an example for such a kind of project (although it is not a meson project).
In such a case, the subprojects should be installed (and in an ideal world, the user is able choose, if the wants static or shared libraries).

Therefore, I believe, a build system cannot know implicitly , which subproject should be installed and which should not. But I, as a project author, do know what I want and I want to be able to describe it in my meson.build file.
And there should be a common mechanism, how to describe this, since it is a common use case.

_(In contrast to prior posts in this thread, it would prefer to not change existing behavior, if possible)_

[eli-schwartz]

Or, subprojects should only build statically, because it's quite unlikely you want to install conflicting goop to /usr and if you're using private copies of your dependency you probably want them to be, well, private.

Required data files have been mentioned in the past, and here too I'd argue you do not want to conflict the system versions of such files; in order to robustly implement such a wrap it would need to be configurable where this data is expected to be found, and point it to a superproject-specific private directory in the subproject options in meson.build.

But under no circumstances EVER should a subproject install directly to the superproject's --prefix. Shared, static, data or otherwise.
If people want to install their subprojects, they should be required to opt in via some kwarg that they define at the same time they define default_library=shared and their custom overridden --libdir (or --datadir if applicable).

(Unless you're using a toplevel meson.build merely to list a bunch of subprojects as a meta-build system to build a bunch of projects in parallel, which I believe some people do, but this should be opt in, not opt out. Currently meson's default behavior assumes this very exclusive use case is the default way to install things. It's therefore impossible to use subprojects sanely without gating install logic behind meson.is_subproject().)

[nirbheek]

Unless you're using a toplevel meson.build merely to list a bunch of subprojects as a meta-build system to build a bunch of projects in parallel, which I believe some people do

FWIW, this is how GStreamer is built for development purposes, and for targeting certain embedded environments.

[dcbaker]

Or, subprojects should only build statically, because it's quite unlikely you want to install conflicting goop to /usr and if you're using private copies of your dependency you probably want them to be, well, private.

On linux and other *nix systems this is true. But for OSes like Windows you're expected to bundle all of your dependencies up and plop them in C:\Program Files\, and there are very legitimate reasons that you might want to have .dll's instead of statically linking, or you might have runtime files from your dependencies you need.

[eli-schwartz]

I'd consider that to be a private prefix/libdir. :)

[tp-m]

Do we really want to attempt to do some kind of automagic figuring out what bits from the subprojects "need" installing based on dependencies?

I would contend that that's pretty much doomed to make everyone unhappy.

I don't think it's possible to make things work right automagically for all the different use cases / scenarios people have.

Which IMHO means it has to be left to the person doing the installing.

How about something like:

• install only install bits from the main project, skipping subprojects
• install-all installs everything
• install-$subproject install bits from subproject only

Plus perhaps (suggested by various people elsewhere):

• install-libs: install only libs (main project only)
• install-exes: install only executables (ditto)
• install-dev: install only headers and .pc files (ditto)
• ...
• (some things would obviously have to be tagged so meson knows what type it is)
• install-all-libs: install only libs, but from main project + subprojects
• install-$subproject-libs: install only libs from $subproject
• etc.

[tp-m]

Or I guess that could be expressed in a simpler way by installing everything by default and adding options for type tags and subprojects to meson install (someone else might have suggested this on IRC already)

[dcbaker]

@tp-m, I think this is what you're looking for https://github.com/mesonbuild/meson/issues/7007 :)

[tp-m]

Oh my bad, sorry for the noise. I knew I had seen this before somewhere :blush:

[eli-schwartz]

@tp-m,

That's why I said:

If people want to install their subprojects, they should be required to opt in via some kwarg that they define at the same time they define default_library=shared and their custom overridden --libdir (or --datadir if applicable).

Ideally subprojects would default to not install (except on Windows), and developers could whitelist the things they need installed from the superproject (perhaps using a list of tags as discussed in the other issue). Or they could override the options and tell the subproject to install, using a custom prefix/libdir option.

For library(), the state of default_library is a pretty good heuristic for whether you want to install it as a subproject. But meson could just force people to be explicit.

[xclaesse]

The os has nothing to do here. On Linux I build gstreamer with tones of subprojects, all shared libraries, with /urs as prefix, but with DESTDIR at installation, then I can copy those files from build machine to my devices.

In any case, 'ninja install' is an alias to 'meson install', and should always install everything from all subprojects for backward compatibility, there is no reason to change that. We can add options to 'meson install' for more flexibility of course, I like the idea of tags, we can also add --no-subproject, or --only-subprojects=list, why not.

[eli-schwartz]

02:39 PM <xclaesse> if you install whatever in /usr you deserve to suffer anyway

It seems this use case is undesirable. Please close this as WONTFIX so people know what they're getting into when they consider using meson subprojects.

[xclaesse]

What I mean is you can't install stuff in /usr then complain that it breaks stuff from your distro. Linux has /usr/local for that and it is Meson's default prefix.

You're over simplifying the problem by thinking that static link subprojects and not install anything will just work. That's not true. Subprojects could install data files (e.g. i18n, images, etc). Your use-case is perfectly valid, and I worked hard to make default_library a per-subproject option exactly for that, but it's not the only use-case.

This issue is definitely not a WONTFIX, I totally support the idea of adding meson install --no-subproject. I also like the idea of installation 'groups' https://github.com/mesonbuild/meson/issues/7007, but that's not exclusive.