Meson: 0.51.0: cpp_std setting not being applied

[phillipjohnston]

This is tested with 0.51.0 and master @ commit 54b1c43277d16dcef2d8acc98d131ab9232d2fac.

I have a project which is specifying c_std and cpp_std:

project('libc',
    ['c', 'cpp'],
    default_options : ['c_std=c11','cpp_std=c++17'])

I can see the C standard being applied in build.ninja:

build src/e88f707@@c@sta/crt__Exit.c.o: c_COMPILER ../src/crt/_Exit.c
 DEPFILE = src/e88f707@@c@sta/crt__Exit.c.o.d
 ARGS = -Isrc/e88f707@@c@sta -Isrc/ -I../src -I../include -I../src/gdtoa/include -I../arch/x86_64/include -Xclang -fcolor-diagnostics -pipe -Wall -Winvalid-pch -std=c11 -g -isystem ../printf -isystem ../src/gdtoa/include -isystem ../openlibm/src -isystem ../openlibm/include -isystem ../include -fno-builtin -nodefaultlibs -fno-stack-protector -DNO_ERRNO -DIFNAN_CHECK -DGDTOA_NO_ASSERT -DNO_FENV_H -Wno-reserved-id-macro -nostdinc

Build I don't see the C++ standard being set:

build printf_test@exe/printf_test_test_suite.cpp.o: cpp_COMPILER_FOR_BUILD ../printf/test/test_suite.cpp
 DEPFILE = printf_test@exe/printf_test_test_suite.cpp.o.d
 ARGS = -Iprintf_test@exe -I. -I.. -I../printf/test -Xclang -fcolor-diagnostics -pipe -Wall -Winvalid-pch -Wnon-virtual-dtor -g -Wno-format -Wno-format-invalid-specifier -Wno-old-style-cast -Wno-missing-prototypes

And the incorrect standard is further reflected by the failure to recognize the noexcept keyword:

[1/2] Compiling C++ object 'printf_test@exe/printf_test_test_suite.cpp.o'.
FAILED: printf_test@exe/printf_test_test_suite.cpp.o
c++ -Iprintf_test@exe -I. -I.. -I../printf/test -Xclang -fcolor-diagnostics -pipe -Wno-format -Wno-format-invalid-specifier -Wno-old-style-cast -Wno-missing-prototypes -MD -MQ 'printf_test@exe/printf_test_test_suite.cpp.o' -MF 'printf_test@exe/printf_test_test_suite.cpp.o.d' -o 'printf_test@exe/printf_test_test_suite.cpp.o' -c ../printf/test/test_suite.cpp
In file included from ../printf/test/test_suite.cpp:31:
../printf/test/catch.hpp:420:63: error: expected ';' at end of declaration list
        SourceLineInfo( char const* _file, std::size_t _line ) noexcept
                                                              ^
../printf/test/catch.hpp:427:43: error: expected ';' at end of declaration list
        SourceLineInfo( SourceLineInfo&& )              noexcept = default;

The target is quite simple:

printf_tests = executable('printf_test',
    sources: ['printf/test/test_suite.cpp'],
    cpp_args: ['-Wno-format', '-Wno-format-invalid-specifier', '-Wno-old-style-cast', '-Wno-missing-prototypes'],
    include_directories: [include_directories('printf/test')],
    native: true
)

This happens with both clang and GCC.

[phillipjohnston]

Meson itself is picking them up correctly:

Compiler options (for host machine):
  Option        Current Value Possible Values                                         Description
  ------        ------------- ---------------                                         -----------
  c_args        []                                                                    Extra arguments passed to the C compiler
  c_link_args   []                                                                    Extra arguments passed to the C linker
  c_std         c11           [none, c89, c99, c11, c17, gnu89, gnu99, gnu11, gnu17]  C language standard to use
  cpp_args      []                                                                    Extra arguments passed to the C++ compiler
  cpp_eh        default       [none, default, a, s, sc]                               C++ exception handling type.
  cpp_link_args []                                                                    Extra arguments passed to the C++ linker
  cpp_std       c++17         [none, c++98, c++03, c++11, c++14, c++17, c++1z, c++2a, C++ language standard to use
                               gnu++11, gnu++14, gnu++17, gnu++1z, gnu++2a]

[phillipjohnston]

Possibly relates to #5495?

[scivision]

Does #5560 fix it? Are you using clang?

[phillipjohnston]

Will check shortly. I see this with clang and gcc. (mentioned at the bottom of the main post)

[phillipjohnston]

@scivision that does not fix the problem. My builds still fail and no C++ standard FLAG is present.

c++ -Iprintf_test@exe -I. -I.. -I../printf/test -Xclang -fcolor-diagnostics -pipe -Wall -Winvalid-pch -Wnon-virtual-dtor -g -Wno-format -Wno-format-invalid-specifier -Wno-old-style-cast -Wno-missing-prototypes -MD -MQ 'printf_test@exe/printf_test_test_suite.cpp.o' -MF 'printf_test@exe/printf_test_test_suite.cpp.o.d' -o 'printf_test@exe/printf_test_test_suite.cpp.o' -c ../printf/test/test_suite.cpp
In file included from ../printf/test/test_suite.cpp:31:
../printf/test/catch.hpp:420:63: error: expected ';' at end of declaration list
        SourceLineInfo( char const* _file, std::size_t _line ) noexcept
                                                              ^
../printf/test/catch.hpp:427:43: error: expected ';' at end of declaration list
        SourceLineInfo( SourceLineInfo&& )              noexcept = default;

[phillipjohnston]

Also, just tested: doesn't matter what std version I pick, none of the settings are applied for C++.

[jpakkane]

Managed to replicate this. The fix wlll be in 0.51.1.

[jpakkane]

@Ericson2314 this is actually quite complicated. The reason for this is that native: true causes the target to be tagged for build and then it picks up the wrong settings. In native compilation there are both build and host and they have different contents, at least for compiler options.

Trying to make the dicts in PerMachine to be the same breaks as well as ignoring native: true when building natively.

[jpakkane]

The unit test change is simple:

diff --git a/run_unittests.py b/run_unittests.py
index 90e5c9d8..a1e56b67 100755
--- a/run_unittests.py
+++ b/run_unittests.py
@@ -5242,7 +5242,9 @@ endian = 'little'
         testdir = os.path.join(self.unit_test_dir, '50 std remains')
         self.init(testdir)
         compdb = self.get_compdb()
+        self.assertEqual(len(compdb), 2)
         self.assertRegex(compdb[0]['command'], '-std=c99')
+        self.assertRegex(compdb[1]['command'], '-std=c99')
         self.build()

     def test_identity_cross(self):
diff --git a/test cases/unit/50 std remains/meson.build b/test cases/unit/50 std remains/meson.build
index ac6f9e24..51ae906e 100644
--- a/test cases/unit/50 std remains/meson.build        
+++ b/test cases/unit/50 std remains/meson.build        
@@ -1,2 +1,10 @@
 project('std_remains', 'c', default_options: ['c_std=c99'])
+
 executable('prog', 'prog.c')
+
+# Check that native: true does not affect the use of c_std in
+# non-cross builds
+
+if not meson.is_cross_build()  
+  executable('prog2', 'prog.c', native: true)

Making it actually work is trickier.

[Ericson2314]

@jpakkane the current behavior is what I expect. The build.c_std vs c_std distinction is supposed to be the same so native and cross builds behave the same. The Ux logic is if you don't care about cross, then you wouldn't bother with native: true (that's why we removed my warning), and then you shouldn't notice any breakage, but if you do care about cross, then you should be careful with your native: true and build.c_std.

CC @dcbaker

[jpakkane]

If you do this:

project('xxx', 'c', default_options: ['c_std=c99'])
executable('foo', 'foo.c', native: true)

Then it _must_ use -std=c99 to build the target. No ifs, buts or maybes. There is not even any option to set the standard version for this case, because build. options are not exposed when not cross compiling. Not doing that is counterintuitive and unexpected and will bury us in a stream of bug reports.

It does not matter so much how this is solved (like maybe make c_std mirror the host setting when native compiling) but this _has to work_.

[Ericson2314]

@jpakkane Well that is why I originally did both build. and host. prefixes, with the unprefixed options affecting both build and host. You were worried about too many options and @dcbaker thought just build. and unprefixed is fine, so we went with that instead.

There is not even any option to set the standard version for this case, because build. options are not exposed when not cross compiling.

That is not true last I checked? The build.* options are always exposed.

[jpakkane]

That is not true last I checked? The build.* options are always exposed.

So they are. They really should not be, though. They clutter complicate things a lot for most people who don't cross build. They should only be shown when cross building. The output of meson configure has gotten almost unreadable with so many options.

But taking things one by one. If one sets default_options: ['c_std=c99', 'build.c_std=c99'] then it does work. @phillipjohnston is that a suitable fix and UI experience for you?

Having build. options always there makes things brittle for subproject usage. Suppose you have a project that builds a tool to generate sources and requires, say, c11 and sets build.c_std in its default options. Thus far everything works but if it is then used as a subproject (native building, not cross) and the master project has set c_std but not build.c_std, then things will break and the cause of that is not at all obvious.

[phillipjohnston]

@jpakkane, honestly, I'd rather stick with 0.50.1 than use that workaround. My main reason is _this used to work as expected_. Were I to upgrade, and were that to be the recommended solution moving forward, I would simply go back to manually managing standard flags in my project and not use the built-in meson options.

I thought your initial statement hit the nail on the head:

If you do this: [...] Then it must use -std=c99 to build the target. No ifs, buts or maybes. [...] Not doing that is counterintuitive and unexpected and will bury us in a stream of bug reports.

Adding build.c_std=c99 remains a counterintuitive and unexpected approach. The option name is not obvious. There's nothing in the name build.c_std that communicates to me "oh hey this is for the native build and that other one is for cross".

I also share your concern about subprojects. I have multiple projects that can be used standalone or as subprojects (such as this one, which is a libc implementation).

The reason I found this issue was that I started from a barebones system and walked through my project setup instructions to validate them - and suddenly I started getting strange build failures that didn't make sense and weren't reproducible on my main machine (which was still on 0.50.1). I've been using meson heavily for a year and a half, and here we are talking about the failures. So I don't expect people consuming my library to get that right if they're not meson experts.

[phillipjohnston]

My expectation is that the standard version specified for a project applies to everything if it's the default in my project.

I'm sure someone can come up with a great reason for needing different standards applied for cross/native builds. But I'm not one of them - and I'm pretty much always cross-compiling since I work on embedded systems.

[jpakkane]

For cross compilation the reason for the different values is simple: suppose you build a codegen tool that uses C++17 but your deployment target only has C++11 (or even 98). The problem in this particular case is that it causes confusion _even when only native compiling_.

[phillipjohnston]

Like I said, I'm sure there are good reasons :).

Your second point is correct: this error came up and I didn't have a cross configuration applied.

[jpakkane]

For cross compilation projects specifying both c_std and build.c_std makes sense and is expected. You set the default values once and then don't have to think about it any more. It adds a bit of boilerplate, sure, but also enables the functionality mentioned above.

But what I'm concerned with is that this makes things more tricky and unexpected for the majority of people who never cross compile. Having a build.c_std that seemingly does nothing is perplexing.

[phillipjohnston]

If that's how it is as the project marches on - I'll adapt. If that makes sense for supporting the largest number of use cases, great.

The thing that confuses me is "it's expected". Because it's not "expected" based on the sudden breakage of previously working projects, the contents of the 0.51.0 release notes, nor the documentation.

[jpakkane]

It is mentioned, but perhaps not as explicitly as it could have been...

[phillipjohnston]

Well, now I feel like a jerk because I even re-read the release notes to make sure I didn't miss anything. That's what I get :).

[phillipjohnston]

So to summarize my understanding after the discussion:

• I deal with the way it is for cross-builds moving forward
• Something needs to be figured out for targets marked native: true when not cross-compiling
• I'll update some documentation to help clarify the points that confused me

[phillipjohnston]

Well, here's what I changed to:

project('Embedded Artistry libc',
    ['c', 'cpp'],
    default_options : [
        # build.* options affect native: true targets when cross-compiling
        'c_std=c11', 'build.c_std=c11',
        'cpp_std=c++17', 'build.cpp_std=c++17',
    ])

And then I get a warning:

WARNING: Unknown options: "build.c_std, build.cpp_std"

But the build still succeeds even with that warning.

[jpakkane]

I tested and got the same warning, but meson configure says that it has been properly set to c99.

[Ericson2314]

@jpakkane I don't quite understand your subproject example. How default options with subprojects normally work? Why is the cross stuff special?

@phillipjohnston I apologize for not writing a louder "this is a breaking change" changelog entry. You might want to change your comment to:

        # `build.*` options affect `native: true targets`
        # plain options affect `native: false` targets.

because there is no cross-specific behavior. The whole point of the build. change affecting native builds is knowing which targets get which args, without knowing whether it's a native or cross build. In other words, it's about making the meson.build file more predictable regardless what the end user does with it. I don't expect everyone to love this build. change, but I want to at least make clear what it's motivation is.

[Ericson2314]

Also weird with the warning. I looked at CoreData.set_options and didn't see how it could possibly have an effect if the warning is tripped.

[jpakkane]

ow default options with subprojects normally work? Why is the cross stuff special?

Suppose you have a project that builds a code generator tool and runs it. It is set up to work both as native and cross builds. It sets both c_std and build.c_std in its default options. Things work.

Then you have a different project that uses it as a subproject. It does not build anything that needs running, thus it sets only c_std in its default options, not build.c_std.

in the former case, both c_std and build.c_std are set and things work. In the latter case _only_ c_std is set because you can only set those options in the master project. Thus build.c_std is unset, leading to potential build failures. When this happens the person merely using the subproject gets very confused, because they don't know (or even care) that build.c_std exists, nor that it should be set to some specific value.

[Ericson2314]

@jpakkane OK I didn't know that subprojects default options were just thrown out. This doesn't really seem right to me? And I assume it can break things that have nothing to do with native vs cross?

Here are two other options:

• default_options are scoped per-project.
• The master project must set a c_std and build.c_std that is "at least as permissive" as the subproject's.

[jpakkane]

That is how it has always behaved. The point is that for existing options this is fully visible and explicit. Not so much for build. options because they are somewhat hidden. I have spent a lot of time thinking about this and I have not come up with a good way to split this up, but the following setup would be the most usable and would match (mostly) how things have worked thus far:

• When compiling natively, there is only one c_std (possibly also c_args) et al and it is used for all compilations
• When cross compiling, there is a c_std and build.c_std where the former is used for cross compiling and the latter for native: true compilations.

[Ericson2314]

@jpakkane

That is how it has always behaved. The point is that for existing options this is fully visible and explicit.

How it always worked is thus dynamic scope, which undermines the simplicity of the meson language. And I still don't think this is a cross specific issue. Say the subproject had a C++ component and set a build.cpp_std default option, and then the master project doesn't use C++. Same issue! The subproject has no idea what the master project will be, and the master subproject has no "mandatory arguments" or whatever it must tell the subproject, establishing a contract between them. [mandatory arguments could let the master project author know "oh, I need to pass some sort of cpp_std".]

...but the following setup would...

I still think there is no justification for changing the behavior of things when a cross-supporting project is built natively. That gets us right back to the messy situation we were in before where it's hard to write cross-supporting projects simply and correctly, because everything works different depending on whether the user happens to do a native or cross build.

However consider this. You could have native-only projects where all build systems == all host settings and native flags are prohibited. If a native-only project has on a cross-aware subproject, then the subproject gets host and build both set by the native project's combined build = host settings. Unlike what you wrote, "native-only" is an intrinsic property of the meson project, not the user-specified build settings (build and host machines, etc). It is static / lexical.

[scivision]

what should be done with PR #5560?

[Ericson2314]

@scivision that PR looks orthogonal. @dcbaker's suggestions look good so start with that.

[scivision]

OK good, thanks.

[Akaricchi]

Please remind me, why is the language standard even a configuration property again?

[Ericson2314]

It is a compiler flag, affects what is allowed, what is warning, etc.

[Akaricchi]

What I meant is why is it set at a global build-option level, rather than, say, per target or per project in the build rules.

[Ericson2314]

Oh! Yes I do agree we are shooting ourselves in the foot with the current treatment. (Did you see my previous comments about that?)

[scivision]

Yes particularly when meshing legacy and modern code from distinct groups, different -std may need to be applied. So far I have been able to workaround this by using target fortran_args and similar, counting on that the compiler prioritizes the rightmost argument.

[nirbheek]

*_std were per-project when we didn't have subprojects, but then everything became global when we added those. I think everyone still expects these options to be per-project (we have another issue about that) so we could move to that, but it's not something we can do in a stable release.

[jpakkane]

I still think there is no justification for changing the behavior of things when a cross-supporting project is built natively. That gets us right back to the messy situation we were in before where it's hard to write cross-supporting projects simply and correctly, because everything works different depending on whether the user happens to do a native or cross build.

Not really. A different way of saying what I am aiming for here is this:

When not cross building all build.XXX options are mandated to have the same value as the
corresponding XXX option.

Build definitions in files (flags for for-host and for-build targets etc) remain just as they are now. It's just that the _external options_ you can set from the outside are not duplicated. There are only one set of those.

Say the subproject had a C++ component and set a build.cpp_std default option, and then the master project doesn't use C++. Same issue! The subproject has no idea what the master project will be, and the master subproject has no "mandatory arguments" or whatever it must tell the subproject, establishing a contract between them. [mandatory arguments could let the master project author know "oh, I need to pass some sort of cpp_std".]

Setting up default values for new options like this could be added (it might even work like that already, don't remember) but it does not really work for build. variants for existing options. You can't really know when they should spring into existance. For example you could have a master project that does not define c_std at all but which does build cross stuff. When the project call is first evaluated it is impossible to know whether the master project will use build. options or not or whether they come from a subproject. It might even be the case that building of build target is conditional based on some option.

It can get even more complicated than this but the outcome of this is clear: if there is to be build.foo options in addition to plain foo option, they both must come into existance at the same time.

[TheQwertiest]

I think everyone still expects these options to be per-project (we have another issue about that) so we could move to that (...)

^ We sure do =)
Corresponding issue: https://github.com/mesonbuild/meson/issues/1889

[Ericson2314]

Build definitions in files (flags for for-host and for-build targets etc) remain just as they are now. It's just that the external options you can set from the outside are not duplicated. There are only one set of those.

Are you saying that they can be separately varied internally, but CLI flags and env vars initialize them both in lock step in native builds?

[Ericson2314]

hen the project call is first evaluated it is impossible to know whether the master project will use build. options or not or whether they come from a subproject.

Right now, subprojects only influence the master project with the global args functions, right? I'm suspicious of those two. Really anything that is global is highly non-compositional and liable to have confusing interactions.

[jpakkane]

Global might not be best of design ideas, but in my defence it was created before subprojects were invented.

That being said you can only add global arguments in the master project before any subproject is invoked. Trying to set them from subprojects or from the master project after a subproject call aborts immediately with a hard error.

[Ericson2314]

@jpakkane Sounds good, I don't mind to accuse you of wanting all of the status quo :) Glad to learn that mutating the options in those other places is banned too!