Meson: custom_target replaces back-slashes with slashes in the command arguments

[keszybz]

Not sure about the details, but at least \t becomes /t and \\ becomes \:

project('meson3', 'c')

printf = find_program('printf')
custom_target('foo',
          output : 'foo',
          command : [printf, 'a\n\tb\nccc\n'],
          capture : true)

custom_target('foo2',
          output : 'foo2',
          command : [printf, 'a\\n\\tb\\nccc\n'],
          capture : true)

$ ninja-build && tail foo foo2
==> foo <==
a
/tb
ccc

==> foo2 <==
a
/tb
ccc

This makes it quite hard to write regexps and printf lines and such.

1512 is probably related.

[keszybz]

... or maybe meson should do a single layer of unescaping by itself, like python, so that it's possible to embed newlines and tabs in strings. Meaning that if I want to pass "\t\n\" to the program, I should write "\\t\\n\\".

[QuLogic]

See #737.

[jpakkane]

As the link indicates this is a thorny problem because we need to support Windows natively. Sadly there does not seem to be a way to pass all possible characters through both shell and cmd.exe (and other backends) that would work reliably.

The recommended way is to instead write your pipelines in standalone scripts and running those from Meson. Granted this is some amount of work beforehand but it is beneficial in several ways:

• different programming languages (and thus paradigms) are cleanly separated in separate files
• standalone scripts are a _lot_ easier to edit and debug
• no need to think about quoting quotes and all that terror when you have characters that have special meaning in Meson, shell _and_ awk (for example)
• portability becomes easier (this is not such a big issue in Linux only projects, but most are cross platform)
• standalone scripts can be converted to Python one by one, and once that is done you no longer have a build dependency on Unix userland and can compile your project with nothing but Python 3 + Ninja

[nirbheek]

Honestly the piece of code that does that is wrong and we should remove it, but we just need to figure out why GStreamer needs it on Windows and fix whatever is broken in that regard.

Some of that will be on the GStreamer end, but I suspect some of that is bugs in Meson too.

[jpakkane]

To clarify: yes, we _should_ pass all special characters through all shells and systems properly. Not doing that is a bug. However _in practice_ this is is a really thorny problem and it is unlikely to ever work 100% reliably. Standalone scripts that are just invoked are always reliable, can be tested in isolation and are easy to convert to a different language (e.g. from Awk to Python or what have you) should the need arise.

[johandc]

This is also a problem when using the redirect symbol on Unix >.

Here is an example:
custom_target('lss', output: lss_name, command: [find_program('arm-none-eabi-objdump'), '-h', '-S', exec.full_path(), '>', lss_name], depends: exec, build_by_default: true)

This will call arm-none-eabi-gcc with quotes around the redirect symbol '>'.

/usr/bin/arm-none-eabi-objdump -h -S /xx/yy/zz '>' zz.lss

[jpakkane]

You should use the capture keyword instead.

[jpakkane]

Also, don't do exec.full_path() and depends : exec. Just put the exec object in the command list and Meson will take care of everything for you.

[nirbheek]

I think we should try removing that snippet and see what breaks. It will only get harder to remove it as time goes on.

[nirbheek]

I'll get to the bottom of why this was needed in gstreamer this week. FTR, the first step is setting up cerbero which is a massive yak-shave unless you've done it before.

[rulatir]

The recommended way is to instead write your pipelines in standalone scripts and running those from Meson.

So, I already knew Meson required wrappers to be written:

• whenever you need to execute a pipeline of several binaries
• whenever the build tool must be run from a specific directory

Now this list is extended with:

• whenever the command contains backslashes

This is outright ridiculous. I think Meson should come with a BIG BOLD CAPITALIZED FLASHING SIDE SCROLLING WARNING that using it requires writing TONS of code, and it can only possibly make sense for your project if the amount of code you WILL have to write (and debug, and test) in order to accommodate all the quirks and deficiencies and opinionatednesses of Meson is less than say 15% of the actual codebase you want to build with it.