Meson: 0.55 considers SKIP in tap output as failed

[matthiasclasen]

GTK has a some tests that produce output like

1..227
...
ok 57 /FinalizeObject/GtkConstraint
ok 58 /FinalizeObject/GtkConstraintGuide
ok 59 /FinalizeObject/GtkConstraintLayoutChild # SKIP Skipping GtkConstraintLayoutChild
ok 60 /FinalizeObject/GtkConstraintLayout
ok 61 /FinalizeObject/GtkCssProvider
...
ok 227 /FinalizeObject/GtkWindowHandle

that is, 227 ok's and an exit code of 0.

And meson 0.55.0 on my local system says:

3/3 gtk:gtk+object / objects-finalize FAIL           11.85s (exit status 0)

If I shorten the test to leave out all the cases that emit a # SKIP comment, meson 0.55 is happy again.

0.54 in our ci thinks everything is just fine.

I also think that everything with our testsuite is just fine, and meson regressed.

[smcv]

Bisection says this regressed in 0c3bb153.

[ptomato]

Despite the fix being included in 0.55.1, this still doesn't seem to work. Any skipped tests still cause the test suite to fail. I believe this should be reopened.

[ptomato]

Despite the above, it seems to have been fixed eventually in 0.55.3, so :shrug:

[nirbheek]

Despite the above, it seems to have been fixed eventually in 0.55.3, so 🤷

Yes, the discussion happened on https://github.com/mesonbuild/meson/pull/7618.