Cabal: `cabal new-test` fails if there is no tests

[vrom911]

If my package has no test stanza and I run cabal new-test I get:

cabal: Cannot run tests for the target '' which refers to the package
mypackage-0.0.0 because it does not contain any test suites.

with exit code 1.

Though it would be much friendly if this command would succeed with exit code 0 but produce some descriptive message. For example, it would help to make a uniform config for CI (ex: Travis).

My version of cabal:

$ cabal --version
cabal-install version 2.3.0.0
compiled using version 2.3.0.0 of the Cabal library

[m-renaud]

Would hiding this behaviour behind a flag be an acceptable solution? I imagine that changing this behaviour for the default could very well break existing usages. I can see situations where both return values could be useful though, so it seems reasonable to support.

Maybe something like --allow-no-test-suites or something like that.

[vrom911]

@m-renaud, yes, I agree that this is a reasonable solution!

[fgaz]

@m-renaud I'd like the opposite: --disallow-no-test-suites, since the identity of test results is success

edit: maybe with a warning/notice about the flag

[m-renaud]

@fgaz I'm a slight -1 on that name since it's a double negative which is confusing to begin with. It would mean to trigger the new behaviour would need to pass --disallow-no-test-suites=False which is just mind bending ;P

Also, for optional flags the usual default value is False, but this one would require a default value of True which is also inconsistent.

[m-renaud]

@23Skidoo @fgaz I'm +1 on changing the default behaviour to "success when no tests", in which case I agree, but I'd prefer something like --fail-when-no-test-suites since is avoids the double negative.

[fgaz]

Yeah, the name I suggested is horrible :)

Name it whatever you want, I just want it to default to success on no tests

[vrom911]

I also like this suggestion more as it makes more sense that "no tests" is the positive result of the tests. I'm just happy with any solution at the moment, cause I need to set up a bunch of ifs for Cabal to overcome this issue on CI.

[fgaz]

Fixed by #5768

[m-renaud]

For posterity, the new default behaviour for cabal v2-test is to _SUCCEED_ if there are no test suites, if you want the command to fail if no test suites are discovered then you should pass --test-fail-when-no-test-suites.

@vrom911 you should be able to remove the ifs from your CI once this change is released.