Catch2: catch_discover_tests() - cmake's ctest cannot find tests with commas

[svgsponer]

Description

I want to use catch_discover_tests() to automatically register test for ctest. For some of the test cases no test were detected. All of them contained a comma (",") and after removing them the tests are detected correctly.

Steps to reproduce

Create a TEST_CASE with a "," in it:

#define CATCH_CONFIG_MAIN  // This tells Catch to provide a main() - only do this in one cpp file
#include <catch2/catch.hpp>

TEST_CASE("Test case with a, in it", "[dummy]")
{
    REQUIRE(1 == 0);
}

Create CMakeLists.txt:

cmake_minimum_required(VERSION 3.8)

find_package(Catch2 REQUIRED)
enable_testing()

add_executable(mytest test.cpp)
target_link_libraries(mytest Catch2::Catch2)

include(CTest)
include(Catch)
catch_discover_tests(mytest)

Compile and run ctest:

cmake .
make
ctest --verbose

Produces output:

[...]
test 1
    Start 1: Test case with a, in it

1: Test command: /home/svgsponer/playground/catch_comma/mytest "Test case with a, in it"
1: Test timeout computed to be: 1500
1: ===============================================================================
1: No tests ran
1:
1/1 Test #1: Test case with a, in it ..........   Passed    0.00 sec

100% tests passed, 0 tests failed out of 1

Total Test time (real) =   0.01 sec

Expected output:

[...]
test 1
    Start 1: Test case with a, in it

1: Test command: /home/svgsponer/playground/catchbug/build/mytest "Test case with a, in it"
1: Test timeout computed to be: 1500
1:
1: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1: mytest is a Catch v2.2.3 host application.
1: Run with -? for options
1:
1: -------------------------------------------------------------------------------
1: Test case with a in it
1: -------------------------------------------------------------------------------
1: /home/svgsponer/playground/catchbug/test.cpp:4
1: ...............................................................................
1:
1: /home/svgsponer/playground/catchbug/test.cpp:6: FAILED:
1:   REQUIRE( 1 == 0 )
1:
1: ===============================================================================
1: test cases: 1 | 1 failed
1: assertions: 1 | 1 failed
1:
1/1 Test #1: Test case with a, in it ...........***Failed    0.00 sec

0% tests passed, 1 tests failed out of 1

Total Test time (real) =   0.00 sec

The following tests FAILED:
          1 - Test case with a, in it (Failed)
Errors while running CTest

I created a small repository with this example setup. Repo

Extra information

• Catch version: v2.2.3
• Operating System: Linux Mint 18.3 Sylvia
• Compiler+version: clang 6.0.0

[horenmar]

This is a problem with the idiosyncratic nature of Catch's test specs, where comma is parsed as an OR operator, rather than a literal comma. Thus, if you call catch like this: ./tests "test case with, comma", it won't find any tests, because it looks for 2 tests instead, one named "test case with" and the second named "comma".

However, you can escape the commas: ./tests "test case with\, comma", which will look for a test named "test case with, comma" and this is what the script should do to support them properly.

[horenmar]

Fixed in the linked PR.