Catch2: Having multiple unique test cases on the same line results in compilation errors

[BobLoblaw27813789132]

If you have a situation where there are multiple TEST_CASE() definitions on the same line (even though they have a unique name), there is a compilation error.

I created a very contrived example of this issue:

#define CATCH_CONFIG_MAIN
#include "catch.hpp"

using namespace std;

class my_adder_class {
public:
    virtual int add(int a, int b) = 0;
    virtual int sub(int a, int b) = 0;
};

class my_adder_imp1 : public my_adder_class
{
public:
    int add(int a, int b)
    {
        return a + b;
    }
    int sub(int a, int b)
    {
        return a - b;
    }
};

class my_adder_imp2 : public my_adder_class
{
public:
    int add(int a, int b)
    {
        return a + b;
    }
    int sub(int a, int b)
    {
        return a - b;
    }
};

my_adder_class* get_imp(string name)
{
    if (name == "my_adder_imp1")
        return new my_adder_imp1;
    else
        return new my_adder_imp2;
}

#define MYTESTCASE(myname, mytestname)\
TEST_CASE(myname "_" mytestname)\

#define GENERATE_TEST_ADDING_TWO_NUMBERS(class_imp)\
MYTESTCASE(class_imp, "add two numbers")\
{\
my_adder_class* imp = get_imp(class_imp);\
REQUIRE(imp->add(1, 2) == 3);\
}\


#define GENERATE_TEST_SUBTRACTING_TWO_NUMBERS(class_imp)\
MYTESTCASE(class_imp, "subtract two numbers")\
{\
my_adder_class* imp = get_imp(class_imp);\
REQUIRE(imp->sub(3, 2) == 1);\
}\


#define GENERATE_ALL_TESTS(class_imp)\
    GENERATE_TEST_ADDING_TWO_NUMBERS(class_imp)\
    GENERATE_TEST_SUBTRACTING_TWO_NUMBERS(class_imp)


// TESTS ARE GENERATED BELOW HERE
GENERATE_ALL_TESTS("my_adder_imp1")
GENERATE_ALL_TESTS("my_adder_imp2")

In Visual Studio 2015, if i try to compile this, I get the following errors:

error C2374: 'anonymous-namespace'::autoRegistrar73': redefinition; multiple initialization note: see declaration of 'anonymous-namespace'::autoRegistrar73'
error C2084: function 'void ____C_A_T_C_H____T_E_S_T____73(void)' already has a body
note: see previous definition of '____C_A_T_C_H____T_E_S_T____73'
error C2374: 'anonymous-namespace'::autoRegistrar74': redefinition; multiple initialization note: see declaration of 'anonymous-namespace'::autoRegistrar74'
error C2084: function 'void ____C_A_T_C_H____T_E_S_T____74(void)' already has a body
note: see previous definition of '____C_A_T_C_H____T_E_S_T____74'

The documentation states that the name of the TEST_CASE must be unique. All of the TEST_CASE() names that get generated are unique per the preprocessor logic. It would appear that the line number must also be unique since it plays a role in the generation of these test cases.

If I replace GENERATE_ALL_TESTS with the macro expanded it will successfully compile, since they are all on their own lines. Example:

// TESTS ARE GENERATED BELOW HERE
//GENERATE_ALL_TESTS("my_adder_imp1")
GENERATE_TEST_ADDING_TWO_NUMBERS("my_adder_imp1")
GENERATE_TEST_SUBTRACTING_TWO_NUMBERS("my_adder_imp1")
//GENERATE_ALL_TESTS("my_adder_imp2")
GENERATE_TEST_ADDING_TWO_NUMBERS("my_adder_imp2")
GENERATE_TEST_SUBTRACTING_TWO_NUMBERS("my_adder_imp2")

[lightmare]

Do you have latest Catch? It should be using __COUNTER__ instead of __LINE__.

In your particular case, since you're constructing test cases with macros per class name, you can work around the issue by wrapping them in namespace named after the class:

#define GENERATE_TEST_ADDING_TWO_NUMBERS(class_imp)\
namespace class_imp ## __test_ns {\
MYTESTCASE(#class_imp, "add two numbers")\
{\
class_imp imp;
REQUIRE(imp.add(1, 2) == 3);\
}}

GENERATE_ALL_TESTS(my_adder_imp1)

But I think you'd be better off doing less macro-fu. Saving a few keystrokes isn't worth the loss of clarity of actual test case bodies, imo.

namespace
{
template<typename T> void test_all_ops()
{
    SECTION("addition") {
        T imp;
        REQUIRE(imp.add(1, 2) == 3);
    }
    SECTION("subtraction") {
        T imp;
        REQUIRE(imp.sub(3, 2) == 1);
    }
}
}

TEST_CASE("impl1") { test_all_ops<my_adder_imp1>(); }
TEST_CASE("impl2") { test_all_ops<my_adder_imp2>(); }

[BobLoblaw27813789132]

Do you have latest Catch? It should be using __COUNTER__ instead of __LINE__.

I just pulled down the catch.hpp single header version which states:

Catch v1.5.6
Generated: 2016-06-09 19:20:41.460328

This is the latest release advertised on the main page. It does not appear to use __COUNTER__ and still has this issue.

But I think you'd be better off doing less macro-fu.

I am completely for avoiding macros. The reason I did it that way was because I needed to dynamically generate test methods for various implementations of class interfaces. I can see if I can work your template idea into what I have, thanks.

[philsquared]

1.5.6 definitely has the __COUNTER__ implementation - although it's not enabled for all compilers. You say you're using VS2015 so it _should_ work - but you could try #defineing CATCH_CONFIG_COUNTER

[BobLoblaw27813789132]

but you could try #defineing CATCH_CONFIG_COUNTER

For what its worth I tried that on 1.5.6 with VS2015 and that fixed it.

The reason for this is as follows (This seems like a bug):

I'm using the combined header file. The check to see whether the compiler supports __COUNTER__ is on line 284. My VS2015 passes the macro evaluation and so it defines CATCH_CONFIG_COUNTER. The problem is that the logic for setting INTERNAL_CATCH_UNIQUE_NAME is on line 65 (before the CATCH_CONFIG_COUNTER gets defined). This (catch_common.h) would need to be moved after line 284 for it to know about CATCH_CONFIG_COUNTER.

Because of this, the combined header file will always use __LINE__ instead of __COUNTER__ regardless of whether the compiler supports __COUNTER__.

[philsquared]

I believe the commit on 11th Jan fixes this issue. @BobLoblaw27813789132 I appreciate it was a while ago you reported this (and have a workaround) - but if it is still relevant to you and you are able would you be able to confirm that this is resolved for you without needing the CATCH_CONFIG_COUNTER #define?
Otherwise I'll close this in a while.

[englishb]

Just found this fix, and it helped me out. I have a less contrived scenario;

We have >100 test functions that each need to be exercised in different ways that are selectable via the tags at the command line, so I have a macro that defines four different test cases for a given function, calling it with different parameters and concatenating the function name with some strings to generate the name passed into TEST_CASE. Writing these test cases by hand with the correct tags and parameters had been very error prone, now we list each function once in a macro and it expands into multiple test cases.

I tested in MSVC2013; it was a problem with catch v1.5.4, and I updated to v1.9.3 and the issue is fixed.

[horenmar]

Since this has been resolved in current Catch, I am going to close this.