Catch2: Access command line arguments from test cases

[RPGillespie6]

Description

I am running into the problem described here:
http://stackoverflow.com/questions/25211110/find-external-test-file-for-unit-test-by-relative-path-c-cmake-guest

I've read up how to specify your own main with Catch, but it doesn't seem like there is a good way to get the command line arguments into the actual test cases. So far this is what I have come up with:

#define CATCH_CONFIG_RUNNER
#include "catch.hpp"
#include <string>

std::string file_path; //TEST_CASES can access via global variable

int main( int argc, char* argv[] )
{
  Catch::Session session; // There must be exactly one instance

  file_path = string(argv[1]);

  int numFailed = session.run();
  // Note that on unices only the lower 8 bits are usually used, clamping
  // the return value to 255 prevents false negative when some multiple
  // of 256 tests has failed
  return ( numFailed < 0xff ? numFailed : 0xff );
}

Is there a better way I could be doing this?

[philsquared]

Storing your config data in globals is pretty much the only way to do it at the moment.
However you have another problem - you're not passing the rest of the command line on to Catch - so there's no way to do anything else with it (e.g. run specific tests or tags).

Working around that, currently, is tricky - because you'd have to pass on a char* array to run() that doesn't contain the filename (but still has the first argument - the exe name - as well as any arguments from position 2 on). It can be done but it depends if that's what you really want to do?

[RPGillespie6]

So basically I would have to extract the information I care about, and then build a fake argv and argc with my custom argument removed to pass on to Catch?

It seems like it would be nice if the argument parsing functionality were completely decoupled from the rest of Catch's logic similar to:

#define CATCH_CONFIG_RUNNER
#include "catch.hpp"
#include <string>

std::string file_path; //TEST_CASES can access via global variable

int main( int argc, char* argv[] )
{
  Catch::ArgumentParser parser;
  parser.addArgument("-a");   // add some custom arguments
  Catch::Options options = parser.parse(argc, argv);

  file_path = options["a"]; //Alternatively, you could make `options` globally accessible via static member variable so you could access it in TEST_CASE. i.e. Catch::options

  int result = Catch::Session().run(options);
  return ( result < 0xff ? result : 0xff );
}

Are you using a home-brew argument parser or an existing one like https://github.com/jarro2783/cxxopts?

[philsquared]

Yes I agree it needs to be more composable - and the current situation is not ideal.
It's a "home-brew" argument parser that I spun out into it's own project: https://github.com/philsquared/Clara - one of the aims there was to make it easier to work with but I just haven't had time to get back to that yet (and there are some interesting challenges - e.g. if you compose parsers and more than one part wants to use positional arguments, who gets them?).
At this point I'm deferring it to when Catch2 gets forked, and becomes C++11, then I'll make Clara C++11 too and that will clean up a lot and make things easier.

So it's all on the cards, but not in the immediate future, so I apologise that the current situation is not ideal.

[horenmar]

@RPGillespie6 If you can guarantee that your own config is always last N arguments, you can just set last N argv indices to nullptr and decrease argc by N, which is relatively painless.

[philsquared]

Ah yes, much easier at the end - good idea :-)

[RPGillespie6]

@horenmar I like that solution - thanks!

if you compose parsers and more than one part wants to use positional arguments, who gets them?).

What do you mean "who gets them"? Positional arguments should have a name attached that anyone can access:

Catch::Options options; //Global options

int main( int argc, char* argv[] )
{
  Catch::ArgumentParser parser;
  parser.addArgument("positional1");   // add first positional argument
  parser.addArgument("-o", "output_file");   // add non-positional argument
  parser.addArgument("positional2");   // add second positional argument
  options = parser.parse(argc, argv);

 // From this point on, everyone has global access to all arguments with options["name"] 
 int result = Catch::Session().run(options);
}

Maybe I'm being unrealistic by wanting my argument parsing experience to be like Python's?

At any rate, the workaround outlined by @horenmar is great and we can close the issue unless you want to keep it open for conscience nagging reasons :P.

[philsquared]

giving positional arguments to all parsers is one possibility. Maybe the best option - that's something I still need to think about (including looking at what other implementations do).

Anyway, I'm happy to close this - we have other tickets relating to improved CL parsing

[lightmare]

It seems like it would be nice if the argument parsing functionality were completely decoupled from the rest of Catch's logic similar to:

It kinda is, there's only one bit missing: #381
Then you could define additional options, like here for example: https://github.com/mapnik/mapnik/blob/094994ef540353b846490bf5c0dd127d00740dcb/test/unit/run.cpp#L26-L38

[VioletGiraffe]

It would be nice if the reference documentation (perhaps, the Command Line section?) demonstrated how to add custom command line options and pass them on to tests. So far I'm not finding any examples of how to it properly.

[philsquared]

Hi @VioletGiraffe - this (closed) thread is outdated. Since then I followed through on the Clara rewrite, with an emphasis on being compassable and extendible. While it could do with a bit more documentation, there should be enough here to get you going:

https://github.com/catchorg/Catch2/blob/master/docs/own-main.md#adding-your-own-command-line-options

[VioletGiraffe]

Oh! So I was right to say that I couln't find any info rather than to say there is no info :) Thanks (and sorry for poking into this old discussion).

[philsquared]

No problem :-)