Ava: Allow specifying custom reporters as an configuration option

[petetnt]

Currently I am piping the output to the reporter like specified in the docs:

package.json

  "scripts": {
    "test": "ava | tap-foo-reporter",
  }

This works great, but this has a caveat of not being able to override options when running tests from the command line:

npm test -- --verbose 

# -->
ava | tap-foo-reporter "--verbose"

I think there might be come command line magic that would replace the flags passed in to the correct place, but I was thinking that it would be nice to be able to specify the reporter in the ava configuration:

package.json

{ 
  "scripts": {
    "test": "ava",
  },
  "ava": {
    "...": "...",
    "reporter": "tap-foo-reporter"
  }
}

Specifying a reporter would make ava to pipe the output automagically to the said reporter (unless --watch is also passed, in which case the default behavior of not piping it in occurs).

This is pretty low priority and there might be some other caveats I am not aware of as of yet (and running ava --flags-here | tap-foo-reporter instead of npm test -- --flags-here is barely more work in most cases anyway) but something I thought that would be nice to have :cat2:.

[sindresorhus]

This works great, but this has a caveat of not being able to override options when running tests from the command line:

Your example is invalid. It should be ava --tap | tap-foo-reporter and --verbose is mutually exclusive with the --tap option, so passing --verbose when using --tap does nothing. Can you share an actual use-case?

[petetnt]

@sindresorhus sorry about that! Verbose was a bad example. Consider something not mutually exclusive, like title matching: Say I want to run all the tests that contain COUNTER_ADD in the title with --match

My package.json:

  "scripts": {
    "test": "ava | tap-spec",
  },
  "ava": {
    "files": [
      "src/js/__tests__/*.js"
    ],
    "source": [
      "**/*.{js,jsx}",
      "!dist/**/*"
    ],
    "failFast": true,
    "tap": true,
    "require": [
      "babel-register"
    ],
    "babel": "inherit"
  },

npm test -- --match="*COUNTER_ADD*"

Obviously runs ava | tap-spec "--match="*COUNTER_ADD*"" which just runs all the tests as expected. If I could just have the test script as ava and the reporter in the ava options then I could pass whatever options I want to and have it report back to me, without having to do npm test -- --options-here | tap-spec` every time (which like I said before is really not much of problem anyway...).

It would get tricky if I had to pass options for the reporter too, so something like reporter: ["tap-spec", {"exclusive-option-for-reporter": "true"}] would be nice too. But then again one could just alias the spec+options command to something shorter / easier to remember so it might be too much of overhead to achieve something in ava that one can work around easily.

TL;DR: I am lazy and I'd like to run all the things with npm test, with reporters and the ability to pass arguments to the test runner itself too.

[vadimdemedes]

Not sure we want an option for it, as we would need to handle output piping and spawning tap-* process ourselves.

Let's see what others think.

[novemberborn]

We're already moving away from allowing --tap output with watch mode. And .only will soon work best under watch mode, so that'll be affected too. Like --match these features are handy during development, not when running tests on a CI server.

It'll probably be good practice to set up a watch:test script that is used during development. It's easy enough to pass --match to an npm run watch:test invocation. TAP output can still be configured in the package.json and will be silently ignored in watch mode.

TL;DR I don't believe we need to support custom reporters. During development AVA's mini and --verbose reporters should be used, in watch mode. For CI purposes the --tap reporter can be piped external TAP reporters.

[petetnt]

@novemberborn Fair enough :+1: Closing this.

[bcherny]

@novemberborn My tests are asserting equality between long multiline strings, and I'd love to use a custom JSON diff reporter during watch mode. The test output is very hard to read with the built in reporters. Is there a way to configure this?

[novemberborn]

@bcherny the reporters discussed here do not impact the assertion messages. I think you already found a different issue which relates to that. Let's continue the discussion in Gitter.