Cucumber-js: How will parallel option work with event protocol?

[kozhevnikov]

Custom formatters that only hook on test-run-finished like JSON formatter work as intended when parallelised but formatters that use earlier events like test-(step|case)-started and are ordered have their output logged out of sequence invalidating the output document structure. Is there anything planned to address that? May be buffer the output on per worker bases and log it all at once on master once completed?

[aslakhellesoy]

I think each event should contain enough information to be able to link it to previous events rather than buffering.

[kozhevnikov]

Not sure how that would work if only 'end' events are hooked as it would miss beginning of test's output. To use pretty formatter and IntelliJ-based editors' test runners as examples, they work as follows:

1. Log 'opening' statement on start event, in case of pretty -- Scenario: Foo, in case of IntelliJ -- TeamCity-formatted comment #teamcity[testStarted]
2. Run the test that prints its own logs after formatter's initial output
3. Log closing statement on end event, pass/fail summary or ##teamcity[testFinished]

The order is important as to have opening statement before anything that the test logs and closing statement afterwards and no other test should log anything in parallel to that or they'll mix up which output belongs to which test. Buffering would seem to take care of that as each output will be isolated and logged on master in atomic operation.

[charlierudolph]

We can pass a isParallel option to custom formatters to let them know that outputting things on test-case-started / test-step events will get jumbled and thus they should only output on test-case-finished events.

[smzelek]

We need more discussion on this issue. Currently, using any formatter at all has a risk of crashing the cucumber runner due to I/O conflicts while trying to write to a terminal. The progress bar formatter would be ideal for the parallel runner, if it only moved the bar along and then reported a summary at the end.

[aslakhellesoy]

The plan is to transition to standalone formatters that consume a stream of messages from cucumber.

See roadmap and dots-formatter and pretty-formatter (WIP) in the monorepo

[vBrestkin]

We need this feature very much!!!

[yevgen-getalo]

Hello! Is there any chance to run cucumber js in parallel with allure reporter? When I try to run with cucumber-js --parallel 2 -t @debug --format reporter.js:./dummy.txt I just got TypeError: Cannot read property 'sourceLocation' of undefined
Thank you in advance!

[mikejoo]

Same issue as @yevgen-getalo here.

[davidjgoss]

Closing as v7.0.0 uses the new messages protocol mentioned above, where $THING_started and $THING_finished events can be reliably be tied together via ids. The eventDataCollector.query object (an instance of @cucumber/query) can help with some of this.