Cucumber-js: How to use cucumber-messages

[xuanzhaopeng]

Refer to the doc https://github.com/cucumber/cucumber-js/blob/master/docs/cli.md#Formats

The message doesn't work, but I believe it's because we don't have new release. So I build it locally, and it seems works properly when I do

cucumber-js --format message --parallel 3

And from code side, I saw all format now are using envelope instead of test-case-start extra, so I believe we can already write customized format in parallel execution mode, such as allure-cucumberjs

So I'd like to ask when we plan to release new version?

Thanks a lot

[davidjgoss]

There should be a 7.0.0 out quite soon, there are a few things left to fix up.

The current docs can be found here https://github.com/cucumber/cucumber-js/tree/6.x

[aslakhellesoy]

@davidjgoss Before we make a release I'd really like to verify that the messages produced by the message formatter can be consumed by tools like json-formatter and html-formatter.

The compatibility-kit aims to make this a little easier. The run-formatter script does this.

[davidjgoss]

@aslakhellesoy yeah, I’ve been meaning to have a play with that anyway so will do it soon.

Also based on call from the other day we should change the message formatter to output NDJSON rather than binary?

[aslakhellesoy]

Yes! We’re not in a high throughput low latency environment so the benefits of json (human readable) outweighs the marginal benefits of binary.

[aslakhellesoy]

Also, I’m releasing the whole monorepo today and tomorrow so there will be some major releases to upgrade

[xuanzhaopeng]

Hey @aslakhellesoy @davidjgoss , I built local version from master branch. I rewrite allure-cucumberjs to consume envelope , and build the allure report based on this.eventDataCollector.

In non-parallel mode, It can:
1) consuming envelop success
2) inject variable into World

In parallel mode, it can:
1) consuming envelop success
2) Cannot inject variable to World. because Formmater is working on Master, instead of Slave.

PS: here is the code for World injection:
https://github.com/allure-framework/allure-js/blob/master/packages/allure-cucumberjs/src/CucumberJSAllureReporter.ts#L84

// options is coming from the Formmater constructor: 
options.supportCodeLibrary.World.prototype.allure = this.allureInterface;

May I ask how could we inject variable to World in parallel mode from customized Formmater?

Thanks

[xuanzhaopeng]

In other words, Formmater cannot identify which slave send the message back, and the SLAVE_ID only exposed to each slave, Is it possible to add SLAVE_ID on TestCaseStarted message

[charlierudolph]

There should be a 7.0.0 out quite soon, there are a few things left to fix up.

@davidjgoss you please create an issue where we start tracking what tasks we want to complete before 7.0.0?

May I ask how could we inject variable to World in parallel mode from customized Formmater?

@xuanzhaopeng I don't see the reason we should allow this. To me formatters should not effect execution. What is the reason you need this? Maybe we could solve that in a different way.

In other words, Formmater cannot identify which slave send the message back, and the SLAVE_ID only exposed to each slave, Is it possible to add SLAVE_ID on TestCaseStarted message

Why does the formatter want to know what slave ran each test case? Adding the slave to the message is certainly possible, but I'd like to understand why you want it.

[xuanzhaopeng]

Hey @charlierudolph , for 1st point, I think it's very specific allure related issue, maybe we could discuss it in separated thread.

For adding slave id into message, is because:
When run test in parallel, Allure report can show the time slot like this(maybe also other reporter consumers):

So it can be achieved by allure.addLabel(LabelName.THREAD, slaveId), so if we could get Slave ID from TestCaseStarted message, this information could be easily used by allure, and show exact execution duration for each slave.

I made an simple test locally by using exist message proto(testCaseStarted.platform.cpu)

                testCaseStarted: {
                    attempt,
                    testCaseId: this.testCaseId,
                    id: this.currentTestCaseStartedId,
                    platform: {
                        cpu: process.env.CUCUMBER_SLAVE_ID || '-1'
                    }
                },

refer to code
https://github.com/cucumber/cucumber-js/blob/master/src/runtime/pickle_runner.ts#L306

And it works as the picture above when I do cucumber-js --parallel 3

[xuanzhaopeng]

Hey @charlierudolph , do you think we can put slave number inside of testCaseStarted Message in 7.0.0 release?