Cucumber-js: Time for a release? `message` formatter doesn't work in latest but works on `master`

[sroze]

Hi there,

In the latest release, the message formatter doesn't work, unfortunately. It does in master, which is great news. Would it be a good time for a release? :)

Thank you 💜

[aslakhellesoy]

Hi @sroze - have you tested the generated messages with any consumers? If so, which ones?

I'd like to make sure the generated output works with the following:

• html-formatter
• json-formatter

@vincent-psarga - can we update the cck to validate this as part of the monorepo build?

[aslakhellesoy]

The following should be upgraded before cutting a release:

• "cucumber-expressions": "^8.3.0" -> "@cucumber/cucumber-expressions": "^10.2.0"
• "cucumber-messages": "^8.0.0" -> "@cucumber/messages": "^12.1.1"
• "cucumber-tag-expressions": "^2.0.3" -> "@cucumber/tag-expressions": "^3.0.0"
• "gherkin": "^9.0.0" -> "@cucumber/gherkin": "^13.0.0"

If a release were to be made without these upgrades, the messages produced by the message formatter would be incompatible with the latest releases of cucumber-html, json-formatter and anything else relying on the latest version of @cucumber/messages. And that wouldn't be useful for anybody I think.

[sroze]

@aslakhellesoy I'm actually using them with my own parser to integrate them in an auto-generated documentation (in a plugin for mkdocs) so I'm using --format message:output.ndjson and reading that output.ndjson file. So far so good, much better than the previous JSON output I must say 👌

[vincent-psarga]

@vincent-psarga - can we update the cck to validate this as part of the monorepo build?

Just to be sure I understand this correctly: run the latest release xxx-formatter against the release of messages to ensure we do not skip a release of the formatters ?
That definitly makes sense. Although I don't know if that should be part of the CCK or a second CI job for each formatter.

[aslakhellesoy]

@sroze very cool! Keep in mind that there have been some backwards incompatible changes in the @cucumber/messages library from 8.0.0 to 12.1.1. It has also been renamed. Any library depending on it will have to be updated too.

The other Cucumber implementations (JVM/Java and Ruby) use the latest versions of the messages library, so your tool wouldn't be able to consume those.

@sroze do you have a link to your project?

[aslakhellesoy]

@vincent-psarga I was suggesting that we could verify that cucumber.js is compliant with our cck. We'd have to implement step definitions, run cucumber.js with the message formatter, and then check if both json-formatter and html-formatter can consume those messages.

Aren't we doing this already for cucumber-ruby and cucumber-jvm?

[vincent-psarga]

Ah ok I did not see we were in the cucumber-js repo (that's the problem when checking notifications on the phone ...).

I guess it'd be doable, but I'm not sure we should do it in the monorepo itself, more in this repository. For ruby, we have two validations ran against the CCK:

• one done in cucumber-ruby, which validates cucumber-ruby master against the latest release of messages (but uses the master of json-formatter and html-formatter)
• one done in the monorepo, which uses the current master of messages and the latest release of cucumber-ruby. It had to be disabled as there were discrepancies between what the supported messages version of cucumber-ruby (it could be re-enabled but would have to be disabled again after the next release of messages)

And as far as I recall, the JS steps are already implemented for producing the golden NDJson with fake-cucumber, so this part is almost done (only the imports may have to be updated)

[davidjgoss]

@aslakhellesoy I'm working on #1318 right now which adds validation against CCK and updates to latest gherkin, messages etc. Just working through the remaining gaps, not long left - aiming to finish in next couple of days. I think we'll be in good shape to release 7.0.0 after that.

I wanted to check your intent with the javascript CCK package actually - I noticed it was published to npm under @cucumber/compatibility-kit a while ago but hasn't been updated in a while. For now I am just copying files from the monorepo to work against, but having the CCK features and message fixtures as a dependency would seem like a nice way to do it.

[sroze]

Thank you @davidjgoss, looking forward to being able to use the new version. While this is being done, this is what I have to do in order to use the master version of cucumber-js. The main challenge is that every time I run npm i again, npm complains that node_modules/cucumber is a Git repository. Do you know a better way? 🤔

diff --git a/package.json b/package.json
index e78cd20..8702cb7 100644
--- a/package.json
+++ b/package.json
@@ -30,7 +30,8 @@
     "typeorm": "ts-node -r tsconfig-paths/register ./node_modules/typeorm/cli.js -f src/orm-migratio
ns-config.ts",
     "typeorm:migrate": "ts-node -r tsconfig-paths/register ./node_modules/typeorm/cli.js -f src/orm-
migrations-config.ts migration:run",
     "typeorm:revert": "ts-node -r tsconfig-paths/register ./node_modules/typeorm/cli.js -f src/orm-m
igrations-config.ts migration:revert",
-    "pod:migrate": "node ./node_modules/typeorm/cli.js -f dist/orm-migrations-config.js migration:run"
+    "pod:migrate": "node ./node_modules/typeorm/cli.js -f dist/orm-migrations-config.js migration:run",
+    "postinstall": "cd node_modules && mv cucumber cucumber-old && git clone https://github.com/cucumber/cucumber-js && mv cucumber-js cucumber && cd cucumber && npm i && npm run build-local && cd ../.."
   },
   "dependencies": {
@@ -86,7 +87,7 @@
     "@typescript-eslint/eslint-plugin": "^3.4.0",
     "@typescript-eslint/parser": "^3.4.0",
     "common-tags": "^1.8.0",
-    "cucumber": "^6.0.5",
+    "cucumber": "cucumber/cucumber-js#master",
     "cucumber-pretty": "^6.0.0",
     "cucumber-tsflow": "^3.2.0",
     "eslint": "^7.3.1",

[sroze]

May I be annoying and chase on the following to cut a release?

1. ✅ @aslakhellesoy the dependencies have been upgraded indeed.
2. ✅ @davidjgoss's (massive) work on CCK has been merged (which rightly identified the testResult -> testStepResult issue)

[aslakhellesoy]

We actually did cut a release @sroze. Try out @cucumber/cucumber - this is a prerelease. See migration guide for how to update.

[sroze]

Oh great, I'll give it a try, thank you! Didn't know because no tag was pushed 🙃

[aslakhellesoy]

Thanks for the heads up @sroze - I had indeed forgotten to push the tag. It's up now.

Any thoughts about a package like this to help more people discover the new version?

• https://github.com/cucumber/cucumber-js/issues/1440
• https://github.com/cucumber/cucumber-js-package-upgrade/

[davidjgoss]

7.0.0 is now released, see the changelog for migration instructions. Thanks!