Karma: v6 CLI arg verification breaks extra args parsing

[longlho]

Hello, currently karma CLI doesn't allow explicit test files as arg so we're passing them at the end and parse them out in karma.conf.js and v6 breaks it so I'm essentially asking if:

1. Karma CLI can support explicit test files passed in as args OR
2. Make strict mode be optional

I understand that this is not previously a feature but a lot of stack overflow answers seem to point to this mechanism being the workaround to further modify karma conf using CLI arg.

The way we're calling it is essentially karma start karma.conf.js [testFiles...]

Thanks!

[johnjbarton]

We hit the same issue.

I wonder if we can use -- testFiles and change our karma.conf.js to look in process.argv._?
https://github.com/yargs/yargs/blob/HEAD/docs/tricks.md#stop-parsing

[devoto13]

@johnjbarton Good idea! We can both retain strict validation of Karma options and allow users to pass arbitrary custom options. The only concern is that we already use this feature for clientArgs in karma run command, so they will conflict. Need to think a bit about how we're going to handle it.

[devoto13]

So I think the best solution would be to introduce --client-args flag for karma run command, then add a warning that the behavior of karma run -- my-arg form is going to change and point to the new --client-args flag. Allow passing arbitrary arguments after -- on any karma command. If this sounds like a good plan, I'll work on the PR in the coming days.

[paulie4]

This also broke the main feature of karma-jasmine-spec-tags, see their Issue 12.

[devoto13]

@paulie4 Thanks for sharing!

Looking at this library I start to think that maybe we should provide an API for a plugin to register their CLI arguments. This way there will be one standard mechanism and as a bonus, we can include parameters provided by plugins in the karma --help output. Which will be a neat feature.

The solution from https://github.com/karma-runner/karma/issues/3625#issuecomment-766438809 will not work for karma-jasmine-spec-tags without extra changes on their side. At this point, I lean towards temporarily reverting the strict parameters validation and gathering more feedback on the custom parameter use cases. So we can design a mechanism, which works for everybody.

[longlho]

Agree. Seems like a big change w/o some sort of RFC.

[matz3]

karma-ui5 is another plugin which made use of the feature to pass CLI args to override config options (e.g. --ui5.url, see https://github.com/SAP/karma-ui5#url).