Stryker: Specify lines to mutate

[Vitaly-Rudenko]

Is your feature request related to a problem? Please describe.
I would like to be able to specify certain lines to mutate. This is necessary to automatically test changes in pull requests.

Describe the solution you'd like
-m some/file.js:1-5;8-12,some/other/file.js:21,30-40 would mutate only 1-5, 8-12 lines of some/file.js, and 21, 30-40 lines of some/other/file.js

[simondel]

Isn’t what you’re looking for more in the lines of incremental analysis? #322

[Vitaly-Rudenko]

I'm not sure. I need this to only perform mutation testing for specific lines that have been changed in a specific Pull Request. So I have list of files with corresponding lists of lines that have been changed. I want to provide that into the Stryker so it can check only specified lines of code.

[Vitaly-Rudenko]

I suppose "Incremental analysis" issue implies that the feature I describe exists.

[Djaler]

I'm not sure. I need this to only perform mutation testing for specific lines that have been changed in a specific Pull Request. So I have list of files with corresponding lists of lines that have been changed. I want to provide that into the Stryker so it can check only specified lines of code.

But what if your PR changes test?

[Vitaly-Rudenko]

I need to run mutation test using all PR's specs, but only mutate selected lines of specified files.
Is this what you're asking?

I just need to check whether those lines are covered with tests.

[Garethp]

I'd also really like this feature. My use-case is similar to incremental analysis, but more straightforward. I'm building a IntelliJ plugin for Stryker, and I'm looking to implement a Re-run failed mutants feature, similar to how you can rerun only failed Jest/Mocha tests. Knowing what mutants failed in the last test run, it'd be easy to just pass in the info needed to identify which mutants I want to run again.

Though the problem sounds the same as incremental analysis (being able to skip mutants you know are dead), I think the end user use-case is different (being able to only check changed code vs. being able to target specific mutants for specific reasons).

If there are Plugin API hooks that I could use to filter out mutants once they've been detected but before being finalised, I'd be happy to write a Plugin to take care of this feature

[Garethp]

One example for an end-user where specifying lines to mutate would be better than incremental analysis would be a use-case where you've got your CI build telling you that lines 23 and 55 have surviving mutants, so you checkout master and want to run just those two lines through the mutator. No need to rebuild the incremental analysis when changing branches, just specify those two lines and run

[Vitaly-Rudenko]

Yes, that's exactly what I meant. This feature will be very useful for CI and scripts.

[nicojs]

Thanks for your feedback everyone.

If there are Plugin API hooks that I could use to filter out mutants once they've been detected but before being finalized, I'd be happy to write a Plugin to take care of this feature

We're busy with a complete overhaul of the code with #1514 . I think the plugin you want to request is similar to the type-checker plugin I've theorized there. I think we could make a more generic plugin that can mark mutants in any number of MutantStates (where we can also add Ignored, which is what you're looking for). The use case seems similar enough. Maybe we can rename it as a CheckerPlugin? @simondel what do you think?

[Garethp]

@nicojs So, you're thinking of having a generic plugin that would allow for various forms of Mutant pre-checks (for lack of a better term off the top of my head), where you could have a pre-check for type compilation (that would be nice, being able to check for compilation errors before hitting the testRunner) or a pre-check for user excluded mutants and one for more context based filtering (for example, checking if a mutant is the dependants argument in a React Hook, or whatever else you might not want to mutate in your Domain).

I think that's pretty cool, it would allow a lot of flexibility for people to write plugins to help Stryker be more suited to their particular domain.

[nicojs]

I've opened #2240 for a checker API. Here you can filter mutants before they are tested in a test runner.

However, the check api will have no knowledge of syntax. So checking if a mutant is the dependants argument in a React Hook is probably not possible (without again parsing the file). It's probably better to exclude those mutants, see #1472.

We're planning exclude functionality after the migration to mutation switching. No time line I can give at the moment unfortunately, but work is progressing.

[Vitaly-Rudenko]

Thanks for the update! Is there anything I could help with?

[nicojs]

@Vitaly-Rudenko you can help by reviewing the checker API as it stands now. Would allowing you to implement a CheckerPlugin solve your issue?

[Vitaly-Rudenko]

@Vitaly-Rudenko you can help by reviewing the checker API as it stands now. Would allowing you to implement a CheckerPlugin solve your issue?

Yes, I'll review the PR asap. I'm not an expert in the stryker's source code though. But providing a simple API for implementing a custom checker would solve my issue, as I'm willing to create a plugin that suits my needs.

Thanks!