Stryker: no coverage and error

[kmdrGroch]

Summary

I get an error (times a lot [])
(11272) ERROR log4js A worker log process hung up unexpectedly { Error: read ECONNRESET at TCP.onread (net.js:659:25) errno: 'ECONNRESET', code: 'ECONNRESET', syscall: 'read' } and report is bad (like it doesn't test good(?))

function add(a, b) {
    return a + b;
}
module.exports = add;

```js
const add = require("../src/add.js");
describe("add.js", () => {
it("2 + 4 should equal 6", () => {
expect(add(2, 4)).toBe(6);
});
});

### Command I run:
- node ./node_modules/stryker/bin/stryker run --basePath=C:\Users\19gro\Desktop\hi

### Stryker config

<!--- Please place your stryker config below. Feel free to change paths in the files and mutate arrays if you cannot share them. -->

```js
const basePath = process.argv[3].replace("--basePath=", "");
const path = require("path");

module.exports = config => {
    config.set({
        "coverageAnalysis": "perTest",
        "files": [
            path.join(basePath, "src/**/*.js"),
            path.join(basePath, "tests/**/*.js")
        ],
        "karma": {
            "config": {
                basePath,
                "browsers": [ "ChromeHeadless" ]
            },
            "configFile": "../../karma.conf.js",
            "project": "custom"
        },
        "mutate": [ path.join(basePath, "src/**/*.js") ],
        "mutator": "javascript",
        "packageManager": "npm",

        "reporter": [ "html", "clear-text" ],
        "testFramework": "jasmine",
        "testRunner": "karma"
    });
};

Karma config

module.exports = config => {
    config.set({
        "basePath": config.basePath || "/",
        "browsers": config.coverage ? [ "ChromeHeadless" ] : [ "Chrome" ],
        "client": { "clearContext": false },
        "colors": true,
        "coverageReporter": {
            "dir": "reports",
            "reporters": [
                {
                    "subdir": "coverage",
                    "type": "html"
                }
            ]
        },
        "failOnEmptyTestSuite": false,
        "files": [
            "./src/**/*.js",
            "./tests/**/*.spec.js"
        ],
        "frameworks": [ "jasmine" ],
        "preprocessors": {
            "./src/**/*.js": [ "webpack", "sourcemap" ],
            "./tests/**/*.spec.js" : [ "webpack", "sourcemap" ]
        },
        "reporters": config.coverage ? [ "progress", "coverage" ] : [ "progress", "kjhtml" ],
        "webpack": {
            "cache": true,
            "devtool": "inline-source-map",
            "module": {
                "rules": [
                    {
                        "enforce": "post",
                        "exclude": /node_modules|\.spec\.js$|tests/,
                        "test": /\.js$/,
                        "use": {
                            "loader": "istanbul-instrumenter-loader",
                            "options": { "esModules": true }
                        }
                    }
                ]
            }
        }
    });
};

Stryker environment

+-- stryker - current
+-- stryker-api - current

+-- [email protected]
+-- [email protected]

Your Environment

| software | version(s)
| ---------------- | -------
| node | 10.6.0
| npm | 6.1.0
| Operating System | win 10

Add stryker.log

stryker.log

[simondel]

Thanks for opening this issue! Could you try to downgrade to Karma 2.0.5? Karma 3.0.0 has been out for two days and we haven't tested our support her.

[kmdrGroch]

@simondel standard testing with Karma 3.0.0 seems to be working. But ill try with 2.0.5 too

[kmdrGroch]

with karma 2.0.5 does not work too but there is no errors at least

[kmdrGroch]

stryker.log
here you have log for 2.0.5

[simondel]

Could you open up C:/Users/19gro/Desktop/front-tools/reports/mutation/html/index.html

In your log file I see that you have one test and Stryker found two mutants

[kmdrGroch]

it does created 2 mutants but it seems it doesn't test them

function add(a, b) 0{
    return 1a + b;
}

first

function add(a, b) {}

second

function add(a, b) {
    return a - b;
}

state -> noCoverage

[simondel]

Awesome. This sounds like a small project. Would it be possible to share the sources with me? You could send them to [email protected] or create a repo on GitHub.

[kmdrGroch]

I'll send you it this evening I suppose or tomorrow, since on GitHub I have older version.

[simondel]

Sounds good 👍

[kmdrGroch]

@simondel sent

[simondel]

Thanks! With your example I'm able to run Stryker if I remove your basePath logic, set the basePath in your karma config to "basePath": config.basePath || "", and use the following Stryker config:

module.exports = config => {
    config.set({
        files: [
            'src/**/*',
            'tests/**/*',
            '*.js'
        ],
        "coverageAnalysis": "perTest",
        "karma": {
            "config": {
                "browsers": [ "ChromeHeadless" ]
            },
            "configFile": "karma.conf.js"
        },
        "mutate": [ "src/**/*.js" ],
        "mutator": "javascript",
        "reporter": [ "html", "clear-text" ],
        "testFramework": "jasmine",
        "testRunner": "karma",
    });
};

Note: The files array could probably be removed in your case since you're working with a git repo.

With the settings above, when I run stryker run from the command line I get the following output:

[kmdrGroch]

@simondel have you tried running tests from different folder? (not where all these node_modules are)

[kmdrGroch]

i have structure like -> front-tools/ (folder with all node_modules, configs, logic stuff etc) and hi/ in completely different directory. I want to run tests from in hi/, have reports there etc.
Strange is that it creates all temp files and results in front-tools/ not in hi/ - maybe that is problem too -> Stryker can't have basePath set(?) [sent another zip to you with "structure" of this]

[simondel]

We have an open issue to add the option to specify a project root: https://github.com/stryker-mutator/stryker/issues/650

Because your karma config always points to the hi/ folder in your use case, the tests that contain the mutants and/or are instrumented to collect code coverage are never executed since they are in front-tools/.stryker-tmp/sandboxXXXXXXXXXX.

This is unfortunately not a use case that we support right now.

[kmdrGroch]

@simondel do you plan add this feature in the nearest future?

[nicojs]

As long as configured paths are relative, we should be able to support it once #650 is solved, right @_simondel?

@kmdrGroch, what does this mean in karma configuration: "basePath": "/". The root of your disk?

[simondel]

@nicojs I think we would be able to support this after #650. The basepath "/" indeed points to the root of your disk.

@kmdrGroch I don't expect us to have the feature in the next month or so, but it is definitely something that is important to us and will added in the future.

[kmdrGroch]

@nicojs "/" returns current directory, so if it's launched in front-tools it should get front-tools directory. Root of the disc will be obtained with C:/; @simondel that's nice. I'll close this issue for now since it is "solved". If you could message me somehow after it is added, i would be grateful :)

[nicojs]

@kmdrGroch

"/" returns current directory,

Great! I'll add that to #650, we should support that as well.

We can send you a message, but you can also watch the issue itself. You'll get notified as soon as there is an update