Karma: Duplicate console log messages when conflicting reporters are defined

[eklundjoshua]

Description of the problem:

In our project we have both Mocha & Remap-Istanbul defined as reporters for our Karma Tests. It seems as though these two reporters both have similar functionality in that they both intercept console.log messages that are written out from the JavaScript so there are always two log messages for every console.log message that we write. I do not think there is a feature yet, but it would be neat to add functionality that basically disables console.log messages for one plugin but leaves them for another so that way you can have the functionality of both plugins but only allow 1 of them to output console.log messages.

Environment Details

Karma Version 0.13.22

Steps to reproduce the behaviour

1. Defined reporters: ["mocha", "coverage", "karma-remap-istanbul"] in the karma.conf.js for my angular app
2. Ran my tests
3. Console logs that were written in the code were duplicated for every test that had been executed.
4. Removed "karma-remap-istanbul" from the reporters list, reran the tests, and the duplication went away.
5. Added "karma-remap-istanbul" back to the reports list and removed "mocha", reran the tests, and the duplication went away.

[maksimr]

@eklundjoshua could you post karma config for karma-remap-istanbul?

Thanks!

[eklundjoshua]

@maksimr Thanks for the follow up! Here is the karma.conf.js that I'm currently using:

var path = require('path');
var webpackConfig = require('./webpack.config');
module.exports = function (config) {
    var _config = {
        // base path that will be used to resolve all patterns (eg. files, exclude)
        basePath: '',

        // frameworks to use
        // available frameworks: https://npmjs.org/browse/keyword/karma-adapter
        frameworks: ['jasmine'],

        // list of files / patterns to load in the browser
        files: [
            { pattern: './karma-shim.js', watched: false }
        ],

        // list of files to exclude
        exclude: [],

        // preprocess matching files before serving them to the browser
        // available preprocessors: https://npmjs.org/browse/keyword/karma-preprocessor
        preprocessors: {
            './karma-shim.js': ['coverage', 'webpack', 'sourcemap']
        },

        webpack: webpackConfig,

        webpackMiddleware: {
            // webpack-dev-middleware configuration
            // i. e.
            stats: 'errors-only',
            noInfo: true
        },

        coverageReporter: {
            dir: 'coverage/',
            reporters: [{
                type: 'json',
                dir: 'coverage',
                subdir: 'json',
                file: 'coverage-final.json'
            }, {
                type: 'html',
                dir: 'coverage',
                subdir: 'html'
            }]
        },

        remapIstanbulReporter: {
            src: 'coverage/json/coverage-final.json',
            reports: {
                lcovonly: 'coverage/json/lcov.info',
                html: 'coverage/html',
                'text': null
            },
            timeoutNotCreated: 1000, // default value
            timeoutNoMoreFiles: 1000 // default value
        },

        webpackServer: {
            noInfo: true // please don't spam the console when running in karma!
        },

        // test results reporter to use
        // possible values: 'dots', 'progress', 'mocha'
        // available reporters: https://npmjs.org/browse/keyword/karma-reporter
        reporters: ["mocha", "coverage", "karma-remap-istanbul", "html"],

        htmlReporter: {
            outputFile: 'reports/unit_tests/index.html',
            pageTitle: 'Unit Test Results',
            useCompactStyle: true,
            groupSuites: true
        },

        // web server port
        port: 9876,

        // enable / disable colors in the output (reporters and logs)
        colors: true,

        // level of logging
        // possible values: config.LOG_DISABLE || config.LOG_ERROR || config.LOG_WARN || config.LOG_INFO || config.LOG_DEBUG
        logLevel: config.LOG_INFO,

        // enable / disable watching file and executing tests whenever any file changes

        // start these browsers
        // available browser launchers: https://npmjs.org/browse/keyword/karma-launcher
        browsers: ['PhantomJS'], // you can also use Chrome

        // Continuous Integration mode
        // if true, Karma captures browsers, runs the tests and exits
        singleRun: true
    };

    config.set(_config);
};

[maksimr]

@eklundjoshua

From remap-instanbul

Try remove 'text': null from remapIstanbulReporter reporters

Thanks

[wesleycho]

I've seen this issue in my app as well, it turns out that it doesn't matter what reporters you are using with remap-istanbul - I was able to isolate it as a karma coverage reporter issue.

Edit: Correction, I misspoke - this is very clearly a karma-remap-istanbul issue, not karma. It does not matter what reporters you send to it either.

Edit 2: Digging further into it, this seems to occur if I use karma-remap-istanbul or karma-junit-reporter on top of coverage. This may be a bug within karma itself with how it handles reporting, or something about the combination of plugins.

Edit 3: This does not occur if I exclusively use karma-coverage and karma-remap-istanbul. Exclusively using karma-coverage and karma-junit-reporter causes no output to be shown. Using dots, karma-coverage, and karma-remap-istanbul causes the log to output one in the log color in the shell, and one in just pure text.

I suspect that there is a mutation happening in the mocha, dots, and progress reporters that bleed over in karma-coverage. Will dig into the reporter area closer to understand how the system works & determine which plugin is the true culprit here.

[wesleycho]

After doing some chasing around, I am mostly convinced this is because karma-remap-istanbul is using baseReporterDecorator, so when writes are triggered in the base reporter, it propagates to the karma-mocha-reporter and karma-remap-istanbul.

I would recommend for fixing this that the writing of the logs be delegated to another reporter type that is not base, and have dots and progress use that. This would be a breaking change, and would need to be scheduled for a major release if this approach is adopted.

[eklundjoshua]

Thank you @maksimr. I tried your recommendation but still saw the duplication of logs. Thank you @wesleycho for your detailed findings as well!

[wesleycho]

Just so a solution is out in the open, we could either move to create another reporter type that can be decorated around, or we can try making a mixin pattern of sorts to do this. A mixin pattern would probably be more flexible than an inheritance pattern here, and not create problems in case we missed other features of the reporter we missed.

[JPMardelli]

I've got similar issues when running with the karma coverage reporter and the karma mocha reporter. Any of the config options to turn off console reporting do it for both, can't find a way to turn one of and not the other without rewriting them. Wish there was an option for which reporter to send console output to. Also unrelated, but I don't understand why the coverage reporter doesn't respect the color flag for console reporting.

[wesleycho]

Alternatively, we could add a new hook inside karma for running actions after reporters, since some of the things being done by reporters like the karma-junit-reporter and karma-remap-istanbul are actually doing post test processing. This would minimize pain to other reporters, and the reporters causing the multiple logging can then be refactored to avoid this. Maybe a postReporters config option that takes an array of plugins to run after the reporters are finished running.

Thoughts on this @dignifiedquire ?

[dignifiedquire]

The reporters really need some work, as we have already seen in https://github.com/karma-runner/karma/pull/2397 and https://github.com/karma-runner/karma/issues/1411. I am okay with creating a breaking change if we can figure out a good solution that gets rid of the varying issues and makes it easy and clear on how to correctly extend and add reporters.

[ahacke]

Any news regarding this issue? It is labeled as discussion. But nothing happened for nearly 8 month.

[corinna000]

I'm here as an Angular developer. The Angular CLI by default configures three reporters: one which is built-in to the Angular build setup (needed for clear messages due to webpack), one for Chrome integration (kjhtml), and one is the base progress reporter. Thus, any time there's an error in the test, the console logs three times. Unfortunately, this makes the console output almost useless as trying to figure out which error to read and whether a new error has appeared causes me to go cross-eyed.

I've worked around this for now by setting reporters: [/*'progress', 'kjhtml'*/], but this doesn't seem like the right long-term fix. Ideally, I'd be able to configure each reporter individually to suppress console output.

@dignifiedquire do you still have an opinion on the issue?

[jhildenbiddle]

I am also running into this issue using karma-mocha-reporter and karma-coverage-istanbul-reporter.

My intention is to have the mocha reporter display test results in the console while the coverage reporter generates lcov+html files and displays a text-summary in the console. This works great except for the fact that the coverage reporter displays duplicate test results.

I expected to find an option in the coverage reporter for disabling test results in the console, but according to the author this is a karma issue that he cannot work around.

Thoughts @vojtajina or @dignifiedquire?

Thanks!

[erwindemoel]

Any updates or a solution to this problem? I'm also interested because we have the same problem.