Angular-cli: ng test --code-coverage in 6.1 improperly detecting branches

[jhage-accesso]

Bug Report or Feature Request (mark with an x)

- [ X ] bug report -> please search issues before submitting
- [ ] feature request

Command (mark with an x)

- [ ] new
- [ ] build
- [ ] serve
- [ X ] test
- [ ] e2e
- [ ] generate
- [ ] add
- [ ] update
- [ ] lint
- [ ] xi18n
- [ ] run
- [ ] config
- [ ] help
- [ ] version
- [ ] doc

Versions

OS: macOS Sierra
Node: v8.9.4
NPM: 6.2.0

Ng Version:

Angular CLI: 6.1.0
Node: 8.9.4
OS: darwin x64
Angular: 6.1.0
... animations, cli, common, compiler, compiler-cli, core, forms
... http, language-service, platform-browser
... platform-browser-dynamic, router

Package                           Version
-----------------------------------------------------------
@angular-devkit/architect         0.7.0
@angular-devkit/build-angular     0.7.0
@angular-devkit/build-optimizer   0.7.0
@angular-devkit/build-webpack     0.7.0
@angular-devkit/core              0.7.0
@angular-devkit/schematics        0.7.0
@angular/cdk                      6.4.1
@angular/flex-layout              6.0.0-beta.16
@angular/material                 6.4.1
@ngtools/webpack                  6.1.0
@schematics/angular               0.7.0
@schematics/update                0.7.0
rxjs                              6.2.2
typescript                        2.9.2
webpack                           4.9.2

Repro steps

Run ng test --code-coverage
Notice that things like import statements are counted as if else branches.

The log given by the failure

No log file

Desired functionality

I expect to see things like import statements and decorator metadata to not be marked as if else branches in the code coverage report.

Mention any other details that might be useful

[wkurniawan07]

Not providing any solution here, but the problem is likely to be in Devkit 0.7 instead of CLI 6.1.

[dimitriy-k]

same issue after upgrade to 6.1.1. coverage dropped from 100%
Statements : 98.13% ( 3667/3737 )
Functions : 99.47% ( 752/756 )

[kevinbosman]

Had the same problem after upgrading to cli 6.1.1 + devkit 0.7.1. Fixed by downgrading @angular-devkit/build-angular to 0.6.8

Before (with coverage issues):

Angular CLI: 6.1.1
Node: 8.11.2
OS: win32 x64
Angular: 6.1.0
... animations, common, compiler, compiler-cli, core, forms
... http, language-service, platform-browser
... platform-browser-dynamic, router

Package                           Version
-----------------------------------------------------------
@angular-devkit/architect         0.7.1
@angular-devkit/build-angular     0.7.1
@angular-devkit/build-optimizer   0.7.1
@angular-devkit/build-webpack     0.7.1
@angular-devkit/core              0.7.1
@angular-devkit/schematics        0.7.1
@angular/cli                      6.1.1
@ngtools/webpack                  6.1.1
@schematics/angular               0.7.1
@schematics/update                0.7.1
rxjs                              6.2.2
typescript                        2.9.2
webpack                           4.16.3

After (coverage working properly again):

Angular CLI: 6.1.1
Node: 8.11.2
OS: win32 x64
Angular: 6.1.0
... animations, common, compiler, compiler-cli, core, forms
... http, language-service, platform-browser
... platform-browser-dynamic, router

Package                           Version
-----------------------------------------------------------
@angular-devkit/architect         0.7.1
@angular-devkit/build-angular     0.6.8
@angular-devkit/build-optimizer   0.6.8
@angular-devkit/core              0.7.1
@angular-devkit/schematics        0.7.1
@angular/cli                      6.1.1
@ngtools/webpack                  6.0.8
@schematics/angular               0.7.1
@schematics/update                0.7.1
rxjs                              6.2.2
typescript                        2.9.2
webpack                           4.16.3

[brian428]

I'm guessing something is wrong in the creation or handling source maps?

[alexw10]

Hope this gets fixed soon need to be on latest version of the @angular-devkit/build-angular because of the issue in 0.6.8 of it not exiting with proper error code but now this coverage report issue is a problem.
Willing to try to help fix it as well if anyone can point me where to start looking.

[jfpicard1]

Maybe it's the way that the sourceMaps are handled. If I add "sourceMap": true, in the test section, with @angular-devkit/build-angular 0.7.0 it seems to work.

See: https://stackoverflow.com/questions/51257459/if-else-path-not-taken-in-imports-in-angular-istanbul-code-coverage-report/51676221#51676221

[brian428]

I'm on 0.7.1, and adding "sourceMap": true to the test section (or via the command line) doesn't seem to affect the messed up coverage report output.

[jfpicard1]

Maybe you have a tsconfig.json with "sourceMap": false ?

Here my DevDependecies:

  "devDependencies": {
    "@angular-devkit/build-angular": "~0.7.0",
    "@angular/cli": "^6.1.1",
    "@angular/compiler-cli": "^6.1.0",
    "@angular/language-service": "^6.1.1",
    "@types/jasmine": "^2.8.8",
    "@types/jasminewd2": "~2.0.2",
    "@types/node": "^10.5.5",
    "codelyzer": "^4.4.2",
    "jasmine-core": "^2.99.1",
    "jasmine-spec-reporter": "^4.2.1",
    "karma": "^2.0.5",
    "karma-chrome-launcher": "^2.2.0",
    "karma-cli": "^1.0.1",
    "karma-coverage": "^1.1.2",
    "karma-coverage-istanbul-reporter": "^2.0.1",
    "karma-html-reporter": "^0.2.7",
    "karma-ie-launcher": "^1.0.0",
    "karma-intl-shim": "^1.0.3",
    "karma-jasmine": "^1.1.2",
    "karma-jasmine-html-reporter": "^0.2.2",
    "karma-phantomjs-launcher": "^1.0.4",
    "karma-typescript": "^3.0.12",
    "karma-typescript-angular2-transform": "^1.0.2",
    "karma-typescript-plugin": "^0.1.2",
    "lite-server": "^2.2.2",
    "rxjs-tslint-rules": "^4.7.2",
    "ts-node": "^7.0.0",
    "tslint": "^5.11.0",
    "tslint-angular": "^1.1.2",
    "tslint-consistent-codestyle": "^1.13.3",
    "tslint-microsoft-contrib": "^5.1.0",
    "typescript": "^2.7.2"
  }

[alexw10]

I can confirm that setting sourceMap: true fixed the issue for me I am on 0.7.2 @brian428 @jfpicard1
Guessing this shouldn't need to be on since it worked without it before?

[brian428]

Hmm, updated to 0.7.2, but still no luck. tsconfig.spec.json doesn't set sourceMap: false. I even added sourceMap: true just to try it (even though it's already set to true in the base tsconfig.json), but had no effect. The test section of my angular.json also includes this, but no luck there either:

"test": {
  "builder": "@angular-devkit/build-angular:karma",
  "options": {
    "main": "src/test.ts",
    "karmaConfig": "./karma.conf.js",
    "sourceMap": true,
    ...
  }
}

None of these tweaks make any difference. The coverage report is generated but the highlighting is all wrong.

[frankrue]

I have tried various versions and combinations of "sourceMap" set to true/false with no luck. My particular use case is a library that will later be imported by another app, so downgrading anything is proving problematic, as the "library" generation in Angular CLI 6 is fresh and shiny and new.

[Bartosz-D3V]

Same issue as @frankrue - setting sourcemap to true did not resolve the issue.

EDIT: A bit more clarification - it does not work for libraries generated with Angular CLI 6.
sourceMap option was added to angular.json and all tsconfig files.

[gms1]

fixed by adding "sourceMap": true
thanks to @jfpicard1

[pcroc]

Adding sourceMap: true does appear to fix the code coverage problem for me, but it seems to make failure stack traces disappear:

Without sourceMap: true:

HeadlessChrome 70.0.3508 (Linux 0.0.0) CheckoutRegisterPageComponent should create FAILED
    TypeError: this.companyService.getCompanyRefs is not a function
        at CompanyDetailsFormComponent.ngOnInit (webpack:///../common-ui/src/app/shared/company-details-form/company-details-form.component.ts?:47:33)
        at checkAndUpdateDirectiveInline (webpack:///./node_modules/@angular/core/fesm5/core.js?:9545:19)
        at checkAndUpdateNodeInline (webpack:///./node_modules/@angular/core/fesm5/core.js?:10809:20)
        at checkAndUpdateNode (webpack:///./node_modules/@angular/core/fesm5/core.js?:10771:16)
        at debugCheckAndUpdateNode (webpack:///./node_modules/@angular/core/fesm5/core.js?:11404:38)
        at debugCheckDirectivesFn (webpack:///./node_modules/@angular/core/fesm5/core.js?:11364:13)
        at Object.eval [as updateDirectives] (ng:///DynamicTestModule/CheckoutRegisterPageComponent.ngfactory.js:70:5)
        at Object.debugUpdateDirectives [as updateDirectives] (webpack:///./node_modules/@angular/core/fesm5/core.js?:11356:21)
        at checkAndUpdateView (webpack:///./node_modules/@angular/core/fesm5/core.js?:10753:14)
        at callViewAction (webpack:///./node_modules/@angular/core/fesm5/core.js?:10994:21)
HeadlessChrome 70.0.3508 (Linux 0.0.0): Executed 1 of 130 (1 FAILED) (0 secs / 0.791 secs)

With sourceMap: true:

HeadlessChrome 70.0.3508 (Linux 0.0.0) CheckoutRegisterPageComponent should create FAILED
    [object ErrorEvent] thrown
HeadlessChrome 70.0.3508 (Linux 0.0.0): Executed 1 of 130 (1 FAILED) (0 secs / 0.883 secs)

[prajwalmr62]

I was able to fix the issue by adding "sourceMap": true, in angular.json test section.

DevDependency versions are,

    "@angular-devkit/build-angular": "~0.7.0",
    "@angular/cli": "~6.1.2",
    "@angular/compiler-cli": "^6.1.0",
    "@angular/language-service": "^6.1.0",
    "@types/jasmine": "~2.8.6",
    "@types/jasminewd2": "~2.0.3",
    "@types/node": "~8.9.4",
    "codelyzer": "~4.2.1",
    "jasmine-core": "~2.99.1",
    "jasmine-spec-reporter": "~4.2.1",
    "karma": "~1.7.1",
    "karma-chrome-launcher": "~2.2.0",
    "karma-coverage-istanbul-reporter": "~2.0.0",
    "karma-jasmine": "~1.1.1",
    "karma-jasmine-html-reporter": "^0.2.2",
    "protractor": "~5.3.0",
    "ts-node": "~5.0.1",
    "tslint": "~5.9.1",
    "typescript": "~2.7.2"

[seckardt]

In my case passing the sourceMap flag via CLI did fix the issue (ng test --source-map) while setting it in tsconfig.json didn't seem to have any effect.

[bitjson]

I think https://github.com/angular/angular-cli/pull/11967 solves this issue. Looking at collaborators who might be familiar with the code, maybe @qiyigg can take a look?

[jhage-accesso]

@hansl will this be making it into the ng7 release?

[FrancescoBorzi]

I can confirm this issue. And I can confirm that running it using ng test --code-coverage --source-map solves the issue.

Here's my ng -v output:

Angular CLI: 6.2.1
Node: 10.9.0
OS: linux x64
Angular: 6.1.7
... animations, common, compiler, compiler-cli, core, forms
... http, language-service, platform-browser
... platform-browser-dynamic, router

Package Version

@angular-devkit/architect 0.8.1
@angular-devkit/build-angular 0.7.5
@angular-devkit/build-optimizer 0.7.5
@angular-devkit/build-webpack 0.7.5
@angular-devkit/core 0.8.1
@angular-devkit/schematics 0.8.1
@angular/cli 6.2.1
@ngtools/webpack 6.1.5
@schematics/angular 0.8.1
@schematics/update 0.8.1
rxjs 6.3.2
typescript 2.9.2
webpack 4.9.2

[Sharondio]

Also confirm running ng test --code-coverage --source-map solves this, despite all settings being set to sourceMap:true. Of course, this advice conflicts with the advice given in #11968. 😞

[brian428]

Unfortunately, I'm on the latest versions and I'm still getting the wonky coverage highlighting in the coverage report. Even using ng test --code-coverage --source-map.

Angular CLI: 6.2.1
Node: 8.11.3
OS: win32 x64
Angular: 6.1.7
... animations, common, compiler, compiler-cli, core, forms
... http, language-service, platform-browser
... platform-browser-dynamic, router

Package Version

@angular-devkit/architect 0.8.1
@angular-devkit/build-angular 0.8.1
@angular-devkit/build-optimizer 0.8.1
@angular-devkit/build-webpack 0.8.1
@angular-devkit/core 0.8.1
@angular-devkit/schematics 0.8.1
@angular/cli 6.2.1
@ngtools/webpack 6.2.1
@schematics/angular 0.8.1
@schematics/update 0.8.1
rxjs 6.3.2
typescript 2.9.2
webpack 4.18.0

And in case the versions related to Karma matter:
+-- [email protected]
+-- [email protected]

[fluky]

Using --sourceMap=true does not solve the problem for me as it actually causes the test to hang. I.e. when I specify --sourceMap=true it goes to a blank screen and remains there indefinitely.

[brian428]

Could we get an update from someone on the Angular team on this? It's been open for 5 weeks as a priority 1 bug, but there hasn't been any comment from the team yet.

[alexeagle]

we're planning a bug bash sprint starting in 1.5 weeks; as a P1 this issue will be on it.

[brian428]

FYI, I made the local change that was done in https://github.com/angular/angular-cli/pull/12291/files, but the coverage report highlighting is still messed up. Not sure if that's expected for now (I'm on @angular-devkit/build-angular 0.8.1, and don't know if other things have changed beyond that one schema change), but figured I'd mention it.

[lanegoolsby]

For anyone here from Google, I may have found a more permanent fix than using the CLI flag. Set "sourceMap": true in angular.json under projects -> [yourProject] -> architect -> test -> options.

Making the change in the tsconfig.spec.json or tsconfig.json had no effect for me.

My environment:

Angular CLI: 6.1.3
Node: 8.11.3
OS: win32 x64
Angular: 6.1.2
... animations, common, compiler, compiler-cli, core, forms
... http, language-service, platform-browser
... platform-browser-dynamic, router

Package Version

@angular-devkit/architect 0.7.3
@angular-devkit/build-angular 0.7.3
@angular-devkit/build-optimizer 0.7.3
@angular-devkit/build-webpack 0.7.3
@angular-devkit/core 0.7.3
@angular-devkit/schematics 0.7.3
@angular/cli 6.1.3
@ngtools/webpack 6.1.3
@schematics/angular 0.7.3
@schematics/update 0.7.3
rxjs 6.2.2
typescript 2.7.2
webpack 4.9.2

[pixx3l]

Just had the same problem start happening to me,

• angular: 6.1.10
• devkit 0.8.5

@lanegoolsby 's solution worked out for me, didn't have to add anything into the CLI commands, just the angular.json file edit

[PikaChokeMe]

Thought it might be worth noting that I'm still seeing this issue...
I'm currently using Angular 6.2.7
I currently have "@angular-devkit/build-angular": "~0.8.7",

After adding "sourceMap": true to my angular.json test options, checking that it was included in tsconfig.json, and manually trying to run ng test --code-coverage=true --watch=false --source-map=true I _still_ have had zero success getting code coverage working and detecting branches properly.

[richmwatts]

Is there any update on this? I have also set the sourceMaps property to true and believe I am now seeing more "believable" code coverage metrics. However, the highlighting in the reporting is horribly horribly wrong.

How can I have any confidence over the quoted coverage metrics?

[jhage-accesso]

This is fixed in ng7

[richmwatts]

I am still having the same issue with all dependencies completely up to date

[ash4aque]

This is fixed in ng7

Does this means it is not fixed for ng6

[marthinus-engelbrecht]

This is still broken, running these dependencies

"devDependencies": {
    "@angular-devkit/build-angular": "~0.13.0",
    "@angular/cli": "~7.3.8",
    "@angular/compiler-cli": "~7.2.0",
    "@angular/language-service": "~7.2.0",
    "@types/node": "~8.9.4",
    "@types/jasmine": "~2.8.8",
    "@types/jasminewd2": "~2.0.3",
    "codelyzer": "~4.5.0",
    "jasmine-core": "~2.99.1",
    "jasmine-spec-reporter": "~4.2.1",
    "karma": "~4.0.0",
    "karma-chrome-launcher": "~2.2.0",
    "karma-coverage-istanbul-reporter": "~2.0.1",
    "karma-jasmine": "~1.1.2",
    "karma-jasmine-html-reporter": "^0.2.2",
    "protractor": "~5.4.0",
    "ts-node": "~7.0.0",
    "tslint": "~5.11.0",
    "typescript": "~3.2.2"
  }

With --source-map=true it gives me 100% branch coverage, but with branches 0/0. And I don't have 100% branch coverage. With false I get the right numbers but the report is all mixed up.

[diamond2010]

FYI, I'm on the latest versions (Angular 8) and I'm still getting a non-accurate coverage report. Even using ng test --code-coverage --source-map. IMHO its a bug which is related to a problem with the MAPPING of the source-maps. Not use of source-maps itself.

Here is my configuration:

package.json

{
  "name": "web",
  "version": "0.0.0",
  "scripts": {
    "ng": "ng",
    "start": "ng serve",
    "build": "grunt && ng build --prod",
    "build-dev": "grunt && ng build --extract-css --watch --aot --output-hashing=bundles",
    "build-debug": "grunt && ng build --source-map --extract-css --output-hashing=bundles",
    "build-svg": "grunt",
    "test": "ng test",
    "test-ci": "ng test --watch=false --browsers=ChromeHeadlessIuE --code-coverage",
    "lint": "ng lint",
    "e2e": "ng e2e",
    "livereload": "livereload ./dist -e html -u true -w 250"
  },
  "private": true,
  "dependencies": {
    "@angular/animations": "^8.0.3",
    "@angular/common": "^8.0.3",
    "@angular/compiler": "^8.0.3",
    "@angular/core": "^8.0.3",
    "@angular/elements": "^8.0.3",
    "@angular/forms": "^8.0.3",
    "@angular/platform-browser": "^8.0.3",
    "@angular/platform-browser-dynamic": "^8.0.3",
    "@angular/router": "^8.0.3",
    "@ngrx/store": "^8.0.1",
    "@types/leaflet-fullscreen": "^1.0.4",
    "@types/leaflet.markercluster": "^1.4.0",
    "classlist.js": "^1.1.20150312",
    "core-js": "^2.6.9",
    "deep-equal": "^1.0.1",
    "leaflet": "^1.4.0",
    "leaflet.markercluster": "^1.4.1",
    "ngx-cookie-service": "^2.1.0",
    "rxjs": "^6.4.0",
    "s": "^0.1.1",
    "swiper": "^4.5.0",
    "tslib": "^1.10.0",
    "video.js": "^7.5.5",
    "zone.js": "~0.9.1"
  },
  "devDependencies": {
    "@angular-devkit/build-angular": "~0.800.6",
    "@angular/cli": "^8.0.6",
    "@angular/compiler-cli": "^8.0.3",
    "@angular/language-service": "^8.0.3",
    "@types/jasmine": "^3.3.13",
    "@types/jasminewd2": "^2.0.6",
    "@types/leaflet": "^1.4.3",
    "@types/node": "^10.14.10",
    "@types/swiper": "^4.4.3",
    "@types/video.js": "^7.2.10",
    "codelyzer": "^5.0.1",
    "grunt": "^1.0.3",
    "grunt-cli": "^1.3.2",
    "grunt-replace": "^1.0.1",
    "grunt-sass": "^3.0.2",
    "grunt-svgstore": "^2.0.0",
    "jasmine-core": "^3.3.0",
    "jasmine-spec-reporter": "^4.2.1",
    "karma": "^4.0.1",
    "karma-chrome-launcher": "^2.2.0",
    "karma-coverage-istanbul-reporter": "^2.0.5",
    "karma-jasmine": "^2.0.1",
    "karma-jasmine-html-reporter": "^1.4.0",
    "karma-junit-reporter": "^1.2.0",
    "livereload": "^0.8.0",
    "node-sass": "^4.11.0",
    "npm-check-updates": "^3.1.12",
    "protractor": "^5.4.2",
    "sass-loader": "^7.1.0",
    "ts-node": "^8.3.0",
    "tslint": "^5.18.0",
    "typescript": "~3.4.5",
    "webpack": "^4.35.0"
  }
}

Angular.CLI

``_ _ ____ _ ___ / \ _ __ __ _ _ _| | __ _ _ __ / ___| | |_ _| / △ \ | '_ \ / _ | | | | |/ _` | '__| | | | | | |
/ ___ \| | | | (_| | |_| | | (_| | | | |___| |___ | |
/_/ __| |_|__, |__,_|_|__,_|_| ____|_____|___|
|___/

Angular CLI: 7.3.6
Node: 10.15.2
OS: darwin x64
Angular:
...

Package Version

@angular-devkit/architect 0.13.6
@angular-devkit/core 7.3.6
@angular-devkit/schematics 7.3.6
@schematics/angular 7.3.6
@schematics/update 0.13.6
rxjs 6.3.3
typescript 3.2.4

-bash-3.2$

# angular.json

{
"$schema": "./node_modules/@angular/cli/lib/config/schema.json",
"version": 1,
"newProjectRoot": "projects",
"projects": {
"web": {
"root": "",
"sourceRoot": "src",
"projectType": "application",
"prefix": "db",
"schematics": {
"@schematics/angular:component": {
"styleext": "scss"
}
},
"architect": {
"build": {
"builder": "@angular-devkit/build-angular:browser",
"options": {
"outputPath": "dist",
"index": "src/index.html",
"main": "src/main.ts",
"polyfills": "src/polyfills.ts",
"tsConfig": "src/tsconfig.app.json",
"assets": [
"src/favicon.ico",
"src/assets/fonts",
"src/assets/icons/sprite.svg",
"src/assets/images",
"src/assets/sounds",
"src/static",
"src/sw.js"
],
"styles": [
"src/styles.scss"
],
"scripts": [
]
},
"configurations": {
"production": {
"fileReplacements": [
{
"replace": "src/environments/environment.ts",
"with": "src/environments/environment.prod.ts"
}
],
"optimization": true,
"outputHashing": "all",
"sourceMap": true,
"extractCss": true,
"namedChunks": false,
"aot": true,
"extractLicenses": true,
"vendorChunk": false,
"buildOptimizer": true
}
}
},
"serve": {
"builder": "@angular-devkit/build-angular:dev-server",
"options": {
"browserTarget": "web:build"
},
"configurations": {
"production": {
"browserTarget": "web:build:production"
}
}
},
"extract-i18n": {
"builder": "@angular-devkit/build-angular:extract-i18n",
"options": {
"browserTarget": "web:build"
}
},
"test": {
"builder": "@angular-devkit/build-angular:karma",
"options": {
"progress": false,
"main": "src/test.ts",
"polyfills": "src/polyfills.ts",
"tsConfig": "src/tsconfig.spec.json",
"karmaConfig": "src/karma.conf.js",
"styles": [
"src/styles.scss"
],
"scripts": [],
"assets": [
"src/favicon.ico",
"src/assets"
]
}
},
"lint": {
"builder": "@angular-devkit/build-angular:tslint",
"options": {
"tsConfig": [
"src/tsconfig.app.json",
"src/tsconfig.spec.json"
],
"exclude": [
"/node_modules/"
]
}
}
}
},
"web-e2e": {
"root": "e2e/",
"projectType": "application",
"architect": {
"e2e": {
"builder": "@angular-devkit/build-angular:protractor",
"options": {
"protractorConfig": "e2e/protractor.conf.js",
"devServerTarget": "web:serve"
},
"configurations": {
"production": {
"devServerTarget": "web:serve:production"
}
}
},
"lint": {
"builder": "@angular-devkit/build-angular:tslint",
"options": {
"tsConfig": "e2e/tsconfig.e2e.json",
"exclude": [
"/node_modules/"
]
}
}
}
}
},
"defaultProject": "web"
}
```

If some more informations are needed dont hasitate to write me. I am very interested to help solving this problem.

Thanks in advance

[angular-automatic-lock-bot[bot]]

This issue has been automatically locked due to inactivity.
Please file a new issue if you are encountering a similar or related problem.

Read more about our automatic conversation locking policy.

<sub>_This action has been performed automatically by a bot._</sub>