Angular-cli: Bug: Code Coverage Report

[niklas-wortmann]

- [x] bug report -> please search issues before submitting
- [ ] feature request

Versions.

$ ng --version
@angular/cli: 1.0.0-rc.2
node: 6.9.5
os: win32 x64
@angular/common: 4.0.0-rc.5
@angular/compiler: 4.0.0-rc.5
@angular/core: 4.0.0-rc.5
@angular/forms: 4.0.0-rc.5
@angular/http: 4.0.0-rc.5
@angular/platform-browser: 4.0.0-rc.5
@angular/platform-browser-dynamic: 4.0.0-rc.5
@angular/router: 4.0.0-rc.5
@angular/cli: 1.0.0-rc.2
@angular/compiler-cli: 4.0.0-rc.5

Repro steps.

I have made a sample project, to reproduce this weird behaviour.
Please take a look at: https://github.com/Loster102/angular-coverage-sample
When I run 'ng test -cc true', there is a successful testrun.
But when I look at the coverage report, the branch coverage is really strange.
It says, that the constructor is not fully covered.

Desired functionality.

I would have expected that even the constructor is fully covered.

Mention any other details that might be useful.

In my real world application, there are some further strange behaviours, exactly like the one above.
I try to reproduce these in the sample project. In this application, I'm using [email protected] and @angular/[email protected].

[hotforfeature]

There are two related issues both in Istanbul and TypeScript:
https://github.com/gotwarlost/istanbul/issues/690
https://github.com/Microsoft/TypeScript/issues/13455

The latest suggestion from both threads is to run tests with coverage targeting ES6 instead of ES5. Unfortunately from what I can tell, the Angular CLI doesn't use tsconfig.json when running tests, and the suggestions of using --compilerOptions '{\"target\":\"es6\"}' won't work since Angular CLI doesn't recognize that.

At the moment, there's nothing to do, adding /* istanbul ignore next */ comments everywhere around the constructor won't help. We're deadlocked while waiting for Angular, Istanbul, or TypeScript to change something.

[filipesilva]

If there's something we can do to help, we'd like to.

[stratus1011]

@hotforfeature you can actually change the "target" to es6 in the src/tsconfig.spec.json with the version of angular cli that you are using. However, if you are running tests on phantomjs it will fail due to the "const" keyword/ES6/ES2015 not being supported yet. There is a beta version of phantomjs that they have released that you could try which should have support for it, but may have other issues.

[sandman21dan]

tsconfig.spec file has es6 as a target, running on CLI-1.0.0 still get all the missing branches from all the constructors, maybe I'm missing something?

[cpa-level-it]

I have the same problem as @sandman21dan.
Changed target from es5 to es6 in tsconfig.spec.json and constructor branches are still not covered.

Any idea ?

[raguilera82]

Any solutions? It is really frustrating :/

[SURENDARREDDY]

Since it is being 2 months, just want to know the progress.
Is there any ETA for this issue resolution?

[cortopy]

Changing target to es6 in the main tsconfig.json doesn't work either

[rdsedmundo]

Same problem here.

[phillipzada]

Anyone got anything on this issue resolution?

I found an ugly workaround. Set the constructor params to @Inject(TYPE) private variable: any resolves it but then you loose intellisence and type-checking.

[dinvlad]

Changing to es6 doesn't work here as well. Is there a feasible solution in the works?

[rg1220]

Reproduced this issue by simply adding http: HTTP to the constructor of a newly created project created with angular-cli: https://github.com/rg1220/angular-cli-coverage-issue-5526

[brunolm]

I have the same issue :( ... Why is it blocked? Can we expect a fix in the near future?

[brunolm]

Workaround

As a workaround I found the special comment /* istanbul ignore next */

If you place after your class, the constructor gets ignored. Has to be exactly after the brace, do not break lines.

Example

export class InternalComponent {
    constructor(private authService: any) { 
    }
} /* istanbul ignore next */

Generates

define(["require", "exports"], function (require, exports) {
    "use strict";
    Object.defineProperty(exports, "__esModule", { value: true });
    var InternalComponent = (function () {
        function InternalComponent(authService) {
            this.authService = authService;
        }
        return InternalComponent;
    }()); /* istanbul ignore next */
    exports.InternalComponent = InternalComponent;
});

Which ignores this line exports.InternalComponent = InternalComponent; that causes the issue.

Note: If you have static members in your class add the comment after the last one, example:

static readonly authKey = 'access_token'; /* istanbul ignore next */

To ignore super

super() /* istanbul ignore next */;

If you have generics and/or @Inject in your constructor params, try to break lines.

With those hacks I got 100% coverage.

UPDATE: Well, yes, this is pretty ugly and might not work in all cases.

[davinkevin]

@brunolm This workaround works if you don't use decorator on attribute in your code.

If I remove the @Effect I have 100% coverage... do you have an idea ?

[filipesilva]

Heya, just wanted to mention that I added the blocked tag because I spent some time trying to fix it and as far as I can tell this is a limitation of the code coverage tools right now. @hotforfeature's comment (https://github.com/angular/angular-cli/issues/5526#issuecomment-288095948) explains the current situation well, I couldn't find any further developments.

Using an es6 target is not a real solution per se either, overall the CLI doesn't really support outputting es2015 (see https://github.com/angular/angular-cli/issues/2907),

[UnkJester]

Is there any other update? Has this been raised with the code coverage tools support? (I'm a bit new to this) I'm getting the issues @davinkevin mentioned, which breaks the only known workaround (other than lowering the branch requirement, which I'm being told is unsavory).

[kenisteward]

@filipesilva does #7610 now resolve this in the sense that we can provide es2015 for code coverage to run and it should work?

[webcat12345]

@brunolm , it works but /* istanbul ignore next */ what does this hack do exactly?

[brunolm]

@webcat12345 /* istanbul ignore next */ ignores the line bellow in the coverage. https://github.com/gotwarlost/istanbul/blob/master/ignoring-code-for-coverage.md

Following my instructions it will place the ignore statement exactly at a part of the generated code that is not being accessed.

It is a really ugly workaround and might not work in all cases.

[webcat12345]

Thanks @brunolm

Because of above kinds of issue, I was going to apply this hack to everywhere lol. you saved my life.

[brunolm]

@webcat12345 well, if you need to see real metrics it might be worth to attempt to do it. Currently there is no other way.

[kenisteward]

do note @webcat12345 @brunolm when they add --aot support to ngtest then we'll be able to target es2015 from the resolved #7610 and it will fix everything for tests.

[dinvlad]

With the release versions (Angular 5.0.0 and CLI 1.5) all coverage now appears perfect!

There was a transient remap problem due to something in RC 8 and/or possibly TS>2.4. Is everyone else's experience the same?

[filipesilva]

@dinvlad that's somewhat surprising to me, because I haven't actually added any fixes in 1.5.0 for this. But maybe the change in compiler pipeline helped somehow, or one of the dependencies that does coverage had fixes in it.

Can you share if added any workarounds or modifications in your project to address the coverage problems?

[filipesilva]

Now that I think about it, @hansl fixed a problem with sourcemaps in https://github.com/angular/angular-cli/pull/8230. Maybe that was the root cause of this issue in the first place.

Can other people confirm if your codeverage is now correct?

[rg1220]

@filipesilva - I can confirm it fixed the issue with constructors. I assume other similar issues were caused by the same thing.

I updated my test repo to Angular 5 and the coverage report is now correct:
https://github.com/rg1220/angular-cli-coverage-issue-5526/tree/feature/update-to-angular-5

[dinvlad]

@filipesilva I just made sure that all tsconfig files are identical to the auto-generated ones from a new project. No workarounds at all.

[kenisteward]

I'm using 4.4.6 until a library i'm using fixes a deprecated API. Just chiming in that CLI 1.5.0 and NG 4.4.6 does not correct it. I think you may have to specifically be on NG5.0

[FaniloRandria]

I confirm that Angular 5 solves the problem.
I move from:

• angular 4.4.6 to 5.0.0
• angular/cli 1.4.9 to 1.5.0
• nguniversal 1.0.0-beta.3 to 5.0.0-beta.5
• angular/flex-layout 2.0.0-beta.9 to 2.0.0-beta.10

on this project and I can reach 100% code coverage on branches now.

I did a code coverage with angular 4.4.6 and another one with angular 5.0.0 on the same code, and results are below:

angular 4.4.6:

angular 5.0.0:

[brunolm]

I upgraded to Angular 5 in a big project.

Almost everything worked without workarounds, there is just one scenario where it didn't work:

One of the tests for it:

it('calls setLocale', () => {
  spyOn(MomentDateAdapter.prototype, 'setLocale').and.callThrough();

  const adapter = new MomentDateAdapter();

  expect(adapter).toBeDefined();
  expect(MomentDateAdapter.prototype.setLocale).toHaveBeenCalledWith('pt-BR');
});

That's the only thing that didn't work for me.

Great job, Angular 5 is awesome :)

[laxu]

I'm still experiencing the constructor issue when using Jest with Angular 5.0 and jest-angular-preset. Setting tsconfig.spec.json target to es6 does not seem to have any effect.

[el-davo]

Also still experiencing this issue event after moving to angular 5. Its even highlighting comments as not being tests :(

[kenisteward]

@el-davo @laxu did you also upgrade to click 1.5.0?

[el-davo]

Am completely wrong, it seems I had -sm set to false which was completely throwing the coverage off. My bad, Angular 5 FTW!! :dancer:

[laxu]

@kenisteward Yup, cli is 1.5.0 too.

[kenisteward]

@filipesilva I can confirm after upgrading to Angular 5 and CLI 1.5.0 I also got full coverage for the constructors when before i was getting issues.

I'm targeting the default es5 that the cli generates

[Brototype]

@laxu did you find a solution for that issue with jest? I'm facing the same issue :(

[laxu]

@Brototype Unfortunately no. In the mean time you have to either use the /* istanbul ignore next */ workaround or lower branch coverage to account for the issue.

[filipesilva]

I'm closing as it seems this is fixed with 1.5.

[nabeelgit]

I resolved it by running the unit test environment in full privileged mode

[otroboe]

Hey guys !
So de we know how Angular 5 fixed this issue... ? :-P

[kenisteward]

Yeah has to do with the annotations now being parsed as typescript
transformations instead. I'll try to find the article but it now doesn't
put the same ternary operators which were found in the code coverage for
constructors and input/outputs

On Tue, Feb 20, 2018, 12:42 PM otroboe [e-mail] wrote:

Hey guys !
So de we know how Angular 5 fixed this issue... ? :-P

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/angular/angular-cli/issues/5526#issuecomment-367058155,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AMPLtSEh5oxmc3taE5JVdrBecojU8jBsks5tWwQdgaJpZM4Mi9tY
.

[vigohe]

@laxu have you fix this when using jest????

[otroboe]

For me with Jest 22.3.0, I just added this property in my jest.json config file:

{
  ...
  "mapCoverage": true
}

[ahnpnl]

@vigohe I experience the same issue with Jest.
@otroboe the option mapCoverage doesn't help. It was also removed in Jest 22.4
@filipesilva can you tell us exactly what was the fix in Angular 5 ?

[kenisteward]

I don't believe that jest is covered by the cli. In which case this doesn't
belong here but on stack overflow.

On Sat, Mar 3, 2018, 8:33 PM Ahn [e-mail] wrote:

@otroboe https://github.com/otroboe Angular 5 fixed the issue but only
when testing with karma + jasmine, with Jest I also experience the same
issue. @filipesilva https://github.com/filipesilva can you tell us
exactly what was the fix in Angular 5 ?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/angular/angular-cli/issues/5526#issuecomment-370194403,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AMPLtUKPtuq-KCUr64POpqXwqLIUEJjbks5ta0R1gaJpZM4Mi9tY
.

[ggarbereder]

+1

[mboughaba]

It seems most of the issues are solved nevertheless we are still experiencing same issue like @brunolm mentioned

[kellycrowther]

We are experiencing this in our project with constructors and decorators. Running angular 5.2.8, angular-cli 1.7.3, jasmine 3.1.0 running with karma.

[roupify]

I resolve the problem by targetting es6 in tsconfig.spec.json

{
  "extends": "../tsconfig.json",
  "compilerOptions": {
    "outDir": "../out-tsc/spec",
    "baseUrl": "./",
    "module": "commonjs",
    "target": "es6",
    "types": ["jasmine", "node"]
  },
  "files": ["test.ts"],
  "include": ["**/*.spec.ts", "**/*.d.ts"]
}

[macbem]

Just make sure that you have sourcemaps turned on for your tests and coverage will work fine.

[mboughaba]

@macbem Not the case for me, I experience this problem with and without sm flag.

[kellycrowther]

Thanks for the suggestions @macbem and @rousseaufiliong. I previously tried those suggestions along with the other suggestions in this thread like adding /* istanbul ignore next */ and I tried again today for good measure but still find coverage is low for branches because of constructors and decorators. Hopeful an update solves this for our project

[tedp]

Is a fix on the way?

[victormejia]

I experience this problem with having source maps turned off for testing.

[rafaelnery]

@filipesilva with version 1.7.4 the problem continues :(

[mcblum]

We're also still seeing the same problem, Angular 5.2.6, Jest 22.4.3 and Jest Angular Preset 5.2.2. For whatever reason, it's not all the time, but there are issues where some of the params in a constructor will be uncovered. It doesn't seem to make a lot of sense, because it's not consistent.

[soates]

Also seeing this behaviour as described by @mcblum

[dimitriy-k]

indeed, after adding --sourcemaps=false coverage started giving wrong numbers (cli 1.7.4, target es5). When you remove it, coverage is ok.

[russcarver]

Same issues with Angular 5.2.11, Jest 23.1.0 and jest-preset-angular 5.2.2. All the istanbul packages pulled in (inspected via package-lock.json) also appear to be the latest.

        "istanbul-api": "^1.3.1",
        "istanbul-lib-coverage": "^1.2.0",
        "istanbul-lib-instrument": "^1.10.1",
        "istanbul-lib-source-maps": "^1.2.4",

I even tried upgrading to TypeScript 2.9.2 but no luck.

[aleh-vasilyeu]

Why is this issue closed?!
The problem still remains, here is an example of an empty component:

How can I fix this? Thanks.

[rmorrise]

We fixed this by modifying angular.json to add the sourceMap flag:

The flag should be added under the following path: projects.my-app.architect.test.sourceMap

        "test": {
          "builder": "@angular-devkit/build-angular:karma",
          "options": {
            "main": "src/test.ts",
            "karmaConfig": "./karma.conf.js",
            "polyfills": "src/polyfills.ts",
            "tsConfig": "src/tsconfig.spec.json",
            "scripts": [],
            "styles": [
              "src/styles.css"
            ],
            "assets": [
              "src/assets",
              "src/favicon.ico"
            ],
            "sourceMap": true // <-- Add this flag!!
          }
        },

[rmorrise]

Angular CLI team reported that newer CLI versions will correct this automatically:
https://github.com/angular/angular-cli/issues/13093

[Javin007]

Angular CLI team lies. I'm running the latest angular/jest and still seeing this problem. NONE of the above "solutions" work. I've tried /* istanbul ignore next */, sourcemaps (both on and off) setting the target to es6, even adding the deprecated "mapcoverage" setting. Nothing works. Problem still exists without so much as a workaround. This is not a "closed" bug.

[ruimarques]

Angular CLI team lies. I'm running the latest angular/jest and still seeing this problem. NONE of the above "solutions" work. I've tried /* istanbul ignore next */, sourcemaps (both on and off) setting the target to es6, even adding the deprecated "mapcoverage" setting. Nothing works. Problem still exists without so much as a workaround. This is not a "closed" bug.

Before calling a team as liars consider for a moment that you might be missing something or maybe you have encountered an edge case unknown to the community. If you provide a minimum reproducible example (created in stackblitz for instance) it would be more helpful to get your problem sorted out.

Also this issue was closed ages ago - a lot has happend to the CLI since. A regression might have happened in the meantime which, if confirmed, is a reason to reopen it.

[mcblum]

@Javin007 bro... come on. These are some of the most capable people working on software today, so the chances that they are lying to cover up a bug that, at its core, leads to an incorrect coverage report number are almost 0.

I'm using the latest CLI and Jest and ignore next works just fine. I would guess there is something else wrong with your setup.

Please consider the tone you take when commenting on projects which are free for you to use. No one owes you a fix, we're all in this together, and people are doing their best.

[Javin007]

@mcblum @ruimarques

I'm sorry if I come off as a bit irritated, but this is a very easily duplicated bug:

1.) Create new "hello world" project.
2.) Install Jest.
3.) Inject HttpClient into the app.component.ts
4.) Run tests with code coverage.

I JUST did it again to verify. It literally happens every time. On multiple versions of angular, and on every environment. And I've spent the last 48 hours searching for a solution, and have tried every single "fix" found on google to no avail. And this has been a problem for quite some time.

I'm not even saying it's specifically the Angular team's fault. It could very likely be a problem with Jest. However, claiming that "it's fixed" when it's not, and it's so easily duplicated is not helpful. Especially when I see numerous posts on numerous different sites (and even in this thread) claiming so without doing the very basic test outlined above. This original post was from March of 2017. It's now February of 2019, and this thread is a history of people continuing to have the exact same problem without a solution at any time. So this isn't just some problem that "cropped up again."

And yes, I'm irritated at spending two days of seeing people saying "it's fixed" when it simply isn't, and none of the MANY, MANY proffered solutions (everything from creating an entirely new project, to reinstalling angular) are anything but a waste of time. This kind of hand-waving away of the problem doesn't get it solved (obviously, since we're now on year two of the same issue).

If you actually have a solution that fixes the problem, then I and the many other people suffering from this problem would very much love to hear it. If you're not having this issue, then what is it that you're doing differently from the above steps? We would very much like to know.

[mcblum]

@Javin007 since even the steps you list above could result in a different version being installed on my machine than yours, could you please put the exact configuration in a StackBlitz so one of us can grab the code and see what's up? When you say "install jest" there are 1,000 different options, stable vs. alpha version of jest-preset-angular and a bunch of other things that can change, so put your exact configuration in a SB and we can see how you're getting the issue you're getting.

Once again, I'm currently, as I type, working on a large project using Angular CLI + Jest + Istanbul and all of my ignore statements work correctly.

[ahnpnl]

I will drop one cent here:

• first of all, the issue with coverage constructor when using jest with Angular CLI isn't in Angular CLI fix because Angular CLI only fixed for karma + Jasmine.
• jest-preset-angular 7.0.0-alpha.1 fixed the issue with coverage when using jest. However we are trying to make jest-preset-angular to be compatible with jest 24 at the same time. Being compatible with jest 24 + alpha version work are 2 big breaking changes and require some times to release an official version.

[Javin007]

FWIW: Here's a project that has the problem: https://yqnigwvz.github.stackblitz.io

[mcblum]

@Javin007 could you try with the following:

"jest": "23.6.0",
"jest-preset-angular": "7.0.0-alpha.2",

You will need to change your jest config when you update to the alpha of jest-preset-angular. It looks something like this:

"jest": {
    "preset": "jest-preset-angular",
    "setupFiles": [
      "jest-canvas-mock"
    ],
    "setupTestFrameworkScriptFile": "<rootDir>/config/jest/config.ts",
    "transformIgnorePatterns": [
      "<rootDir>/node_modules/(?!@ngrx|@agm)"
    ],
    "transform": {
      "^.+\\.js": "<rootDir>/node_modules/babel-jest",
      "^.+\\.(ts|html)$": "ts-jest"
    },
    "globals": {
      "ts-jest": {
        "tsConfig": "<rootDir>/tsconfig.spec.json",
        "diagnostics": false,
        "stringifyContentPathRegex": "\\.html$",
        "astTransformers": [
          "jest-preset-angular/InlineHtmlStripStylesTransformer.js"
        ]
      },
      "window": true
    },
    "modulePathIgnorePatterns": [
      "<rootDir>/coverage/",
      "<rootDir>/.storybook/",
      "<rootDir>/config/",
      "<rootDir>/dist/",
      "<rootDir>/e2e/",
      "<rootDir>/node_modules/"
    ],
    "collectCoverageFrom": [
      "{apps|libs}/**/src/**/*.ts"
    ],
    "coverageThreshold": {
      "global": {
        "statements": 90,
        "branches": 90,
        "functions": 90,
        "lines": 90
      }
    },
    "testMatch": [
      "**/__tests__/**/*.ts?(x)",
      "**/?(*.)+(spec).ts?(x)"
    ],
    "snapshotSerializers": [
      "jest-preset-angular/AngularSnapshotSerializer.js",
      "jest-preset-angular/HTMLCommentSerializer.js"
    ]
  }

Obviously you'll need to change everything that's different, this is just from my project. Upgrading to the alpha version and sticking with Jest 23 for now should fix it.

[Javin007]

It does indeed seem to be working now. So it would seem it's the fix is indeed in the Jest alpha. Thanks for your help!

[angular-automatic-lock-bot[bot]]

This issue has been automatically locked due to inactivity.
Please file a new issue if you are encountering a similar or related problem.

Read more about our automatic conversation locking policy.

_{_This action has been performed automatically by a bot._}