angular 🚀 - TestBed.configureTestingModule Performance Issue

Seems like the goal is really to have a fresh _component instance_ each time, and for the most part the testing _module_ is going to be static for a particular .spec file. It also seems like configuring the _module_ at each iteration is the part that is really adding on the time. A better pattern could be configuring the _module once_, then testModule.createComponent multiple times.

TestBed.configureTestingModule once and calling TestBed.createComponent on that module multiple times doesn't work, though... maybe I'm missing an obvious way to preserve module state between iterations?

_Edit: FWIW I tried logging the timestamp and it looks like most of the time passes during TestBed.createComponent, not TestBed.configureTestingModule - however I still suspect it's related to module configuration, since adding things to imported modules adds to the time, whether or not those imports are used in the component being tested._

ollwenjones on 28 Oct 2016

I have a noticed unit test execution performance issues that seem directly related to this issue.

I have an ionic 2 / angular 2 app. When trying to unit test my components using TestBed and ComponentFixture classes, there is easily 1-2 seconds spent in the setup of each test that seems directly related to the imports: [IonicModule.forRoot(MyComponent)] that I've added to my testing module via the TestBed.configureTestingModule({...}) method.

When I remove the import, my tests run much faster, but they obviously fail because my component's template no longer compiles.

vmandy on 15 Nov 2016

@vmandy I've been working around this by doing one or more of:

spending more time curating _just what I need_ in the test module
mocking the component dependencies that are time-consuming to bootstrap
overriding the template of the component I'm testing so it doesn't depend on much.

All of which are time consuming and sub-optimal 👎 (except from the point of view that I am really forced to isolate the unit I am testing?)

It seems like there are some _theoretical_ optimizations to this at least, because I _know_ these components are not taking 1-2 secs to bootstrap at run-time.

ollwenjones on 15 Nov 2016

Iam having similar problems. Thought it was caused by wallaby, but it seems to be the TestBed.

https://github.com/wallabyjs/public/issues/885

FlemmingBehrend on 1 Dec 2016

I am seeing the same issue. Any unit test with component compilation is taking close to 1sec...

mlakmal on 15 Dec 2016

Unfortunately I was pulled off the Angular 2 project and haven't looked at this in quite some time. It seems like a lot of the complications of TestBed (test-module setup, generally, not just performance) could be avoided by an approach similar to the enzyme.shallow render approach in the React ecosystem. - where just one unit is rendered and none of it's dependencies - the same thing can be accomplished by over-riding an Angular component's template in the .spec, but that is often tedious/time-consuming as well.

ollwenjones on 15 Dec 2016

we have close to 2000 test cases getting executed just under 1 min in Angular 1 where Angular 2 takes 1min to execute just 100 test cases. It would be nice if we can get a resolution for this issue as it hinders the testing capabilities in large applications.

mlakmal on 15 Dec 2016

@mlakmal I think there are lots of folks relying heavily on e2e tests, or just testing functional logic in component classes

let component = new ComponentClass(mockDep1, mockDep2);
expect(component.doAddition(2,2)).toBe(4);

Which doesn't test any of the template rendering (template logic)... but it does allow easy testing of _class logic_ at least

ollwenjones on 15 Dec 2016

👍1

@ollwenjones thanks, i am going to try that out. i really doesn't want to test any template logic since most of that gets covered by our automated tests, so just testing the component class code should be enough.

mlakmal on 15 Dec 2016

@mlakmal sweet! Makes me happy to contribute something that actually helps someone. 🍻 - FWIW I'm often torn about template-rendering unit tests, because it's so easy to slip into testing the _framework_.

ollwenjones on 15 Dec 2016

🚀1 👍1

I'm in the process of upgrading a pretty big project from rc.4 to 2.3, so it's the first time we're using the NgModule approach.

We have ~1000 unit tests, with maybe 1/2 of those testing components. Prior to upgrade, tests run pretty quickly - around 30s.

After upgrade, they take at leat twice as long, but the worst thing is that they seem kind of "flaky" - Karma will periodically lost the connection to Chrome and often I will need to refresh Chrome multiple times. I suspect that something is timing out but haven't isolated it yet.

@ollwenjones regarding "shallow" component tests, did you know about the NO_ERRORS_SCHEMA that you can set in the testing module?

import { NO_ERRORS_SCHEMA} from '@angular/core';
// ...
TestBed.configureTestingModule({
  declarations: [ /*... whatever */ ],
  schemas: [NO_ERRORS_SCHEMA]
});

This means that the compiler just ignores any elements it does not recognize, meaning you do not need to declare all the components used in the template of the component under test.

michaelbromley on 15 Dec 2016

👍9 🎉3

Found the same issue in Angular's tests: #13500
I was able to improve the situation by creating bespoke testing modules for each test.

marclaval on 15 Dec 2016

@michaelbromley 😮 I had no idea about NO_ERRORS_SCHEMA would have / will probably save me _hours_ - also an opportunity to hopefully speed up some of these tests, as the leaner the test-module the faster TestBed seems to go.

ollwenjones on 15 Dec 2016

👍1

@michaelbromley that also helped me out a lot. Removing MaterialModule.forRoot() from my imports really speed up my tests. 👍

FlemmingBehrend on 16 Dec 2016

👍1

Ideally, we should be able to call TestBed.configureTestingModule() inside a beforeAll(). It just doesn't work because of this line. This is reseting the modules before each test. Perhaps it shouldn't matter, but it does because Angular spends quite a lot of time compiling components in JiT mode. Reseting it throws away this information. Recording a profiling session on karma window led me to this conclusion.

IMO, this should be done:

Be able to configure a module in a beforeAll();
Cache compiled components

Another way to fix that is making reset testing module explicit, but this may break user code.

awerlang on 26 Feb 2017

👍30 ❤3

Is it clear that this isn't something to do with karma-phantom? The tests fly in Chrome for me, but Phantom takes at least a second per test. I've mocked the dependencies out as much as possible and they still run slow in Phantom.

gonzofish on 8 Mar 2017

👎2 👍1

phantom is very slow for me (using karma-webpack, but this is a different issue. I use Chrome now, and you can visually see as the tests run when there are tests that use a lot of components that have to be compiled it gets stuck for a second and runs slowly until it hits service tests which run fast.

Configuring module in beforeAll is not so convenient, because i want to create new mocks after each run and have them (and other providers) in a fresh state.

Caching the compiled components is the way to go imo.

Necroskillz on 8 Mar 2017

You can also use electron and karma-electron to run your tests
headless. Its nearly as fast as chrome. Phantomjs is about 8 times slower
for me.

Cheers

Necros notifications@github.com schrieb am Mi., 8. März 2017, 23:49:

phantom is very slow for me (using karma-webpack, but this is a different
issue. I use Chrome now, and you can visually see as the tests run when
there are tests that use a lot of components that have to be compiled it
gets stuck for a second and runs slowly until it hits service tests which
run fast.

Configuring module in beforeAll is not so convenient, because i want to
create new mocks after each run and have them (and other providers) in a
fresh state.

Caching the compiled components is the way to go imo.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/angular/angular/issues/12409#issuecomment-285195615,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABn5_18VjaBSPZQX1ZQ45mveN2PFMoNBks5rjzB-gaJpZM4KcS5I
.

philipooo on 9 Mar 2017

👍1

AOT support would help here too.

michalmo on 13 Mar 2017

@philipooo with slight modifications to some of Jasmine references (jasmine.createSpyObj() specifically), you can also run your tests without any browser – by using Jest instead of Karma.

For as few as 35 files with total 100 tests Jest has been able to speedup my tests 2.5x (vs Karma on Chrome). Tests are now running in parallel, completely isolated, not to mention superior watch mode with instant feedback.

I've written about it here, if anybody's curious: https://www.xfive.co/blog/testing-angular-faster-jest/.

There's also an issue to integrate it directly into CLI (e.g. under a flag: ng test --jest): https://github.com/angular/angular-cli/issues/4543.

thymikee on 4 Apr 2017

❤13 👍8

@thymikee I'm super excited to hear that. I've had _muuuch better_ experience testing React code with Jest than I ever have with Angular, and I've been persuaded that end-users of trustworthy frameworks don't need to do as much browser-testing.

Last time I checked Jest was using jsdom which did not play nice with zone.js is this no longer the case? 🤞 I'll have to read your blog.

ollwenjones on 4 Apr 2017

So jsdom is not the problem. It's just that tests must run in Zone.js context, which is done by this little lib: jest-zone-patch

thymikee on 4 Apr 2017

👍3

Testing with HTML components is just a pain... I have about 500 unit tests that are 90% looking at the DOM, it takes me 40 minutes to run them all... at this rate, Chrome crashes when it reaches a RAM level too high... the result is that I have to batch my tests by 50. Painful as hell...

There are definitely some memory leaks here and there...

jsgoupil on 10 Apr 2017

bueller?

yairgo on 3 May 2017

😄2

Found out that using FormsModule considerably slows down compileComponents mainly using ngModel.

For example here is a plunker which has a component with 10 inputs and it takes about 3 times more when using ngModel.

oferh on 11 Jun 2017

👍2

@juliemr Hey, I wanted to do a feature request to add a schema that does shallow rendering. Shall keep the stuff here (since what @ollwenjones wrote pretty much is what i want) or create a new issue?
Basically it's:

Being able to instantiate the child components, BUT not it's template. (this is a bit more complex than it sound)
Still having errors when bound to wrong property on the component (unlike error schema)

At the moment I'm creating stub components in my project to create blank components and have them implement same interface to make them consistent the inputs and outputs and makes it easier to my users to test the template of their component. It becomes really difficult once you include stuff like ContentChildren logic combined with templateOutlet directive usage or transclusion.

Possible solution is use normal schema ot 1st component then for the child components use the NO_ERROR_SCHEMA.

dblVs on 12 Jun 2017

So we recently switched to Jest from webpack > karma/phantom/chrome, using the great instructions / tools @thymikee 🎊 which speeds things up a bit (the pre-test webpack build step is removed at least) and simplified setup _tremendously_ - but overall Angular testing still feels like a _struggle_.

The main reason is configuring testing modules is still a _pain_ - results with NO_ERRORS_SCHEMA are spotty, and certain things that are destined to slow things down (like things from @angular/material and DynamicFormsModule) have to be included (so this one simple test I'm working on still takes 6 seconds.) Thanks @oferh for the plunk definitively demonstrating this.

Leads me to say that @dblVs, I think you _should_ do a feature request. I think a _positive_ suggestion with some good design ideas will probably be better received by core people than my negative, "this is slow... 😢"

My opinion is that without such a tool, we aren't empowered by the current tooling to write _unit tests_ at all. All these tests with deep component rendering and all dependencies down the tree provided in the test-module for injection, etc. are _integration tests_, and integration tests are _harder to write._

ollwenjones on 13 Jun 2017

👍3

if it helps anyone, I tried Electron headless browser with angular 2 unit tests and it performs far better than PhantomJS.

mlakmal on 19 Jun 2017

👍2

Yes, PhantomJS is really very old. But the latest version of Chrome can also run as a headless browser and it will be the fastest one.

mlc-mlapis on 19 Jun 2017

@ollwenjones i totally agree, this is an very important/critical feature request. if we want to develop via TDD(also normal testing phase) its very hard and we are struggle with the test runner time performance

ChenReuven on 3 Jul 2017

👍2

This is my current work around:

const oldResetTestingModule = TestBed.resetTestingModule;
beforeAll(done => (async () => {
  TestBed.resetTestingModule();
  TestBed.configureTestingModule({
    // ...
  });
  await TestBed.compileComponents();

  // prevent Angular from resetting testing module
  TestBed.resetTestingModule = () => TestBed;
})().then(done).catch(done.fail));

afterAll(() => {
  // reinstate resetTestingModule method
  TestBed.resetTestingModule = oldResetTestingModule;
  TestBed.resetTestingModule();
});

This way, it only needs to run configureTestingModule and compileComponents once per spec file. After the testing module is initialised, running TestBed.createComponent() is fairly fast.

vvasabi on 12 Jul 2017

👍42 ❤16 🎉11

@vvasabi interesting workaround, one spec test file was very slow, because inside a beforeEach we defined the module who includes a module with many components and some mocked services (with providers array and useClass) Sadly some tests are failing after using your fix. We also use AOT who maybe slowdown the compile.

We also use overrideComponent inside beforeEach as the service is only used in this component.

fixture = TestBed.overrideComponent(ListingFormComponent, { set: { providers: [ { provide: DisplayFieldService, useClass: MockDisplayFieldService } ] } }).createComponent(ListingFormComponent);

But the main problem is clearly the include of a "big" module with many components inside the configureTestingModule.

Time after fix : Executed 53 of 148 (3 FAILED) ERROR (1.261 secs / 0.327 secs) running smooth but 3 failing tests now maybe more as the test suite stopped.
Time before Fix : Chrome 59.0.3071 (Mac OS X 10.12.5): Executed 63 of 148 (skipped 85) SUCCESS (44.235 secs / 44.164 secs) with some freeze execution during tests.

Still investigating.

rbinsztock on 13 Jul 2017

@rbinsztock I would try to isolate the failing tests into their own spec files without my hack applied. Because my hack prevents Angular from resetting the test bed after each test is run, some tests that change states in shared services (i.e. pollute the test bed) may cause other tests to break. For example, if test 1 makes some changes to your MockDisplayFieldService, then test 2 will get back the same MockDisplayFieldService instance with the changes still applied.

Once you figure out which tests are changing the states of shared services and causing others to break, you can try to undo the changes after the assertions. Then, move back those tests that were isolated, and see if they can now pass. Hope this helps.

Edit: I just noticed that your MockDisplayFieldService is provided by ListingFormComponent, so while it likely has a new instance created for each test, other shared services might not.

vvasabi on 14 Jul 2017

👍3

wow great workaround. Tested it in one spec file.

Before

Finished in 6.282 secs / 7.383 secs @ 16:56:55 GMT+0200 (CEST)

After

Finished in 0.17 secs / 0.202 secs @ 16:59:56 GMT+0200 (CEST)

But since it is still a workaround i can not let my dev group exploit that wildly. We have 1000+ unit tests. The whole suite takes about 10 minutes to run. Would love to see an official solution soon.

giniedp on 26 Jul 2017

👍3

I started wondering if this isn't just too much to ask, because of how much compilation has to happen just to have one Angular thing (typescript, decorator annotations, template parsing, all the ng-modules for DI, aot in some cases, etc.) - maybe it's just not be reasonable to expect them to run like some pure JS or even pure TS test? 🙁

ollwenjones on 27 Jul 2017

I think it is ok for a testmodule to be slow on construction time. But we should have the choice whether to create the TestBed before all or before each test case. Right now we are forced to the latter and thus the only option to speedup the tests is to clutter multiple expectations into one single test unit. 👎

giniedp on 27 Jul 2017

👍6

Complelty agree with @giniedp here.

In our case we have two seperate repositories, one with 2500+ test and the other that will reach the 1000 before the end of the month.

The first one still imports many modules in the TestBed configuration, leading to 6-7 minutes to complete a full run. I've started to remove them one by one as i work on each feature to improve the speed. The second one, created way after that, i've said to my team to use as many mocks as possible and to avoid at all cost importing anything in the testbed, and in general, group multiple expect in a single test (wich IMO is wrong). This is muuuuch faster for a full test run (without compilation, about 20 seconds), but the compilation time still take at least a minute if not up to 2 or 3, wich is clearly annoying.

I'd love to see a big performance improvement on tests, since it's where i lose 50% of my time when developping a feature.

Shireilia on 4 Aug 2017

@Shireilia try Jest (with jest-preset-angular) so you can skip the compilation step, parallelise tests and have smarter watch mode

thymikee on 4 Aug 2017

👍6

@thymikee Thanks, i'll give it a try in the coming week :)

Shireilia on 4 Aug 2017

@thymikee Thanks for your work on the preset. Jest works really well for my project. A few observations:

Jest runs faster than Karma because it bypasses Webpack and has the ability to run multiple spec files concurrently. However, the tests, that require Angular components to be instantiated, actually take longer time to run than Karma in Chrome. I am guessing that the problem might be jsdom not as efficient as Blink. Though, some further investigations will be required.
Since Jest does not require all modules to be loaded in order for tests to run, the fewer modules that a test has to import, the faster it will finish. I spent some time isolating modules that did not have to be imported (directly or indirectly), and it gave me some mild improvement on the lap times. (For example, some modules, that weren’t used, got exported by an index.ts file and ended up in the import graph.)
My workaround above is applicable when using Jest, as well.
I noticed that spec files that did not have a unique name would break IntelliJ’s ability to set breakpoints in them. I will try to create a test repo and open an issue in your repo. (Edit: issue opened)

Thanks again. Jest is awesome.

vvasabi on 8 Aug 2017

👍7

@vvasabi thanks for the workaround it drastically sped up my tests. One question though, would there be a way to have a central module that the spec files just use. E.g. go ahead and include everything in one module then have all the spec files use it?

May be an anti pattern but the slowdown for me now is it takes time to set the module of for each spec file. I have 20 so far and they are growing (one for each component, service, directive, etc) So the slow down comes when each spec file is hit. Once the module is built the subsequent it statments run in no time.

Thanks again for the workaround

DrewLandgrave on 12 Aug 2017

@Rhonun Glad to hear that my hack works for you. You can certainly build a common testing module. Just export the module metadata from one file, and your spec files can just import and reuse.

If you want to go one step further and make sure my hack is not repeated in every spec file, consider setting up a util function like this:

const resetTestingModule = TestBed.resetTestingModule,
  preventAngularFromResetting = () => TestBed.resetTestingModule = () => TestBed;
  allowAngularToReset = () => TestBed.resetTestingModule = resetTestingModule;
export const setUpTestBed = (moduleDef: TestModuleMetadata) => {
  beforeAll(async(async () => {
    resetTestingModule();
    preventAngularFromResetting();
    TestBed.configureTestingModule(moduleDef);
    await TestBed.compileComponents();
  }));

  afterAll(() => resetTestBed());
};

Then, just call it inside the outermost describe block of your spec files. Hope this helps.

vvasabi on 14 Aug 2017

👍6

@vvasabi ur work around indeed works well. i cut down around 250 tests from 45 secs to about 15 secs. i really think the ng team should provide the option to setup the testing module in beforeAll as this saves a lot of time, especially in specs with a lot of its.

only drawback so far is that in some specs the dom interaction seems to be broken with the new approach (e.g. setting input values, clicking btns )

another drawback is that it doesnt seem possible to use beforeAll with @vvasabi s approach in some specs and beforeEach in other specs in the same suite.

dasAnderl on 15 Aug 2017

@dasAnderl Glad to hear that my trick worked for you.

some specs the dom interaction seems to be broken (e.g. setting input values, clicking btns)

Since the testing module is now reused across multiple tests, any test that mutates a shared service can potentially affect the result of other tests. One quick way to confirm this is to run the broken tests individually. If they pass when run alone, then they fail because of polluted TestBed.

drawback is that it doesnt seem possible to use beforeAll in some specs and beforeEach in other specs in the same suite

Yes, the workaround will force you to group up tests that can reuse the same TestBed configuration in one describe block. Ones that cannot will have to live in their own compartments. I would like to argue that this might be a cleaner code organisation anyways, whether or not the hack is applied.

vvasabi on 16 Aug 2017

@vvasabi THANK YOU!!! This has been really helpful in getting my test run times down. I'm probably adding far too much metadata to each test which means I need to refactor my components to be more modular (which I'm not quite sure how to do), but in the meantime your suggestion has led to a massive gain in test execution performance.

Before: Executed 502 of 502 (4 FAILED) (16 mins 24.944 secs / 15 mins 51.429 secs)

After: Executed 502 of 502 SUCCESS (2 mins 53.59 secs / 2 mins 5.877 secs)

Still slower than I'd like, but biiiiig improvement!

(Yes, these tests run on a particularly slow environment, where we often get async timeouts that don't happen locally. Simply running the tests faster seems to solve these).

In case someone else needs more specific details for how to apply your code, here it is.

Create a new file called src/test.common.spec.ts (FYI: I had to make some small changes here compared to what you had above). The .spec.ts extension is so that the file isn't included in builds - otherwise ng build will complain with a Cannot find name 'beforeAll' error.

Add the following into the file:

import { TestBed, async, TestModuleMetadata } from '@angular/core/testing';

const resetTestingModule = TestBed.resetTestingModule,
  preventAngularFromResetting = () => TestBed.resetTestingModule = () => TestBed;
let allowAngularToReset = () => TestBed.resetTestingModule = resetTestingModule;

export const setUpTestBed = (moduleDef: TestModuleMetadata) => {
  beforeAll(done => (async () => {
    resetTestingModule();
    preventAngularFromResetting();
    TestBed.configureTestingModule(moduleDef);
    await TestBed.compileComponents();

    // prevent Angular from resetting testing module
    TestBed.resetTestingModule = () => TestBed;
  })().then(done).catch(done.fail));

  afterAll(() => allowAngularToReset());
};

In a specific .component.spec.ts file there may be something like the following:

describe('SomeComponent', () => {
  beforeEach(async(() => {
    TestBed.configureTestingModule({
      declarations: [ OneComponent, TwoComponent ],
      imports: [ SharedModule ],
      providers: [ HttpService ]
    }).compileComponents();
  }));

This then becomes:

describe('SomeComponent', () => {
  let moduleDef: TestModuleMetadata = {
    declarations: [ OneComponent, TwoComponent ],
    imports: [ SharedModule ],
    providers: [ HttpService ]
  };
  setUpTestBed(moduleDef);

Run the tests with ng test

It's kinda obvious, but don't use this pattern any time you need the component/service/whatever you're testing state to be reset between tests.

BurningDog on 16 Aug 2017

👍21

I tested @vvasabi`s work around, and it really worked! My test suit is now almost 10x faster!

Is there any official response from the core team on this subject? Is there any plans to make it faster without this kind of work around?

EDIT: I just found the PR #17710 that would solve this issue, but it is more than 2 months without answer.

willgm on 4 Sep 2017

👍1

I also have a similar issue in the tracker (https://github.com/angular/angular/issues/13963) regarding the fact that all test dependencies are recompiled for every test method. I think if the TestBed would just compile once and then re-use the compiled output for subsequent test methods, that would speed everything up _a lot_.

brian428 on 11 Sep 2017

👍2

I just used this mechanism... I had 100 tests that took about 2min48sec. Converted them over, and the execute in 8sec

ronnyek on 9 Oct 2017

👍2

@robwormald and @alxhub, may we have an update on this tread? There is also an open PR to fix that without any feedback for a long time.

One of the main features of Angular is testability, and this performance issue is really a problem. I have a production app with ~1500, and it is impossible to use TDD without this ugly workaround.

willgm on 9 Oct 2017

👍2

totally agree. i also use the workaround, i encapsulated it so it is well usable. however this should be sth ng offers built in (setup in beforeAll). also in general the test setup fells clumsy in ng. this includes the setup....and injection of services as well....have no comparison how it is in e.g. react...i have the feeling it could be easier somehow...i encapsulated for better usability like this:

let comp: Comp;
setUpTestBed(CfModule, () => comp = new Comp(CfComponent));

for injection i still use the clumsy stuff:

let _restService: RestService;
let _dataService: DataService;

beforeEach(inject([RestService, DataService], (restService: RestService, dataService: DataService) => {
_restService = restService;
_dataService = dataService;
}));

dasAnderl on 11 Oct 2017

@dasAnderl To get a service injected you could also do:

_restService = <RestService>TestBed.get(RestService);

If you want to make it generic (so no casting is required), you can make a util method that like this:

const get = <T>(type: Type<T>): T => <T>TestBed.get(type);

Then, the above example becomes:

_restService = get(RestService);

vvasabi on 11 Oct 2017

👍2

As I mentioned over in https://github.com/angular/angular/issues/13963, the real problem with this workaround is that the dependencies aren't recreated between tests. Meaning any state held by the dependencies can cause bizarre issues. All I think we really need is the option to have the compiled test module reused, since the vast majority of the time is usually taken up in simply compiling templates for the test module.

brian428 on 11 Oct 2017

👍12

@vvasabi Thanks for the workaround, more than 200 tests that took 50 sec before take now 1.5 sec.

flemgs on 25 Oct 2017

👍1

had the same problem and found this:
https://github.com/Quramy/ngx-zombie-compiler

works like a charm

JanEggers on 18 Jan 2018

🎉1

Unfortunately it doesn't work with Angular 5 because the underlying API changed.

However, TestBed now seems to cache the set of aotSummaries used by the test. So it seems like all that would be needed to speed things up substantially would be an option to tell the TestBed to reuse the aotSummaries for subsequent spec methods instead of clearing and recreating them for each spec method.

brian428 on 18 Jan 2018

For anyone that might be interested : with the release of angular 5, i was finally able to move from Karma to Jest. The results for 3250 tests on our monorepo is :
Karma : 12 minutes (2 mins effective run time, 10 compilation) full run
Jest: 126 secs full run

In addition to that, i've installed the vscode plugins to runs tests on the fly in the editor one by one.
This is such a massive gain that i can't even begin to say how happy i am about it :)

And i think that using @vvasabi workaround i might get even better results. Going to give it a try.

Thanks you so much @thymikee for pointing me to Jest.
If anyone wants more info about this, feel free to contact me :)

Shireilia on 19 Jan 2018

We should create a sample repo with best practices for test setup - Having a template cache or setting up AOT summaries.

vikerman on 26 Jan 2018

👍10

@vikerman That would be awesome. It's probably not all that complicated, but since it's undocumented, I just have no idea how to actually do anything with the AOT summaries.

brian428 on 26 Jan 2018

@vikerman @Shireilia could you provide some example somewhere on how to get JEST and maybe combined with @vvasabi hack which could be a starting point for good speedy testing practice? We struggle to get a proper solution to work and could be interested to have a look at the way you were able to get it done.

blackholegalaxy on 6 Mar 2018

@blackholegalaxy sure, i still have to implement @vvasabi hack (business requirement took too much time lately), but got Jest wroking at least. I don't know if i'll have the time to setup a repo in the comming days, but i can share our configuration here as a quick start.

We're using https://github.com/gdi2290/angular-starter as a starter. Nothing much changed exept for how we handle translations (should not have any impact).

Also, make sur to read and follow @thymikee article on the topic : https://www.xfive.co/blog/testing-angular-faster-jest/ wich is the one i used to make things work. The files described in his post are the one i'll put below.

First, we didn't touch the test other than fixing some import statements that were uncorrect, so if you're using karma + jasmine already you should not have to change anything in them. That's important, because @thymikee recommend to change some things regarding the matchers and spys, but we didn't have to do it. So, at first, i'd really recommend to leave your test files untouched.

In our package json, i added this rule:



"jest": {

    "globals": {

      "ts-jest": {

        "tsConfigFile": "src/tsconfig.spec.json"

      },

      "__TRANSFORM_HTML__": true

    },

    "preset": "jest-preset-angular",

    "setupTestFrameworkScriptFile": "/src/setupJest.ts",

    "transformIgnorePatterns": [

      "/node_modules/(?!@ngrx|@VENDOR/FRAMEWORK-NAME)"

    ],

    "testRegex": "(/__tests__/.*|\.(spec))\.(ts|js)$"

  }

Most of the problems i had to get it work was because we have a custom framework (services implementing the hypermedia pattern for our apps mostly, but also some form validators, various utils for logs, store management etc...) and had to add it in the transformIgnorePatterns:



 "/node_modules/(?!@ngrx|@VENDOR/FRAMEWORK-NAME)"

Remove what's unecessary in the regex :)

For a little bit more input, we had to follow the best practices on how we managed it, and that's providing a compiled version, not the .ts files of the framework. After that, everything went nice :)

Our jestGlobalMocks.ts file:



import 'rxjs/add/observable/of';

import 'rxjs/add/observable/throw';

import 'rxjs/add/observable/empty';

import 'rxjs/add/observable/forkJoin';

import 'rxjs/add/observable/timer';

import 'rxjs/add/observable/fromEvent';

import 'rxjs/add/observable/merge';

import 'rxjs/add/observable/interval';

import 'rxjs/add/observable/combineLatest';

import 'rxjs/add/operator/map';

import 'rxjs/add/operator/switchMap';

import 'rxjs/add/operator/share';

import 'rxjs/add/operator/do';

import 'rxjs/add/operator/catch';

import 'rxjs/add/operator/filter';

import 'rxjs/add/operator/take';

import 'rxjs/add/operator/combineLatest';

import 'rxjs/add/operator/debounceTime';

import 'rxjs/add/operator/delay';

import 'rxjs/add/operator/distinctUntilChanged';

import 'rxjs/add/operator/skip';

import 'rxjs/add/operator/last';

import 'rxjs/add/operator/finally';

import 'rxjs/add/operator/takeWhile';

import 'rxjs/Observable';

import 'rxjs/Subject';

const mock = () => {

  let storage = {};

  return {

    getItem: (key: any) => key in storage ? storage[key] : null,

    setItem: (key: any, value: any) => storage[key] = value || '',

    removeItem: (key: any) => delete storage[key],

    clear: () => storage = {},

  };

};

Object.defineProperty(window, 'localStorage', { value: mock() });

Object.defineProperty(window, 'sessionStorage', { value: mock() });

Object.defineProperty(window, 'getComputedStyle', {

  value: () => ['-webkit-appearance']

});

Object.defineProperty(document.body.style, 'transform', {

  value: () => {

    return {

      enumerable: true,

      configurable: true

    };

  },

});

We centralised the import for RXJS in this file, will do better in the future ;)

setupJest.ts:



import 'jest-preset-angular';

import './jestGlobalMocks';

tsconfigs.spec.json:



{

  "compilerOptions": {

    "target": "es5",

    "module": "commonjs",

    "moduleResolution": "node",

    "emitDecoratorMetadata": true,

    "experimentalDecorators": true,

    "allowSyntheticDefaultImports": true,

    "sourceMap": true,

    "skipLibCheck": true,

    "noEmit": true,

    "noEmitHelpers": true,

    "importHelpers": true,

    "strictNullChecks": false,

    "allowUnreachableCode": false,

    "allowUnusedLabels": false,

    "noUnusedLocals": true,

    "noUnusedParameters": true,

    "removeComments": true,

    "noImplicitAny": true,

    "suppressImplicitAnyIndexErrors": true,

    "lib": [

      "dom",

      "es6"

    ],

    "baseUrl": "./src",

    "paths": {

      "@angular/": [

        "node_modules/@angular/"

      ]

    },

    "typeRoots": [

      "node_modules/@types"

    ],

    "types": [

      "hammerjs",

      "jest",

      "node",

      "source-map",

      "uglify-js",

      "webpack",

      "fuse"

    ]

  },

  "exclude": [

    "node_modules",

    "dist"

  ],

  "awesomeTypescriptLoaderOptions": {

    "forkChecker": true,

    "useWebpackText": true

  },

  "compileOnSave": false,

  "buildOnSave": false,

  "atom": {

    "rewriteTsconfig": false

  }

}

Our actual jest version numbers :



"@types/jest": "22.1.1",

"jest": "22.2.1",

"jest-preset-angular": "5.0.0",

And that should be it.

P.S : we're using Angular 5+.

Shireilia on 6 Mar 2018

👍2

I think the problem is that initTestEnvironment is called on every tests run so if you are on watch mode the only good way would be to cache compiled templates. Angular currently is not exposing cache https://github.com/angular/angular/blob/master/packages/compiler/src/jit/compiler.ts#L41

If it would be possible to give custom cache store implementation for testing then we could store cache on global __karma__ object something like this.

__karma__.compiledTemplateCache = __karma__.compiledTemplateCache || new Map<Type, CompiledTemplate>();

@Injectable()
class TemplateCacheStore {
  getTemplateCache(meta) {
    // ... get from global __karma__.compiledTemplateCache
  }

  setTemplateCache(meta) {
    // set to global __karma__.compiledTemplateCache
  }
}

getTestBed().initTestEnvironment(
  BrowserDynamicTestingModule.configure({ templatesCacheStore: TemplateCacheStore }),
  platformBrowserDynamicTesting()
);

anjmao on 21 Mar 2018

The hack by @vvasabi sounds like it would work, at least in some situations. But going through and changing hundreds of spec files would be very time consuming. And there's no telling how changes to future versions of Angular will break the workaround.

The bottom line is that needing to apply hacks like that is really just absurd. It simply shouldn't take 10 minutes to run UI unit tests. It basically cancels out the whole point of TDD.

What's even more frustrating is that all of these problems would be nullified if we could just tell Angular to re-use the AoT summaries and/or re-use the compiled test module. This issue has been ongoing for over 18 months, and it's baffling to me that the Angular folks just seem to be ignoring it.

I don't understand all the inner workings of the TestBed and the test compiler, but from what I can tell, this should be pretty easy to do for someone who actually understands the internals. Angular seems to already create AoT summaries in memory, so exposing a flag to cache and reuse them within a spec file instead of re-creating them each time seems like a straightforward change.

@vikerman was any further thought put into documentation or examples for using a template cache or re-using the AoT summaries?

brian428 on 21 Mar 2018

👍31

I agree... I feel validated by the fact that this issue isn't just affecting us because of some mistake of mine, and it's encouraging to see the community put forth work-arounds, but also disappointing that we're left with implementing hacks with no road-map from core team members... I basically gave up trying to get my team to write unit tests for components... hopefully integration tests will serve us well.

ollwenjones on 21 Mar 2018

I've made a feature request at #22797 which addresses this issue too.

It would:

provide a safer "shallow render" experience than NO_ERRORS_SCHEMA.
prevent the problem of having to declare (or mock) the entire dependency graph of the component under test.
keep guarantees of correctness in respect to the selector and inputs/outputs of child components, making the unit tests resistant to breakage via refactorings.

Please take a look and provide feedback, since I think this would vastly improve the testing experience of Angular apps.

michaelbromley on 23 Apr 2018

👍2

@michaelbromley This is a great idea and I love it, but it seems like a parallel option rather than a real fix for the underlying problem (that the components are recompiled over and over again when they don't need to be). Regardless, I'm on board with the mocking idea 100%.

brian428 on 23 Apr 2018

👍1

Also just to show how solvable this problem seems to be, I set up a global function that is called in the beforeAll() block of all of my tests. It does a stupid, ridiculous hack to replace the logic in resetTestingModule().

I wouldn't recommend that anyone actually use this, but it seems to work fine and it speeds things up by nearly 40%. My point isn't to offer a solution for anyone to use, but simply to show how easy it seems to be to vastly speed things up just by reusing the compiled components:

let tb: any = getTestBed();

// Switch back to original resetTestingModule() function if it exists
if( ( window as any )[ "__testCache" ] ) tb.resetTestingModule = ( window as any )[ "__testCache" ].realResetTestingModule;

// Do a reset on the testing module.
TestBed.resetTestingModule();

// Store reference to original resetTestingModule() function.
let realResetTestingModule = tb.resetTestingModule;
( window as any )[ "__testCache" ] = { realResetTestingModule: tb.resetTestingModule };

// Replace original resetTestingModule() with a custom version that re-uses the moduleFactory and compiler.
// This cuts the test execution time by roughly 40%.
tb.resetTestingModule = () => {
  let mf = tb._moduleFactory;
  let compiler = tb._compiler;
  realResetTestingModule.apply( tb );
  tb._moduleFactory = mf;
  tb._compiler = compiler;
};

brian428 on 23 Apr 2018

@vmandy (I know I'm late) .. but facing this problem of slow unit, out of memory errors as well

The two things I did to get around this are:

1) All variables that are used in beforeEach, are declared at the top of the file inside the first describe. e.g.

describe("FooComponent", () => {
 let var0: TypeFoo;
 let var1: TypeFoo;
 let var2: TypeFoo;

 beforeEach(() => {
   var0 = // something big
   var1 = // something big
   var2 = // something big
 })
 afterEach(() => {
   var0 = undefined
   var1 = undefined
   var2 = undefined
 })

 // my other 'describe', 'beforeEach' and 'it' blocks afterwards
});

This will fixed my out of memory problems.

2) To speed things up. Replace all your IonicModule bits with NO_SCHEMA_ERRORS e.g:

Replace:

imports: [
 IonicModule.forRoot(component),
]

with:

schemas: [
 NO_ERRORS_SCHEMA
]

And you're all good!

ChrisGibb on 25 Apr 2018

@vvasabi @BurningDog @brian428 : is this hack still works with angular 5.1.3?
because i have configured the test suit same way and able to execute the single spec file. but when i try to execute all the spec files, the first one get passed and all other test cases get failed with following error.

Failed: Uncaught (in promise): Error: Illegal state: Could not load the summary for directive AccountDetailsComponent.

if not what are possible workarounds available for me, because our current test suit taking around 10 to 12 min to execute the test cases and if we multiple this number with head count of resources we are spending to much time just to execute the test cases.

kedar9444 on 22 May 2018

😕1

@kedar9444 My project is on Angular 5.2.0, and I have not had to change the way I implemented my hack since I originally published it last year. If you could share some of your code that reproduces the issue, maybe people can take a look.

vvasabi on 22 May 2018

@vvasabi : thanks for responding i can breath now!!! lets see what i have done in my code.

I have created one configuration file for all spec files :

export const configureTestSuite = () => {
    const testBedApi: any = getTestBed();
    const originReset = TestBed.resetTestingModule;

    TestBed.resetTestingModule();
    TestBed.resetTestingModule = () => TestBed;

    afterEach(() => {
        testBedApi._activeFixtures.forEach((fixture: ComponentFixture<any>) => fixture.destroy());
        testBedApi._instantiated = false;
    });

    afterAll(() => {
        TestBed.resetTestingModule = originReset;
        TestBed.resetTestingModule();
    });
}

then used the same in actual test case :

describe('Component: xxxComponent', () => {
    configureTestSuite();

    beforeAll(done => (async () => {

        TestBed.configureTestingModule({
            imports: [SharedModule, BrowserModule...],
            declarations: [xxxComponent, yyyComponent],
            schemas: [CUSTOM_ELEMENTS_SCHEMA],
            providers: [
                { provide: xxxService, useClass: xxxServiceStub }
                ...
            ],

        })

        await TestBed.compileComponents();

    })().then(done).catch(done.fail));

    describe('Component: templateFormComponent', () => {

        let xxxfixture: ComponentFixture<xxxComponent>;
        let xComponent: xxxComponent;

        let yyyfixture: ComponentFixture<yyyComponent>;
        let yComponent: yyyComponent;

        let xService: xxxService
        let el: HTMLElement;

        beforeEach(() => {

            xxxfixture = TestBed.createComponent(xxxComponent);
            xComponent = xxxfixture.componentInstance;

            xService = TestBed.get(xxxService);
        });
    });
});

with this implementation if i tried to run any single spec file [with fdescribe] it gives proper result, but if i tried to run all the spec files in one go the configuration runs the first spec file correctly but through error for all other spec files.

and the error is :

Failed: Uncaught (in promise): Error: Illegal state: Could not load the summary for directive xxxComponent.

if you need more data please mention in comment. thanks in advance. :)

here are some more details:

Angular CLI: 1.7.4
Node: 9.11.1
OS: win32 x64
Angular: 5.2.4
... animations, common, compiler, compiler-cli, core, forms
... http, language-service, platform-browser
... platform-browser-dynamic, router

@angular/cli: 1.7.4
@angular-devkit/core: 0.0.29
@schematics/package-update: 0.3.2
typescript: 2.4.2
webpack: error

kedar9444 on 22 May 2018

here is the code to reproduce the issue.

https://github.com/kedar9444/unit-test-performance

kedar9444 on 23 May 2018

@kedar9444 The Github repo was helpful. All I did to fix your issue was to have this in configure-test-suit.ts:

beforeAll(() => {
  TestBed.resetTestingModule();
  TestBed.resetTestingModule = () => TestBed;
});

You need those 2 lines in a beforeAll block. Hope this helps.

vvasabi on 23 May 2018

After adding the beforeAll part I get this error: Cannot configure the test module when the test module has already been instantiated.
I call configureTestingModule in my beforeEach hooks.

blacksonic on 29 May 2018

👍1

@blacksonic The idea behind this hack is that time is saved by not having to call configureTestingModule once per it block to recompile components. As such, the complaint that the test module has already been instantiated will go away if you only call configureTestingModule once per the outermost describe block.

vvasabi on 30 May 2018

I created a pull request for this problem last year but never got any feedback from the Angular team. (#17710). We are using code very similar to that pull request at my employer in an Angular app with more than 1.000 unit tests.

oocx on 20 Jul 2018

Testing in Angular is really problematic right now because of this performance issue. We are also on a 1000+ tests app, and because of this hack we were able to go from about 30 minutes to about 3 minutes for a full test run. This is really bad, a 30m test run is unacceptable. And even 3m is bad for unit tests for just 1000+ tests.

The problem lies on all the rendering and the dependency on a browser. I hope angular switches to something like JSDom ASAP (and away from Karma and Jasmine), otherwise tests will never be as fast as they have to be.

giggio on 25 Jul 2018

❤8 👍3

@giggio You can already use JSDom with Angular unit tests via Jest. Look for thymikee’s comments in this thread.

vvasabi on 27 Jul 2018

The problem isn't in the real vs fake DOM approach. jsdom-based tests with Jest can cut running time to half. Still too slow for even a dozen tests. I'm afraid Jest has still a long way to go to take advantage of parallelism & fake DOM which in theory looks much better than current implementations. The community would benefit from this of course.

To achieve a tenfold decrease in running time Angular needs to allow the testing module to be configured in a beforeAll() block. Right now it's just naively discarding it after each test. This should be prioritized until caching compilation of sorts takes place. We don't need the _best_ solution now, but we deserve a non-naïve solution at least until something better is created.

So do we want to have a 2x or 10x speedup? And how long are we willing to wait?

awerlang on 27 Jul 2018

👍2

Something that has worked well for us when testing our component library is splitting it up into smaller libraries where possible, then running these library tests in parallel on CI nodes, while also using Karma plugins such as karma-parallel to parallelize even further.

That, coupled with overriding of the resetTestingModule and managing the reset manually as previously mentioned in this thread, has helped reduce times across 6000+ specs enough while we wait for a Bazel / caching build based solution.

mattem on 27 Jul 2018

👍1

we should'nt have to rely on CI tricks to speedup the tests. I think @awerlang is right: Angular should act in a way the tests are ran properly without rebilding the whole component if the setup can be consistent between the tests.

blackholegalaxy on 27 Jul 2018

The frustrating thing about this is that, as I showed earlier in this thread, the logic to simply hold onto the compiled test module is trivial. I just have no idea how to "properly" change the actual Angular code within TestBed or Compiler to allow this, or I'd have sent a pull request long ago.

I realize that there are probably edge cases where my solution won't work (e.g. where individual specs are changing/overriding different parts of the test module), but having a flag to turn the compilation result caching on or off would also be a simple addition.

I'm really just totally baffled that the problem has been this bad for this long, yet no one on the team seems to be prioritizing this.

brian428 on 27 Jul 2018

👍25 😕3

Any update on this? Is any effort being made to speed up test execution when using the TestBed ? How do larger projects at Google that use Angular deal with the slow tests ?

abeymg on 26 Sep 2018

I've just LGTM @oocx https://github.com/angular/angular/pull/17710.
In Google, tests are done with AOT by providing the NgSummaries to TestModuleMetadata
This is the recommended way to get the best performance, but we understand that it might not be easy to get hold of the NgSummary files.
With Ivy coming out soon AOT will be the default. In the meantime, please try out the suggestions in the PR.

kyliau on 26 Sep 2018

@kyliau Do you have any examples of passing AOT summaries here?

mattem on 27 Sep 2018

@kyliau Thanks for the response. I am already using the one of the workarounds mentioned earlier in this thread. My question was more about when the official support for this would land in the core. I'll follow the PR to see when it makes it out.

Question about what you mentioned about Ivy. With Ivy making AOT the default, does the workflow for tests using TestBed change?

abeymg on 27 Sep 2018

Hello everyone! I am trying to implement this approach with configureTestSuite function.
Did someone face issue when with this approach Karma says that I am running 'Empty test suite'?

Here the test code - https://gist.github.com/alex-kel/e549a845e37b00678a17da7340432683

@awerlang @vvasabi probably you guys can help me?

alex-kel on 28 Sep 2018

Hello,
In my team, we just used ng-bullet to accelerate the tests of our different projects.
This allowed impressing gains (ex: 2m30s to 30s), without any problem for the moment.

kekel87 on 29 Sep 2018

👍25 🎉12 ❤9

I tried using ng-bullet but still getting the same “Empty test suite”
message...
сб, 29 сент. 2018 г. в 13:47, Michael notifications@github.com:

Hello,
In my team, we just used ng-bullet
https://www.npmjs.com/package/ng-bullet to accelerate the tests of our
different projects.
This allowed impressing gains (ex: 2m30s to 30s), without any problem for
the moment.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/angular/angular/issues/12409#issuecomment-425635583,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AGshdnkc_522rtrlJCtn2TmcvZSlN4wAks5uf0_EgaJpZM4KcS5I
.

alex-kel on 29 Sep 2018

@kekel87 that's brilliant that someone took on that problem with a tidy library API! When I opened the issue, I never dreamed it would come 2 years later from a 3rd party, though.

ollwenjones on 8 Oct 2018

😄7

So @kyliau, with Ivy now seemingly delayed, can you give some guidance on how to provide the NgSummaries? Is there some example code for this? Does it still work with v7?

Thanks,
Chris

cyungmann on 25 Oct 2018

I've put together a little helper to re-use compilation results for given modules for all your tests. I'd love to hear if it helps others. It's the precompileForTests() function in s-ng-dev-utils.

It seems like the APIs available for things like this keep changing, so we'll see how long this can last. I created this with Angular 8. It sounds like better support for AoT in tests may be coming with Ivy? So maybe this solution won't need to last long! 🤞

ersimont on 29 Jun 2019

This looks an important issue, and becomes more critical as your modules grow.

Ideally it should just be possible to run the TestBed.configureTestingModule inside a beforeAll.

That would be the most elegant solution.

FrancescoBorzi on 14 Jul 2019

👍3

The blog post below explains how to skip the recompilation step so that you can use real dependency injection, compile the component once, and then run all the tests. Thank you to Nikita Yakovenko for sharing this solution!

https://blog.angularindepth.com/angular-unit-testing-performance-34363b7345ba.

davidjpfeiffer on 13 Aug 2019

🎉2

Yep, @davidjpfeiffer, as I mentioned above, this blog post update speak about ng-bullet.

Moreover almost all the projects of my teams have been there for almost a year, and it still works very well! Combined with ng-mock, It really does good jobs !

kekel87 on 13 Aug 2019

What is more disturbing is that this problem with TestBed has been there for years already.

DenysVuika on 13 Aug 2019

👍9

We have more than 600 tests in our project and it takes more than 5 minutes to execute everything. Or 3 minutes to start a simple test. I agree with @brian428 "It basically cancels out the whole point of TDD.".

Working this way is very very bad.

MatMercer on 3 Dec 2019

👍2

@MatMercer something that really helped me was a "recent" Angular update that lets you specify the flag --include, so during development I set up something like:

git diff HEAD^ --name-only | xargs -L 1 dirname | sort | uniq etc and eventually got the output to be something like {dir1/a/b/*.ts,dir2/a/c/*ts} which I passed to ng test --include ${test_glob?}.

It's ugly, but you can always call ng test --include dir/i/am/working/on/ too

I was able to move faster (especially combined with our fork of ng-bullet). And like others have mentioned, karma-parallel also helped bring down 4k tests from 20min to 5min on the full test suite

Goodwine on 3 Dec 2019

@Goodwine I'm currently using angular 5.2.0. ng-bullet for some reason causes problems with npm (the o'l peer dependencies problem). About the karma-parallel, it only bring it down from 5:20 to 4:40, not a huge difference. I tried using your suggestion, but since you already said

what really helped me was a "recent" Angular update

Since I'm using 5.2.0, I don't have the "recent" features.

About the other solutions presented here, I can't use them, since the tests changes the components/use spy a lot. I really don't know what else to do from here. Maybe upgrade the Angular version and pray for it to work?

MatMercer on 4 Dec 2019

you dont need the ng-bullet dependency. The implementation is pretty simple.

this is what we use in our project

import { getTestBed, TestBed } from "@angular/core/testing"

export function configureTestSuite(configureAction: () => Promise<any>) {
  const testBedApi = getTestBed()
  const originReset = TestBed.resetTestingModule
  beforeAll(() => {
    TestBed.resetTestingModule()
    TestBed.resetTestingModule = () => TestBed
  })
  if (configureAction) {
    beforeAll((done) => {
      configureAction().then(done).catch(done.fail)
    })
  }
  afterEach(() => {
    testBedApi["_activeFixtures"].forEach((fixture) => fixture.destroy())
    testBedApi["_instantiated"] = false
  })
  afterAll(() => {
    TestBed.resetTestingModule = originReset
    TestBed.resetTestingModule()
  })
}

with that you have no peer dependency problems. Further you are in control of the behavior so you can revert it at any time, without changing your tests.

giniedp on 4 Dec 2019

@MatMercer

Since I'm using 5.2.0, I don't have the "recent" features.

Be aware that angular versions are only supported for a limited amount of time [docs]
And also, if the Angular team improves the test setup performance like you want, it would be on a
newer version, so you would have to upgrade anyways, there's really not workaround*.

On another Issue about something similar, someone mentioned that the next Angular release with Ivy and R3SomethingTestBed made test setup run much faster even without AOT.
https://github.com/angular/angular-cli/issues/6650

I found a workaround, it's not pretty, basically you use something like git diff HEAD^ --name-only and process the data to generate something that you can pass to const context = require.context(... in the test.ts file, but that means you have to be regenerating the HTML file every time (which I was doing using a gulp task watching changes for TS/HTML files)

(Disclaimer: I'm not a member of Angular team)

Goodwine on 4 Dec 2019

I'm currently refactoring my component tests (49 files) to use ng-bullet that I've added manually from here into my project as @giniedp said. The snippet @giniedp gave is a little bit outdated so I'm giving a warning to the newcomers: use the ng-bullet from here, it's updated and worked with my angular 5.2.0.

@Goodwine Indeed, what you have said is right, if they will give performance improvements in the next releases there is no "workarounds", we will need to upgrade it anyway.

After I finish the refactor I'll share the results here, I'll probably use karma-parallel too.

MatMercer on 4 Dec 2019

it is actually the very same implementation as the latest version of ng-bullet, except that i choose to skip await TestBed.compileComponents().

giniedp on 4 Dec 2019

@giniedp

The signature of your snippet that gave me compilation errors:

export function configureTestSuite(configureAction: () => Promise<any>) {

The signature of the ng-bullet project:

export const configureTestSuite = (configureAction?: () => void) => {

Probably the way we used them in our tests is different (I'm using based in the ng-bullet npm page), that's why your snippet didn't worked for me.

MatMercer on 4 Dec 2019

you are right. That is another minor difference in how we adopted from ng-bullet. The change on your side would have been

-   configureTestSuite(() => {
+   configureTestSuite(async () => {
        TestBed.configureTestingModule({
            // ...
        })
    });

for us it was important not to take away the flow control from the developer, so he is still able to do async stuff inside configureTestSuite

    configureTestSuite(async () => {
        await TestBed.configureTestingModule({
            // ...
        }).compileComponents()
        await moreAsyncStuff()
    });

giniedp on 4 Dec 2019

@giniedp @Goodwine

That's interesting @giniedp. In my use-case we don't care about the flow control.

I have to say thanks! I successfully added ng-bullet in the project and the tests execution went from 5m:30s to 30s. The problem I had with the peer dependencies before was because my node version was too outdated.

Adding karma-parallel didn't did much (28s instead of 30s) and broke a test so I didn't used it.

MatMercer on 4 Dec 2019

@Goodwine

On another Issue about something similar, someone mentioned that the next Angular release with Ivy and R3SomethingTestBed made test setup run much faster even without AOT.

Yes. enabledIvy: true (it's tsconfig.json's option, see https://angular.io/guide/ivy#opting-into-angular-ivy) configures to use TestBedRender3 as the TestBed implementation. It's TestBed for Render 3, a.k.a. Ivy. And Ivy JiT compiler has caching mechanism for compiled modules and components. ( See https://github.com/angular/angular/blob/8.2.14/packages/core/src/render3/jit/module.ts#L49-L66 ).

So if you turn Ivy on, TestBed.configureTestingModule(...).compileComponents() gets faster because the second or later compilation for the same components are skipped.

Quramy on 5 Dec 2019

🚀1

With Ivy, the TestBed infrastructure has been overhauled and no longer requires recompilation of testing modules. I have seen significant speedups (8x) in tests because of this change.

JoostK on 15 Mar 2020

@JoostK Good to hear!

Would nice to have an overview of the performance differences between:

Angular 8 with Karma
Angular 8 with Jest
Angular 9 with Karma (+Ivy)
Angular 9 with Jest (+Ivy)

krokofant on 15 Mar 2020

👍6

Hello everyone,

As @JoostK mentioned, Ivy TestBed was overhauled to avoid unnecessary recompilations between tests, which helped improve overall performance. We did some measurements to compare ViewEngine and Ivy TestBed versions using Material Components test suite, Angular.io app and Angular Framework unit tests - performance improvement was around 30-40%. We also received feedback from developers that they see even better results on their projects.

I’m closing this ticket and feel free to create new ones if you notice TestBed performance issues after switching to Ivy.

Thank you.

AndrewKushnir on 30 May 2020

This issue has been automatically locked due to inactivity.
Please file a new issue if you are encountering a similar or related problem.

Read more about our automatic conversation locking policy.

_{_This action has been performed automatically by a bot._}

angular-automatic-lock-bot[bot] on 30 Jun 2020

Angular: TestBed.configureTestingModule Performance Issue

Most helpful comment

All 110 comments

Related issues