Jest-preset-angular: Ivy compatibility

[wtho]

Background

To turn TypeScript into JavaScript, Angular uses the TypeScript compiler, while also taking advantage of its own enhancements and processes. This practice was taken to the next level when the novel Ivy compiler was introduced, which optimizes the build even more and makes Angular a high-level web framework.

If tests running in jest just apply the TS-to-JS-compilation via ts-jest on the Angular code and try to run a tiny Angular App for a test case, many specialized scenarios miss the enhancements the Angular compiler does during Angular development.

The following identified scenarios are possibly such, although there are probably more:

• automatic recognition of entryComponents #353
• ngcc compilation issues #347
• running a test app without ngZone #354

To keep up with Ivy and further Angular enhancements, we want to depend more on the tools Angular provides.

PR #406 shows we are able to add Angular transformers in the preset.

Basic Transformer Analysis

When looking for TransformatorFactory in the Angular codebase (July 2020), we find several transformers, such as

• inline_resources.ts A transformer to inline resources (such as templateUrl and styleUrls)
• Source File TransformFactory unclear - probably to keep track of which files are compiled
• Parens Wrapper Wraps content in parentheses
• Constructor Parameters Downlevel Transformer downlevels constructor parameters - for JIT compilation - we take advantage of this transformer in jest-preset-angular since #406
• R3 Transformer unclear - adds annotation for closure compiler
• Alias Transformer creates additional exports for exported aliases
• Import Preserving Transformer prevents import declarations from being removed by the compiler
• Declaration Transformer unclear handles .d.ts files
• Ivy Transformer applies changes from an IvyCompilation (incrementally?)

Further objects referenced as transformers are used in program.ts

---

While we could just try to include all of them, it is a much better path to understand what we are doing and why.

Any help analyzing these and figuring out what these do does help a lot.

[ahnpnl]

Hi @JiaLiPassion, I know that you work on zone.js but I wonder if you are also familiar with Angular compiler area ? We need some helps from Angular team to tackle this problem. If you can help us or you can help to get someone to help with this, really appreciated !

[ahnpnl]

Angular compiler design documentation can also be a helpful source https://github.com/angular/angular/blob/master/packages/compiler/design/architecture.md

[JiaLiPassion]

@ahnpnl , sure, I would like to help, and I will try to understand this issue and find out who to contact in the Angular team.

[ahnpnl]

The best scenario is somehow we can invoke a function which is exposed by Angular compiler. The function will create Angular TypeScript Program and produce compiled output. Then we can deliver the output back to jest.

If this scenario is possible, we can reduce lots of maintenance work and everything will be handled by angular source.

I suspect https://github.com/angular/angular/blob/d1ea1f4c7f3358b730b0d94e65b00bc28cae279c/packages/compiler-cli/src/perform_compile.ts#L237 is something might be useful for this approach.

[wtho]

Yeah, I think using the Angular compiler API is the optimal approach, no fiddling with transformers and such!

But we should take a good look at what Angular is doing different from the general compilation for testing with Karma as a reference.

[ahnpnl]

I have tried out

import * as path from "path";
import * as ng from '@angular/compiler-cli'
import * as ts from 'typescript'

const basePath = process.cwd()

function compile(
  oldProgram?: ng.Program, _overrideOptions?: ng.CompilerOptions, rootNames?: string[],
  host?: ng.CompilerHost): {program: ng.Program, emitResult: ts.EmitResult} {
  const options = {
    basePath,
    'experimentalDecorators': true,
    'skipLibCheck': true,
    'strict': true,
    'strictPropertyInitialization': false,
    'types': Object.freeze<string>([]) as string[],
    'outDir': path.resolve(basePath, 'built'),
    'rootDir': basePath,
    'baseUrl': basePath,
    'declaration': true,
    'target': ts.ScriptTarget.ES5,
    'newLine': ts.NewLineKind.LineFeed,
    'module': ts.ModuleKind.CommonJS,
    'moduleResolution': ts.ModuleResolutionKind.NodeJs,
    'enableIvy': false,
    'lib': Object.freeze([
      path.resolve(basePath, 'node_modules/typescript/lib/lib.es6.d.ts'),
    ]) as string[],
    // clang-format off
    'paths': Object.freeze({'@angular/*': ['./node_modules/@angular/*']}) as {[index: string]: string[]}
    // clang-format on
  }
  if (!rootNames) {
    rootNames = [path.resolve(basePath, 'e2e/test-app-v9/src/app/app.component.ts')];
  }
  if (!host) {
    host = ng.createCompilerHost({options});
  }
  const program = ng.createProgram({
    rootNames: rootNames,
    options,
    host,
    oldProgram,
  });
  const emitResult = program.emit();
  return {emitResult, program};
}

test('test test', () => {
  console.warn(compile().emitResult)
})

and discover a few nice things:

• Use internal angular compiler can help us to achieve almost similar to the normal compilation via ng serve or ng build.

• Ivy option can be passed in via extended tsconfig option (see codes above), this will automatically create the correct compiler instance, either NgtscProgram for ìvy or AngularCompilerProgram for VE.

• Internal Angular transformers are already included in the execution. Some transformers are opt-in/out by tsconfig options.

• When calling emit for VE or emit for Ivy, we can add our custom transformers (if needed).

• By default, VE/Ivy compiler will write compiled output to build directory, we can opt-out this behavior by changing writeFile of ng.CompilerHost.

• Since VE/Ivy is just extended version of TypeScript Program, so we can still do anything extra using basic TypeScript Compiler API.

• There are couples of helper functions which can help us:

  • Create Angular compiler options: https://github.com/angular/angular/blob/master/packages/compiler-cli/src/perform_compile.ts#L131

  • Read typescript configuration: https://github.com/angular/angular/blob/master/packages/compiler-cli/src/perform_compile.ts#L141

  • Perform compilation (this looks like the entry of compilation process) https://github.com/angular/angular/blob/master/packages/compiler-cli/src/perform_compile.ts#L246

  • And some other things in perform_compile.ts

Notes

• I can see the benefits of using internal Angular compiler codes, that will simplify a lot on our side related to transformers as well as in sync with Angular. This is also similar to the approach of vue-jest.

• The biggest downside is: we have to write this preset to be a jest transformer instead and we will depend less on ts-jest, probably only the some parts (e.g. hoisting) but not the complete compilation process.

[wtho]

@ahnpnl Awesome! This breakthrough is great!

Ultimately I see it like this: We can either go in the performance optimization by avoiding tsc and ts-jest completely by completely relying on babel and by adding some transformers for the magic Angular adds or we can use Angular's performance optimization by applying their whole compilation process.

The direction taken here is the latter.

To evaluate if this works as we imagine, we have to take some more steps. A roadmap for these next steps would be awesome. Please correct me, if I forgot something here, but it looks like this to me:

1. Add jest transformer angular-jest-transformer to jest-preset-angular
2. Make transformer use Angular compiler-cli
3. Add ts-jest's hoist transformer to compilation (for mocks)
4. Figure out if/how ng works with watch and caching
5. Test a lot

[ahnpnl]

To evaluate if this works as we imagine, we have to take some more steps. A roadmap for these next steps would be awesome. Please correct me, if I forgot something here, but it looks like this to me:

1. Add jest transformer angular-jest-transformer to jest-preset-angular
2. Make transformer use Angular compiler-cli
3. Add ts-jest's hoist transformer to compilation (for mocks)
4. Figure out if/how ng works with watch and caching
5. Test a lot

Yes that is my idea as well.

I did a bit investigation into how Angular CLI invokes Angular compiler. Here is the flow:

ng test ---> architect-command.ts (schedule run) --> create Angular Compiler Plugin (angular_compiler_plugin.ts) 

--> prepare a few transformers (Angular CLI transformers) --> creating Program --> compile codes with Program

Flow explanation

• Prepare a few transformers (Angular CLI transformers): there are some transformers which Angular CLI specific. Depending on enableIvy option as well as JIT/AOT mode, some transformers are opt-in/out, see https://github.com/angular/angular-cli/blob/master/packages/ngtools/webpack/src/angular_compiler_plugin.ts#L1028

• The list of transformers specific to Angular CLI can be found here https://github.com/angular/angular-cli/tree/master/packages/ngtools/webpack/src/transformers

• Creating Program, this can be either normal TypeScript Program or Angular Program, see https://github.com/angular/angular-cli/blob/master/packages/ngtools/webpack/src/angular_compiler_plugin.ts#L354 .

• JIT mode uses TypeScript Program and AOT uses Angular Program. Both programs use the same set of files, compiler host, compiler options. I think we only need to focus on JIT mode for now.

• When compiling codes, both programs use the same set of Angular CLI transformers, see https://github.com/angular/angular-cli/blob/master/packages/ngtools/webpack/src/angular_compiler_plugin.ts#L1328 .

Unknown thing

No idea how this constant value is correctly set for Ivy/VE. This is one of the key things to make TestBed resolve the correct test compiler instance. Do you know something about this constant @JiaLiPassion ?

[ahnpnl]

So I found how the flag ɵivyEnabled value is set.

• ngcc will create a folder __ivy_ngcc__ in each Angular package type. This folder will contain the main script of that package together with source map to that script. Let's say we take an example of @angular/core.

In this picture is a part of @angular/core structure after ngcc. The file core.js is the main file of @angular/core.

• The file core.js which locates in __ivy_ngcc__ is loaded by Angular CLI (probably Angular CLI delegates that task to webpack, not sure) when enableIvy is false in tsconfig. Once this file is loaded, and in test file we do import { ɵivyEnabled as ivyEnabled } from '@angular/core';. The variable ivyEnabled will have the value false

• The same explanation applying to enableIvy: true, which then core.js in core/fesm2015 is loaded which result in the value of ivyEnabled is true if we import it into test file.

So I think our task should first look into AngularCompilerPlugin to replicate some logic here, including something like downlevel metadata.

A new question comes up is: what does TestBed do besides setting up test environment ? Does it compile the component code and put in some sort of memory ? According to https://github.com/angular/angular/issues/38600#issuecomment-682854823

This is indeed a known difference between View Engine and Ivy - in View Engine, components are compiled in the context of some environment - either a platform browser instance, or a test.

In Ivy, there is only one compiled version of a component at any time, and this compilation happens independently of the TestBed

So it seems like the component compilation is done separately in Ivy while TestBed only set up testing environment (like jest globalSetup, globalTeardown, setupFiles, setupFilesAfterEnv). If so, we need to find out how and when the component compilation is executing.

[ahnpnl]

we got an answer from one Angular team member, he suggested that we can open an issue which they can give some insights about how things work internally

[wtho]

Wow, this is great!
I am really looking forward to advance in this!

Thanks so much for all the effort you do in this direction.

[ahnpnl]

Today I asked Angular CLI team about reusing their transformers. They said those are private APIs so we are on our own to get it working.

Lucky part is most of necessary apis in compiler-cli are public.

Started a bit today with replace-resources. The conclusion is: we can’t use it because it requires webpack. So our 2 transformers: inline-files and strip-styles are required and must have.

[ahnpnl]

Update

543 introduces the usage of Angular compiler which solves the issue with entryComponents in #353. The compiler is still in very early stage so not all tests in e2e working yet but I think we are going the right direction.

This compiler tries to replicate the behavior of Angular Compiler Plugin. The compiler only follows the path for JIT mode, AOT mode is more complex and requires ESM full support which is not available now in jest. In the future, it is a nice to have.

Apparently, using Angular compiler host via createCompilerHost seems to do the trick after running ngcc. Running ngcc is a must-have step.

Regarding to the usage of transformers:

• inline_resources.ts A transformer to inline resources (such as templateUrl and styleUrls) : this transformer is only used in AOT mode but it is a very nice transformer which we can learn from it to combine our inline-files + strip-styles into one. @wtho I think this one is interested for you.

• Constructor Parameters Downlevel Transformer downlevels constructor parameters - for JIT compilation - this transformer is a must-have to support forwardRef

I only see the need of using these 2 transformers from Angular, not sure yet about other transformers.

[wtho]

Yeah, I had a look at these two transformers, I used the downlevel transformer in #406 and took some inspiration from the inline_resources.ts for the inline/strip transformers.

I can take over the integration of these in the AngularJestTransformer :+1:

[ahnpnl]

Yes the benefit of using inline-resource for us is:

• gaining performance by walking less 1 time on AST tree

• gaining performance by not asking jest to pass HTML to our jest transformer which saves 1 time transpiling.

A potential future feature is type checking template, but that seems to be only done with AOT because AOT has static analysis.

Anyways, it’s a good transformer for us to follow and apply. I’m not sure if that will affect JIT tests or not, worth to try out.

[ahnpnl]

According to https://github.com/angular/angular-cli/pull/15044, ngcc should always run before running test. So we should find a way to achieve the similar behavior.

I am thinking about 2 ways to tackle this:

• Force users to run ngcc on their own before running tests.

• jest-preset-angular runs ngcc before running tests.

Force users to run ngcc

We can check node_modules to find core-testing.umd.js.__ivy_ngcc_bak. If this file doesn't exist, simply throw an error to indicate that users should run ngcc first.

This approach has the advantage that it helps us to reduce the amount of work on jest-preset-angular and dedicate that to Angular CLI.

The disadvantage is not so convenient. But since ngcc only needs to run in postinstall, I think this approach looks fine.

jest-preset-angular runs ngcc before running tests

To achieve this, we need to create an async function and ask users to use it in jest.config.js via jest config async. For example.

// jest.config.js
module.exports = async () => {
  await runNgccProcess()

  return {
    verbose: true,
  };
};

Because jest.config.js only runs once before jest workers spawn for parallel tests, that is the only place we can run script once. ngcc should not be run inside jest transformer because each jest worker uses one singleton jest transformer instance which we can end up running ngcc for all workers if we run ngcc inside jest-preset-angular transformer.

Besides, if I am not wrong, when using jest projects option, jest will read each project jest.config.js. That means the async function which we expose can only run in root jest.config.js.

Overall, I prefer the approach Force users to run ngcc

What is your opinion @wtho ?

[wtho]

I think the second version will be much more convenient for ci environments. If we manage runNgccProcess() to be fast in non-ngcc-running cases, we should offer it to our users. Also it could be a CLI command coming with jest-preset-angular.

I really like the second version, but this is a comfort-level feature. We have no hurry to implement it. We can open a new issue and maybe find contributors who would want to tackle this.

We could offer something like this:

// jest.config.js
const initNgcc = require('jest-preset-angular/init-ngcc')

module.exports = async () => {
  await initNgcc()

  return {
    verbose: true,
  };
};

[ahnpnl]

Ye expose an async function is simple. The only problem is it is limited to use only in root jest.config.js. The error handling part will be a bit complicated since it needs to check whether or not this function runs in root config.

I think in the meantime, we can implement 1st approach. 2nd approach is nice to have so it can come later.

Seem like angular already had something for us, see example in https://github.com/angular/angular/blob/master/packages/compiler-cli/ngcc/main-ngcc.ts

[ahnpnl]

More information on ngcc

Operations done by ngcc

• Scan the node_modules for packages that are in APF Angular package format by checking .metadata.json files. (It just checks for their existence to determine whether a package is an Angular package or not.).
  Produce Ivy compatible versions for each of these packages

• Produce Ivy compatible versions for each of these packages

Reference: https://indepth.dev/angular-compatability-compiler/

So that means ngcc only cares about stuffs in node_modules.
The option about this preset runs ngcc is the most needed in watch mode. In non-watch mode, it is recommended to run in postinstall hook (applicable for CI too).

[ahnpnl]

v9.0.0-next.1 is out under next tag.

With this release, it is now possible to use jest-preset-angular with Ivy, see PR #562 . One note is to make sure everything run correctly, please run ngcc in postinstall hook.

We, the maintainers, highly appreciate the help from community with testing. Feel free to open any issues related to this next version.

[bbarry]

What is stopping this next version from going live? It seems to work for me at least (not hitting #347 or #353 anymore with various workarounds removed). Having followed along for a few months now I appreciate the complexity of the changes.

Are there any specific aspects you would like us to pay more attention to?

[ahnpnl]

Currently we are checking about performance. I myself saw a downgrade in performance with my work's projects so still needs some investigation there. In general, for now there are 2 blockers:

• Jest 27: it is expected to be released before xmas and it has some breaking changes for jest transformer to adopt to. We want to release v9.0.0 together with jest 27 in 1 major version.

• Performance with next version.

[ahnpnl]

9.0.0-next.2 is out. This release improves performance over 9.0.0-next.1. At least from my work’s projects, I can see the performance is more less similar to 8.3.2.

[ahnpnl]

Updated I've tried to integrate ngcc like Angular CLI did. We can do that, but jest parallel mode will fail. ngcc only works with jest runInBand. We have to find another way to integrate ngcc.

I found a way to run ngcc, see commit https://github.com/thymikee/jest-preset-angular/pull/562/commits/8ee7108015ec16d9c8d8c8fe606982cbb37e787c

[ahnpnl]

v9.0.0-next.3 is out. This release provides a util script to run ngcc in jest.config.js and a few bug fixes.

To use the util script, you can just define it in your root jest.config.js

// jest.config.js
require('jest-preset-angular/ngcc-jest-processor');

module.exports = {
     // ...
}

[snortblt]

Not sure if this is related but we're seeing Error: Error on worker #1: RangeError: Maximum call stack size exceeded. during the ngcc phase after adding the util script. It's thrown when compiling different packages depending on the run, so probably not associated with a particular package.

[ahnpnl]

Do you have a reproduce repo for it ? The util script tries to invoke ngcc instead of running ngcc by package managers.

[snortblt]

Looks like that was an issue with and outdated Jest package. With v6 it works. Still getting other errors but I suspect they're unrelated. Thanks.

[ahnpnl]

9.0.0-next.4 was out with support for Jest 27.

[johncrim]

I'm still seeing really slow start (after ngcc, it appears to hang before any tests complete, but it does finish the first tests after 5-8 minutes, then other tests complete approximately at "normal" speed). Do you have any suggestions for troubleshooting this phase of the test run? I was seeing this in the earlier next versions as well. Would be happy to open a separate issue, but it would be nice to be able to identify the cause.

For example, is there a way to list which files are being compiled?

[ahnpnl]

Setting the environment variable TS_JEST_LOG=ts-jest.log should produce a log file which contains all loggings.

In that file, there will be some loggings containing file name with the text "getCompiledOutput". That is the logging for file names which are compiled.

Also follow tips https://kulshekhar.github.io/ts-jest/docs/options/isolatedModules#performance will help a bit.

Bootstrap speed depends mainly on the amount of files to read. This includes Jest runtime file set + TypeScript compiler(internal jest-preset-angular) file set.

The optimal file set for TypeScript compiler should only include type definition files so compiler API will spend less time on reading file system.

[ahnpnl]

v9.0.0-next.5 was out. In this release we use replace resources transformer from @ngtools/webpack for isolatedModules: false. This will bring several benefits:

• jest-preset-angular now works like 95%+ similar to what Angular does with Karma + Jasmine
• The replace-resources transformer helps us to stay compatible with both CommonJS and ESM. Since Node 15 has marked ESM support as stable (see https://nodejs.org/api/esm.html#esm_modules_ecmascript_modules) and Jest is working on it, we have a nice preparation for that.

[ahnpnl]

v9.0.0-next.6 was out. In this release, both isolatedModules: false and isolatedModules: true use the similar set of AST transformers, except that downlevel ctor transformer only works with isolatedModules: false.

Besides, ESM support for both isolatedModules: false and isolatedModules: true was also completed. Now you can try out Jest ESM with Angular.