Angular-cli: Runtime errors when using buildOptimizer. JIT compilation broken.

[jcgillespie]

🐞 Bug report

Command (mark with an x)

• [ ] new
• [X] build
• [X] serve
• [ ] test
• [ ] e2e
• [ ] generate
• [ ] add
• [ ] update
• [ ] lint
• [ ] xi18n
• [ ] run
• [ ] config
• [ ] help
• [ ] version
• [ ] doc

Is this a regression?

I don't know. I would guess this is a recent thing arising from the introduction of AOT, Ivy, and the build optimizer.

Description

We faced an issue with an application we were developing where the legacy application had content stored in a database. This content is essentially an Angular template and could not be known at compile-time. The situations and constraints in the project led us to create dynamic components/modules on the fly. This worked beautifully in development and then failed when deployed to production.

In diagnosis, this works as long as the build optimizer is NOT enabled. Having a production build, with the optimizer, but without the buildOptimizer, enabled still works.

I believe I've seen unofficially in other issues that using AOT and JIT together, but since it works with the buildOptimizer disabled, this feels like an issue with the optimizer.

🔬 Minimal Reproduction

Here is a reproduction repo
Once you've npm install'd, you can run the repro this way.
ng serve <-- working
ng serve --configuration=production <--working
ng serve --configuration=production-build-optimized <-- broken.
The only difference between the configurations is the buildOptimizer setting.
You can run all three side-by-side-by-side by adding different --port values.

In a working build, you'll see an additional "Runtime Hello!" message as well as the Angular logo. When broken, those elements will be missing and there will be a console error.

If this isn't a bug, that error message is misleading for this situation.. :)

🔥 Exception or Error

main.7000374ef126bad8069c.js:1 ERROR Error: Uncaught (in promise): Error: Angular JIT compilation failed: '@angular/compiler' not loaded!
  - JIT compilation is discouraged for production use-cases! Consider AOT mode instead.
  - Did you bootstrap using '@angular/platform-browser-dynamic' or '@angular/platform-server'?
  - Alternatively provide the compiler with 'import "@angular/compiler";' before bootstrapping.
Error: Angular JIT compilation failed: '@angular/compiler' not loaded!
  - JIT compilation is discouraged for production use-cases! Consider AOT mode instead.
  - Did you bootstrap using '@angular/platform-browser-dynamic' or '@angular/platform-server'?
  - Alternatively provide the compiler with 'import "@angular/compiler";' before bootstrapping.
    at Ae (main.7000374ef126bad8069c.js:1)
    at Function.get (main.7000374ef126bad8069c.js:1)
    at St (main.7000374ef126bad8069c.js:1)
    at new Af (main.7000374ef126bad8069c.js:1)
    at ud (main.7000374ef126bad8069c.js:1)
    at dd (main.7000374ef126bad8069c.js:1)
    at e.md [as compileModuleAndAllComponentsAsync] (main.7000374ef126bad8069c.js:1)
    at e. (main.7000374ef126bad8069c.js:1)
    at Generator.next ()
    at main.7000374ef126bad8069c.js:1
    at T (polyfills.a8d522dff2c1a5c4b9db.js:1)
    at polyfills.a8d522dff2c1a5c4b9db.js:1
    at a (main.7000374ef126bad8069c.js:1)
    at l.invoke (polyfills.a8d522dff2c1a5c4b9db.js:1)
    at Object.onInvoke (main.7000374ef126bad8069c.js:1)
    at l.invoke (polyfills.a8d522dff2c1a5c4b9db.js:1)
    at i.run (polyfills.a8d522dff2c1a5c4b9db.js:1)
    at polyfills.a8d522dff2c1a5c4b9db.js:1
    at l.invokeTask (polyfills.a8d522dff2c1a5c4b9db.js:1)
    at Object.onInvokeTask (main.7000374ef126bad8069c.js:1)

🌍 Your Environment

     _                      _                 ____ _     ___
    / \   _ __   __ _ _   _| | __ _ _ __     / ___| |   |_ _|
   / △ \ | '_ \ / _` | | | | |/ _` | '__|   | |   | |    | |
  / ___ \| | | | (_| | |_| | | (_| | |      | |___| |___ | |
 /_/   \_\_| |_|\__, |\__,_|_|\__,_|_|       \____|_____|___|
                |___/


Angular CLI: 9.1.4
Node: 13.11.0
OS: darwin x64

Angular: 9.1.4
... animations, cli, common, compiler, compiler-cli, core, forms
... language-service, platform-browser, platform-browser-dynamic
... router
Ivy Workspace: Yes

Package                           Version
-----------------------------------------------------------
@angular-devkit/architect         0.901.4
@angular-devkit/build-angular     0.901.4
@angular-devkit/build-optimizer   0.901.4
@angular-devkit/build-webpack     0.901.4
@angular-devkit/core              9.1.4
@angular-devkit/schematics        9.1.4
@ngtools/webpack                  9.1.4
@schematics/angular               9.1.4
@schematics/update                0.901.4
rxjs                              6.5.5
typescript                        3.8.3
webpack                           4.42.0

Anything else relevant?

[karwank]

I'm facing a similar issue. The funny thing is that when I do "ng serve" and having "Live Development Server" started I change the name of "AppModule" to a different name, for example "AppsModule" in both files (app.module.ts and main.ts) it starts to work and vendor.js containst compiler.

[jcgillespie]

I'm facing a similar issue. The funny thing is that when I do "ng serve" and having "Live Development Server" started I change the name of "AppModule" to a different name, for example "AppsModule" in both files (app.module.ts and main.ts) it starts to work and vendor.js containst compiler.

@karwank I'm not sure I'm following. You renamed the module and it worked? That didn't work for me, but I might be misunderstanding your steps

[karwank]

It may be related with: https://github.com/angular/angular-cli/issues/11439

[jcgillespie]

It may be related with: #11439

Possibly. I looked at several other issues (many of which are linked in #11439), but this seemed different because it doesn't involve any third-party libraries. This is all just straight Angular functionality. I would agree with a lot of that discussion in 11439: some granularity of control over which optimizations are run would be nice. Better logging or transparency to what IS happening seems like must-have.

[karwank]

After spending a few days on this problem I have found this post: https://github.com/angular/angular/issues/23916#issuecomment-389214857. And point 3 is very sobering. I don't know exactly what you guys want to achieve. But I realized that we only need to have a way to create a dynamic contents which use already existing elements (components). For example:

<div>
  <p>This is a custom page with previously defined elements</p>
  <app-template-contact-form></app-template-contact-form>
  <app-template-map></app-template-map>
</div

I realized that I don't need to use JIT for this purpose. Components can be lazy loaded just like Kevin Kreuzer had shown here: https://medium.com/angular-in-depth/lazy-load-components-in-angular-596357ab05d8. So after analyzing these examples: https://github.com/LayZeeDK/ivy-dynamic-rendering, https://stackblitz.com/angular/ovpljlvlyxo, I have written my own service:

import {
  Injectable,
  Injector,
  ɵrenderComponent as renderComponent,
  Renderer2,
  ComponentFactoryResolver,
  ViewContainerRef
} from '@angular/core';

@Injectable({
  providedIn: 'root',
})
export class DynamicService {

  public renderer: Renderer2;
  public injector: Injector;

  constructor(
    private cfr: ComponentFactoryResolver,
  ) {}

  loadComponentToRef(componentName: string, target: ViewContainerRef) {
    let targetRendererMethod = this.renderComponentToRef;
    this.loadComponent(componentName, target, targetRendererMethod);
  }

  loadComponentToNode(componentName: string, target: HTMLElement) {
    let targetRendererMethod = this.renderComponentToNode;
    this.loadComponent(componentName, target, targetRendererMethod);
  }

  async loadComponent(componentName: string, target: HTMLElement | ViewContainerRef, targetRendererMethod: any) {

    let renderMethod = function(ComponentType: any, target: HTMLElement | ViewContainerRef) {
      let componentFactory = this.cfr.resolveComponentFactory(ComponentType);
      return targetRendererMethod.call(this, componentFactory, target)
    }

    switch (componentName) {
      case 'app-template-search':
        await import('@modules/template/search/search.component').then(esModule => { renderMethod.call(this, esModule.default, target) });
        break;
      case 'app-template-map':
        await import('@modules/template/map/map.component').then(esModule => { renderMethod.call(this, esModule.default, target) });
        break;
      default:
        break;
    }

  }

  renderComponentToNode(componentFactory: any, node: HTMLElement) {
    let compRef = componentFactory.create(this.injector)
    let parent = this.renderer.parentNode(node);
    this.renderer.insertBefore(parent, compRef.location.nativeElement, node)
    this.renderer.removeChild(parent, node);
    compRef.changeDetectorRef.detectChanges();

    return compRef
  }

  renderComponentToRef(componentFactory: any, ref: ViewContainerRef) {
    let { instance } = ref.createComponent(componentFactory, null, this.injector);
  }

}

[alan-agius4]

BuildOptimizer, indeed cannot be used with JIT code.

This also seems related to https://github.com/angular/angular/issues/36255 and https://github.com/angular/angular/issues/37216

In your case @jcgillespie, you can avoid using JIT mode and using above approach.

[maxtacco]

@alan-agius4 We are in the same boat as @jcgillespie. The approach described by @karwank is well-known but not viable when you have hundreds of possible template variations.

[loedeman]

Since #17947 has been closed as a duplicate and this one seems not to contain a solution yet:

I just created a reproduction scenario containing a dynamic component. Here it is: Angular JIT Production. The readme describes how to quickly reproduce the error mentioned above. Hope this gets resolved, because this currently prevents us from deploying production builds to our customers.

@JoostK already dived deep into the issue and commented to have found the issue: Build optimizer incorrectly considers @angular/compiler as having no side-effects, despite its package.json specifying "sideEffects": true. The culprit is here:

angular-cli/packages/angular_devkit/build_optimizer/src/build-optimizer/build-optimizer.ts
Line 28 in ba0f7ac

 /[\\/]node_modules[\\/]@angular[\\/]compiler[\\/]/, 

Removing that line fixes the issue.

[alan-agius4]

Thanks @loedeman and @JoostK for looking into this.

At the moment there is no story to official support a mixture of both AOT and JIT in the same application. The current stories are exclusive, you either use JIT or AOT.

I do think however, that probably we should remove that allowed list and solely rely on the sideEffects flag in the package manifest.

[JoostK]

Oh, yes, I forgot about the JIT interoperability in optimized builds. In optimized builds, NgModule information is not fully retained so JIT compiled components cannot use AOT compiled NgModules. So removing said line only fixes the issue where all of the compiler is not present, it may still cause issues further down when actually compiling a component.

[loedeman]

@alan-agius4 Okay, I get your point. However, we have a small scenario in which the Angular template for grid columns can only be served from the database. Is there another way to fulfill such a scenario, other than having a ton of extra Javascript (we are talking tens of megabytes difference)?

[FSDRE]

@loedeman @alan-agius4 I am in the same boat, receiving templates from an API (#18139).

"BuildOptimizer, indeed cannot be used with JIT code."

At the moment there is no story to official support a mixture of both AOT and JIT in the same application. The current stories are exclusive, you either use JIT or AOT.

These sentences worry me a lot, actually. I had no idea that those points are actually a thing. I have never come accross any statement or documentation stating the above. Correct me if I am wrong. I know a lot of developers doing the same thing as we do, relying on JIT and AOT including using buildOptimizer: true. Even on big developer conferences you see these kind of things farely often. If I had known better, I probably would have never chosen Angular as a framework for our project. Again, I am a little bit shocked right now.

The thing is: It actually makes a lot of sense in my opinion. Compiling the "static" part of the app AOT and feeding in a couple dynamically compiled components is an extremely flexible approach.

I hope there will be an official solution for these kind of implementations. Otherwise I am pretty much "trapped" in NG 8.2.14 😢

[alan-agius4]

@FSDRE, in your dynamic templates are you referencing other Angular Components, bindings le directives or its static HTML?

If it’s static HTML you don’t need to create dynamic JIT components, you can just inject the HTML from the API in the template.

[FSDRE]

@alan-agius4 You're missing a word there 😄 do you mean referencing other Angular Components?

This is one of our dynamic components:

export class TemplateControlContentComponent {

  public enabled: boolean = true;

  public values: Array<string> = new Array<string>();

}

An dumbed down template from the API (after some processing on the client) would look like this:

<div class="...">
    <!-- Some static HTML here... -->
    <span>Value 1: {{ values[0] }}</span>
    <span>Value 2: {{ values[1] }}</span>
    <span>Is Enabled: {{ enabled }}</span>
     <!-- Some other static HTML here... -->
</div>

CSS for the template is also retrieved from the API. We have a template builder in our C# IDE. The binding tags are actually inserted on the client. There is some databinding mechanism in place that mappes datasource names like {{ ds:CompanyName|format:uppercase }} to the values array. I realized that you cannot import other modules in dynamically compiled modules. So even a ngIf* in the HTML would fail since that needs CommonModule to be imported.

[alan-agius4]

@FSDRE yeah I did mean referencing 😁

[FSDRE]

@alan-agius4 I find more and more issues and posts in which this matter is extensively discussed. Most of the time the answer from the Angular team is something like "don't work against the concept of Angular" or "rethink your software". On the other hand we have many developers using dynamic component compilation because there are definitely resonable usecases where it is a very handy feature.

Now fact is, it worked up to NG 9.0.6 and now it's broken. The worrying part in my opinion is that the Angular team does not seem to want to fix it since like I mentioned "it's against the concept".

Now my question is, can we get somewhat of an official statement as of how this will be handled? It seems to me that the whole JIT thing is going to get scrapped in the future, or why else would it be such a delicate issue? NG 9.0.7 definitely broke many apps and I don't see much support as of how we could fix them. Will we get this feature back or do we have to head in a different direction completely?

[loedeman]

@FSDRE It would definitely be very nice when there could be dynamic support in production, besides the great AOT concept. IMHO, it should not be very hard to implement runtime dynamic module support. Usage of course would require opt-in to prevent users not wanting it to get it anyway...

PS. Solved my dynamic template issue using a plain JavaScript ‘new Function’ call. Ugly, but now both Angular and me are happy 😀...

[Nixon-Joseph]

PS. Solved my dynamic template issue using a plain JavaScript ‘new Function’ call. Ugly, but now both Angular and me are happy ...

@loedeman can you provide a quick example of your solution? (trimmed down of course).

[FSDRE]

@loedeman I am also interested in what you mean by

Solved my dynamic template issue using a plain JavaScript ‘new Function’ call

Can you elaborate?

[FSDRE]

I solved my problems by stepping out of the Angular bubble and implementing Web Components. I did not use Angular Elements since you have to deal with the same issues compared to the AOT compile. I implemented plain simple components in TypeScript and they run in every browser needed. The critical thing was to open up a new Shadow DOM in order to isolate the component styles from the app and the other way around. This way I can receive HTML and CSS styles from the web API and process them in my individual web components without the risk of the incoming styles breaking my app. No Angular compiler, no bloated js, just using browser capabilities.

[dgp1130]

Spoke with @IgorMinar and @alxhub about this. We don't see any supported use case that benefits from JIT + build optimizer. Build optimizer strips out decorators required by JIT mode, so there's no easy way to make this happen. Build optimizer is intended for production use and JIT has significant bundle size and performance impact due by its very nature, so you'd never have acceptable bundle sizes with this combination anyways (and if you want build optimizer, you clearly care about bundle size). Compiling components at runtime is also an XSS vulnerability in itself, so doing so is not a recommended or supported use case. As a result, we don't think Angular should support such a configuration.

[jcgillespie]

@dgp1130 This is a disappointing, if not unexpected, response.

If this isn't going to be a supported scenario, can we get a better, less misleading, error message so people don't waste so much time? The message suggests there is something the user can do to address the issue, but the team has decided this isn't the case.

[dgp1130]

@jcgillespie, agreed that we can do better error messaging here.

Filed #19016 to introduce a better, compile-time message about this.

[FSDRE]

You just officially stripped a feature out of Angular that set it apart from other Frameworks. I don't know if this was the right choice. The "everything must be known at compile time"-approach nowadays feels a little 2010, to be honest. I don't understand that the Angular team does not see any usecases when there are many of them across the issues regarding the topic. At least we got webcomponents...

[FSDRE]

For everyone interested in solving this issue with webcomponents check out my 2 comments in the issue here:

https://github.com/angular/angular/issues/15275#issuecomment-703678535

[alxhub]

@FSDRE can you elaborate on what feature you're referring to?

Angular Ivy was specifically designed to enable AOT + JIT to work together. You can build AOT-compiled applications that load the compiler and dynamically compile components on the fly, and can also benefit from build-time code size optimizations (that is, Terser).

The Angular build-optimizer is something else entirely. It was designed to eliminate things like decorator artifacts that other optimizers could not statically prove were unused. This only makes sense if your codebase is AOT-only - if you're including JIT code then you need those decorators to survive optimization, because the decorators are what trigger the JIT compiler and carry the information needed to compile each class.

There's no feature being removed here. buildOptimizer: true just doesn't make sense for mixed AOT and JIT application, because its _raison d'etre_ is to strip out the metadata that would otherwise be needed for JIT support, in the name of smaller AOT-only builds.

[alxhub]

To elaborate a bit more, it is indeed true that buildOptimizer being a binary setting is overly broad - it would theoretically be possible to apply the build-optimizer only to the AOT parts of a hybrid codebase. However, this is not something the CLI currently can do, and there are lots of challenges around knowing which imported libraries can be run through the optimizer and which might be used in a JIT context. It would be nontrivial to support something like this.

[FSDRE]

@alxhub Today I know deactivating build-optimizer is a solution but it is not the source of the problem and all the issues popping up. The baseline of the issue was that the behavior changed in a hotfix release (9.0.6 -> 9.0.7). From one day to the other nobody could run their production environment anymore. In 9.0.6 you could JIT compile with build-optimizer true. I posted CLI #18139 in July because of it after our production build failed.

The messages we got were:

1. "JIT compilation is discouraged for production use-cases! Consider AOT mode instead."
2. "Alternatively provide the compiler with 'import "@angular/compiler";' before bootstrapping"

The first message is crystal clear and made me worry a lot because I did not understand how one can release such a change in a hotfix. The second message simply did not work, because the build-optimizer was not even mentioned.

Then I realized I was not alone and saw this

And honestly it also didn't read like

Angular Ivy was specifically designed to enable AOT + JIT to work together

but more like a servere bug that needs to be fixed. There are a couple of issues were the matter was discussed and it never sounded like anything of what went wrong was actually planned. So now I am honestly surprised about your above sentence, because it is the first time since June anybody made a statement like this. Everything else I read was more like "shit's on fire, yo" 😄

So don't get me wrong, I don't want to blame anybody. You're the first one (at least for me) who says deactivating the build-optimizer is actually the correct fix and not a bad workaround. But my above statements regarding the hotfix and build messages still count.

I mean, the discussions and build messages made me kick out JIT compilation and replacing it with web components. I probably wouldn't have done that if I did not have the feeling something was very very wrong. Through the issues and comments from other Angular members I had the impression that Angular kinda doesn't care about JIT anymore. I am super happy that it's not the case but the whole communication was a bit weird 😃

[angular-automatic-lock-bot[bot]]

This issue has been automatically locked due to inactivity.
Please file a new issue if you are encountering a similar or related problem.

Read more about our automatic conversation locking policy.

_{_This action has been performed automatically by a bot._}