Angular-builders: CustomWebpack with Devkit/CLI 8.2 causes build options to get overridden/ignored

[justinappler]

Update:
Related issue in Angular CLI repo: https://github.com/angular/angular-cli/issues/16098

Describe the bug
This is a hard bug to describe, but when using custom-webpack for both dev-server and browser, certain options in the browser build get ignored. I spent a lot of time trying to debug what's going on (the devkit was very hard to debug), and it seems that when getTargetOptions is called within custom-webpack/dev-server, it includes the options, but when getTargetOptions is called again inside of executeDevServerBuilder, it is missing some of the options specified.

This manifested for me with 'showCircularDependencies' (which is set to false for my browser build) getting ignored in the dev server build but respected in the browser build).]

This seems to repro pretty easily by simply calling getTargetOptions twice inside of dev server, e.g.

    async function setup() {
        const browserTarget = targetFromTargetString(options.browserTarget);

        const targetOptions1 = context.getTargetOptions(browserTarget).then(opts => // correct);
        const targetOptions2 = context.getTargetOptions(browserTarget).then(opts => // missing some parameters)

        return context.getTargetOptions(browserTarget) as unknown as CustomWebpackSchema;
    }

I hope this is enough to investigate, I can provide more details as well

To Reproduce
Steps to reproduce the behavior:

1. Create a project that uses dev-server
2. Add a non-default property to the browser build, e.g. showCircularDependencies: false
3. Target dev-server at build:browser
4. Notice that the property showCircularDeps is ignored

Expected behavior
Properties specified in the browser build are ignored

Libraries

@angular-builders/custom-webpack: "8.1.0"
@angular-devkit/architect: "0.802.0"
@angular-devkit/build-angular: "0.802.0"
@angular-devkit/build-optimizer: "0.802.0"
@angular/cli: "8.2.0"
@angular/compiler-cli: "8.2.0"
@angular/language-service: "8.2.0"

[anhkind]

I got exactly the same issue with showCircularDependencies: false being ignored completely in dev-server target. Any ideas how to fix this yet?

[just-jeb]

Please update me if this is still the case with [email protected].

[chriswep]

Please update me if this is still the case with [email protected].

it's still the case for me

[flash-me]

The problem here is that @angular-builders will apply its config onto the base webpack config. This was okay, because the CLI webpack-config has been applied before the custom one.
But this Commit changed the behaviour.

On a quick look into the code I'd assume that using mergeConfigs in both cases might solve the issue, but I can image that there was a reason for the if when providing a function. I donno :confused:

[justinappler]

@just-jeb Tried again updating to latest CLI/Devkit, the problem seems to persist.

[just-jeb]

@flash-me the code (both Angular's and custom-webpack') seems completely fine to me. The custom config is still applied after the base one - note that the base one is passed to a transform, which, in turn, merges the config while the custom overrides the base.
Regarding the if, yes, you are right, this is for reason. If custom config exports a function, the base config is passed into this function and then the function is supposed to modify it and return the new config. It's described in the docs.
I'll try to figure out why you're getting this weird behavior, but from the first glance the code seems OK.
One thing I need you to try though. If you run the dev-server build with aot the problem still persists?

[flash-me]

@just-jeb Uhhhm puhh. Donno what exactly the issue was here. In the meanwhile I changed my approach / have my own builders.
I think the problem was a mix of "where do I have to write what" like in angular.json you set flags for the CLI
and in webpack.custom.config.js you are modifying the final WebpackOptions that will be used to execute webpack in the end with the help of ExecutionTransformer.

But all in all, just checked again. Seems to work for me. :+1:
I'm dealing with other issues atm ^^

[just-jeb]

I see. Seems that I also misunderstood the issue.
@justinappler , @chriswep Just to make sure I understand it right now:

1. You use some non-default option in browser build in angular.json (for example showCircularDependencies: false).
2. When you use vanilla builders (@angular-devkit/build-angular) it works as expected both for build and serve.
3. When you use @angular-builders/custom-webpack, without even using custom webpack config (just replacing the builder), the serve ignores this override.
   Is that so?
   Would you mind creating a minimal reproduction repo for me?

[justinappler]

@just-jeb OK, simple repro repo: https://github.com/justinappler/custom-builder-repro-case

If you switch out the custom builders for the devkit versions, the showCircularDependencies flag will be respected (with ng serve).

[mancmanomyst]

Any update on this one? It's preventing me from upgrading to Angular 8

[just-jeb]

@justinappler thank you for investing time in investigation, it really helped me. I confirm the issue - two subsequent calls to context.getTargetOptions return different results.
However, the problem is within Angular Devkit (I did the subsequent call in their code and it still reproduces). Trying to figure out with Angular team what's the root cause.
@mancmanomyst Since the problem is within Angular I don't see it resolved before they release the next version of Devkit. So you'll probably have to wait, sorry.
You can try different 0.80x.x versions of the devkit. If the problem was introduced in one of the minor releases you would be able to upgrade to Angular 8 (just not the latest one).
I'll update once I have more info.

[justinappler]

@just-jeb Is there an issue filed in the cli repo or something else we should track?

[just-jeb]

I asked on Slack channel but didn't get any answer yet.
Wanted to open an issue once I have more info on this.
Narrowed it down to here meanwhile: https://github.com/angular/angular-cli/blob/master/packages/angular_devkit/architect/node/node-modules-architect-host.ts#L102
The options returned from this function are different in subsequent calls. It's probably some problem with Workspace, didn't get to it yet.
You are welcome to open an issue though in Angular CLI repo. Otherwise I'll file it once I have time to get to the root cause.

[just-jeb]

Ok guys, I might have a reasonable workaround for you if your issue is just the showCircularDependencies flag. They use CircularDependencyPlugin in Angualr CLI, so until the issue is fixed you can re-define CircularDependencyPlugin with your own excludes:

plugins: [
      new CircularDependencyPlugin({
        exclude: /your regex here/,
      }),
]

[chriswep]

@just-jeb i tried this but strangely it doesn't remove the warnings coming from my own codebase (i even tried .* regex). if i run it without excludes it shows warnings from node_modules also (which it didn't without the custom plugin) - however excluding those works.

[just-jeb]

@chriswep Interesting... Would you mind trying it with overrideDuplicatePlugins option?

[chriswep]

googling the term "overrideDuplicatePlugins" returns zero results (which is surprising for such a non-random word :-) - is this the correct option / how/where do i use it?

[just-jeb]

It's an option of custom-webpack builder, the docs are here.

[chriswep]

unfortunately this has no effect on the problem:

"customWebpackConfig": {
   "path": "./webpack.config.js",
   "overrideDuplicatePlugins": true
}

i also tried replaceDuplicatePlugins (which is mentioned in the docs and probably what you meant)

[just-jeb]

Yeah, sorry, that's what I meant.
Any chance that your regex is wrong? Would you mind sharing it here?
The thing that would work for sure (if you want to exclude all the files in the project) is completely removing the CircularDependencyPlugin from the plugins array. For that you can use custom webpack config function. Just find the plugin by constructor name and remove it from the array.
But I'd first try playing with the regex.

[chriswep]

Any chance that your regex is wrong?

Of course :-)

that's the warning

[ng] WARNING in Circular dependency detected:
[ng] ../shared/src/courses/entities/Content.entity.ts -> ../shared/src/courses/entities/ContentData.entity.ts -> ../shared/src/courses/entities/Content.entity.ts

that's my regex: exclude: /node_modules|entities/
As i said, i also tried .* to be sure

Can't see why that shouldn't work but that doesn't mean it isn't there.

[mancmanomyst]

@chriswep try /[\s\S]*/

It's worked for me.

Thanks @justinappler for the workaround.

Just in case any newbs to webpack like me come across this,

you need to run npm install --save circular-dependency-plugin if you haven't already got it installed and also require it i.e. const CircularDependencyPlugin = require('circular-dependency-plugin');

Then this will work

const CircularDependencyPlugin = require('circular-dependency-plugin');
module.exports = {
    plugins: [
    new CircularDependencyPlugin({
        exclude: /[\s\S]*/,
    })]
}

[chriswep]

The thing that would work for sure (if you want to exclude all the files in the project) is completely removing the CircularDependencyPlugin from the plugins array. For that you can use custom webpack config function.

@just-jeb thing is i was using the a custom webpack config all along (thus disabling the merging and/or plugin-replacing of CustomWebpack as i learned now). So removing the CircularDependencyPlugin finally did the trick:

    config.plugins = config.plugins.filter(plugin => !(plugin instanceof CircularDependencyPlugin));
    config.plugins.push(
        new CircularDependencyPlugin({
            exclude: /node_modules|shared/
        })
    );

[justinappler]

@just-jeb It looks like the Circular Dependency problem isn't the only one. I've also now noticed that, likely a similar reason, differential loading in production builds isn't working properly when I use the custom builder. Angular doesn't produce the proper es2015 builds and thus the differential loading code doesn't get run (I then get runtime errors when using es2015 features).

[just-jeb]

That's weird. Mind sharing your config?
Does it reproduce with the same sample repo you mentioned before?

[justinappler]

@just-jeb Angular.json is here: https://gist.github.com/justinappler/b5e7decef5f1788842921a5640461d8b I think this is a side effect of the internal webpack config created by Angular not getting respected. I noticed that chunkFilenames are ignored as well and had to add those back. Specifically what I'm seeing is that when I used the stock builders, I get a main-es2015 and main-es5 bundle. With custom builders, I only get a main bundle and that bundle doesn't have certain polyfills (concerning)

[just-jeb]

Does this problem appear only with dev-server builder or also with browser?
What your custom config looks like?

[justinappler]

I'm seeing this in the production configuration of the browser builder. Extra webpack file looks like:

module.exports = (config) => {

  config.node = {
    path: true,
    process: 'mock',
    global: true
  };

  config.plugins = config.plugins.filter(plugin => plugin && !plugin.constructor.name.includes("CircularDependency"));

  return config;
};

[just-jeb]

What about development configuration of browser?

[justinappler]

Same issue, generates a single main.hash.js bundle instead of the main-es2015 and main-es5.

[JulianH]

@justinappler, @just-jeb, i've raised an issue in the angular-cli repo: https://github.com/angular/angular-cli/issues/16098 which should address the problem.
If you have more informations, please add them to the ticket.

[just-jeb]

I forgot to update this issue, but two weeks ago I had a conversation with one of the guys for Angular CLI team. Here what he said:

I have a PR up that should be in the next release that performs a deep clone of the options before passing them to a builder. I unfortunately haven’t had the time to setup a test scenario but this should hopefully address your issue. The behavior also has benefits outside of this scenario as well. Builders should not be able to change the underlying configuration objects and potentially break other builders that could be executed.

That means that hopefully with Angular 9 it will be fixed.

[fasidOnGit]

Alright! that https://github.com/just-jeb/angular-builders/issues/422#issuecomment-532797449 worked!

for

"@angular-builders/custom-webpack": "^9.1.0",
"@angular/cli": "9.1.0",

Thank you @mancmanomyst

[just-jeb]

There are rumors that the issue has been fixed in the latest CLI versions. Could you guys verify that?

[insin]

I just upgraded an app from Angular 8.x to Angular 11.x and the circular dependency plugin workaround above wasn't needed any more :+1: