Webpacker: How to profile this

[manveru]

Is there any way to use --progress --profile with this gem? Our production build takes about 10 minutes and I'd like to know what it spends its time on.

[renchap]

Have you tried RAILS_ENV=production ./bin/webpacker --progress --profile?

[manveru]

Does that differ from what rake assets:precompile does?

[manveru]

So after trying it, seems to work. But I'm not sure why our build times increased so much:

52367ms building modules                                                                  
281ms sealing
31ms optimizing
0ms basic module optimization 
18ms module optimization
20ms advanced module optimization
0ms basic chunk optimization 
0ms chunk optimization 
2ms advanced chunk optimization 
12639ms building modules                                                                 
76ms module and chunk tree optimization
0ms chunk modules optimization 
0ms advanced chunk modules optimization 
32ms module reviving
1ms module order optimization 
26ms module id optimization
49ms chunk reviving
4ms chunk order optimization 
53ms chunk id optimization
268ms hashing
1ms module assets processing 
1143ms chunk assets processing
8ms additional chunk assets processing 
19ms recording
170867ms additional asset processing
81324ms chunk asset optimization
2582ms asset optimization
787ms emitting
Hash: e4608e001b671df6e407
Version: webpack 3.8.1
Time: 322663ms

Major offenders seem to be the SCSS files, but unless they are compiled multiple times, it shouldn't cause so much increase.

[4] ./node_modules/css-loader?{"minimize":true}!./node_modules/postcss-loader/lib?{"sourceMap":true,"config":{"path":"/mnt/big/github/referral-manager/eqipia/.postcssrc.yml"}}!./node_modules/resolve-url-loader!./node_modules/sass-loader/lib/loader.js?{"sourceMap":true}!./frontend/app/admin/css/application.scss 389 kB {0} [built]
            factory:139ms building:12433ms = 12572ms

I don't actually need expensive source maps for production, and that setting got lost in our porting, so I'll bench again after fixing them and report back.

[renchap]

I am also seeing high compile times (but not as high as yours), even in dev with HMR, and I also suspect the SCSS pipeline. I havent got time to investigate yet but I am interested in your findings.
It seems to have appeared some time ago but I dont know if its caused by a Webpacker config change or a dependancy update.

[gauravtiwari]

I am suspecting it to be sourcemaps. @renchap Are you using master or 3.0.2?

[renchap]

I am using 3.0.2, and I forced source maps to config.devtool = 'cheap-module-source-map' because it is needed for React: https://reactjs.org/docs/cross-origin-errors.html#source-maps

Also this only occurs for my main application bundle, the other bundles are fine. Here is what the profile looks like when changing a SCSS file (adding a new line) and using webpack-dev-server with HMR:

webpack: Compiling...
7360ms building modules
14ms sealing
0ms optimizing
0ms basic module optimization
4ms module optimization
0ms advanced module optimization
7ms basic chunk optimization
0ms chunk optimization
1ms advanced chunk optimization
0ms module and chunk tree optimization
0ms chunk modules optimization
1ms advanced chunk modules optimization
13ms module reviving
1ms module order optimization
4ms module id optimization
2ms chunk reviving
0ms chunk order optimization
3ms chunk id optimization
25ms hashing
1ms module assets processing
79ms chunk assets processing
39ms additional chunk assets processing
0ms recording
0ms additional asset processing
378ms chunk asset optimization
100ms asset optimization
221ms emitting
Hash: b7548790e226d1a817ec
Version: webpack 3.8.1
Time: 8261ms

[renchap]

I reorganized my stylesheets and I have found that url(...) calls in my SCSS seems to be the culprit here.
When commenting a bunch of statements containing mostly lines like background-image: url('~images/home/uc-pro-events');, I go from 4.6s to 1.7s (I have other url() calls elsewhere, which can explain why its still so slow.

I am using the base Webpacker file-loader config.
I suspected the [hash] in the output name, but setting environment.loaders.get('file').use[0].options.name = '[name].[ext]' does not help.

Adding .jpg to the url statements helped a bit, the build time went to 4s.

Do you have any ideas on what is going on?

Also as a side note, [path] was added a while ago and file assets now have paths like /packs/_/_/_/_/shared/node_modules/font-awesome/fonts/fontawesome-webfont-af7ae505a9eed503f8b8e6982036873e.woff2.
I dont find it very pretty to have all those _ directories, as well as node_modules mentionned. I havent found a solution, but what was the problem with [name]-[hash].[ext]? If a file has the same content and name, there is no problem if the emitted file is the same.

[gauravtiwari]

@renchap Interesting. Could you please try removing resolve url loader and see what happens?

[gauravtiwari]

It's one of the loaders in style loader.

[renchap]

environment.loaders.get('style').use.splice(3, 1)

Much better, 0.8s for an in-memory rebuild 🚀

The asset filenames are still like _/_/node_modules/font-awesome/fonts/fontawesome-webfont-af7ae505a9eed503f8b8e6982036873e.woff2 (in dev, it looks like /packs/_/_/_/_/shared/node_modules/ in production)

Any idea about what is causing this? I dont know resolve-url-loader, I misread it when looking at the config and thought it was url-loader. From a quick glance it should change the paths to not contain this _ mess, am I right?

[gauravtiwari]

It's actually file loader, so if you remove path from the name or remove context: https://github.com/rails/webpacker/blob/master/package/rules/file.js#L10

since context is source directory anything outside of it is resolved as relative path.

[gauravtiwari]

So, it seems resolve-url-loader is causing all that problem. I will take a look again and see if we can live without it.

[renchap]

If you want me to test some things, feel free to ask!

[gauravtiwari]

@renchap Could you please try out master?