[email protected] slower webpack than @19.x

[Kikobeats]

Description

gatsby@20 take too much time under gatsby build command.

gatsby@19

• Build time: 5m 48s. Total deploy time: 9m 9s
• Build trace

gatsby@20

• Build time: +15m. (The build exceeded Netlify max bundle time)
• Build trace

This happening from the first 20.x build. I was waiting a bit before opening the issue just to be sure it isn't a temporal regression that would be patched on successive versions, but it's still happening

Steps to reproduce

git clone [email protected]:microlinkhq/www.git
yarn
npm run build

Expected result

Although my website project has a medium size, I expected that using latest gatsby version possible keeps the build time in similar numbers (around ~5m) or less.

At gatsby@20, most of the build time the process is spending the time at run queries stage:

 npm run build | gnomon
   0.2670s
   0.0002s   > [email protected] build /Users/josefranciscoverdugambin/Projects/microlink/www
   0.0000s   > NODE_ENV=production gatsby build
   3.3952s
   0.5436s   success open and validate gatsby-configs - 0.370s
   0.0214s   success load plugins - 0.540s
   0.0499s   success onPreInit - 0.019s
   0.0069s   success delete html and css files from previous builds - 0.048s
   0.1099s   success initialize cache - 0.005s
   0.0034s   success copy gatsby files - 0.105s
   0.0056s   success onPreBootstrap - 0.005s
   0.2175s   success createSchemaCustomization - 0.004s
   0.3768s   success source and transform nodes - 0.215s
   2.8366s   success building schema - 0.374s
   0.1529s   success createPages - 2.833s
   0.0022s   success createPagesStatefully - 0.151s
   0.0658s   success onPreExtractQueries - 0.001s
   0.3239s   success update schema - 0.064s
   0.0107s   success extract queries from components - 0.321s
   0.0034s   success write out requires - 0.007s
   0.0023s   success write out redirect data - 0.001s
   0.0021s   success onPostBootstrap - 0.000s
   0.0011s   ⠀
   0.0008s   info bootstrap finished - 8.090 s
 296.1899s   ⠀
   0.7775s   success Building production JavaScript and CSS bundles - 287.921s
  83.9502s   success run queries - 296.959s - 114/114 0.38/s
   0.1889s   success Building static HTML for pages - 79.861s - 213/213 2.67/s
   0.1747s   success onPostBuild - 0.184s
   0.8598s   info Done building in 389.33843624 sec
   0.0002s

     Total   390.5427s

Actual result

This same stage at gatsby@19 is consuming so much less time:

success run queries - 184.718s - 223/223 1.21/s

185s at gatsby@19 vs 297 at gatsby@20!

Probably something internal in the way gatsby perform has changed and I did to rework my queries; but still feeling it's a bit strange, all my queries are static and I'm using useStaticQuery on all of them (see an example).

Just want to figure out what I need to do to back my project on gatsby road 🙂

Environment

npx gatsby info --clipboard

  System:
    OS: macOS 10.15.4
    CPU: (4) x64 Intel(R) Core(TM) i5-7360U CPU @ 2.30GHz
    Shell: 5.7.1 - /bin/zsh
  Binaries:
    Node: 13.12.0 - /usr/local/bin/node
    Yarn: 1.22.4 - /usr/local/bin/yarn
    npm: 6.14.4 - /usr/local/bin/npm
  Languages:
    Python: 2.7.16 - /usr/bin/python
  Browsers:
    Chrome: 80.0.3987.163
    Firefox: 73.0.1
    Safari: 13.1
  npmPackages:
    gatsby: 2.20 => 2.20.23
    gatsby-plugin-canonical-urls: ~2.2.2 => 2.2.2
    gatsby-plugin-catch-links: ~2.2.2 => 2.2.2
    gatsby-plugin-google-analytics: ~2.2.3 => 2.2.3
    gatsby-plugin-netlify: ~2.2.2 => 2.2.2
    gatsby-plugin-netlify-cache: ^1.2.0 => 1.2.0
    gatsby-plugin-react-helmet: ~3.2.3 => 3.2.3
    gatsby-plugin-remove-trailing-slashes: ~2.2.2 => 2.2.2
    gatsby-plugin-sass: ~2.2.2 => 2.2.2
    gatsby-plugin-sitemap: ~2.3.3 => 2.3.3
    gatsby-plugin-styled-components: ~3.2.2 => 3.2.2
    gatsby-source-filesystem: ~2.2.3 => 2.2.3
    gatsby-transformer-javascript-frontmatter: ~2.2.2 => 2.2.2
    gatsby-transformer-json: ~2.3.2 => 2.3.2
    gatsby-transformer-remark: ~2.7.2 => 2.7.2
    gatsby-transformer-yaml: ~2.3.2 => 2.3.2

[wardpeet]

Thank you! Seems like build-javascript is the problem here if I look closely. I can have a look at why that's happening. We changed bundling in a 2.20.7

I'll checkout your github repo: git clone [email protected]:microlinkhq/www.git

[Kikobeats]

Just for debugging purposes, I run a Gatsby Cloud build that took 37min just in the queries stage 😂

[Kikobeats]

Locked dependencies to

{
  "gatsby": "2.19",
  "gatsby-cli": "2.10"
}

gatsby-cli is also necessary, otherwise is throw an error cause the API is different, working fine as a temporal solution

[wardpeet]

Like I thought it's webpack being way slower. I'll be investigating

[pvdz]

(To clarify; the reason is that we're looking into webpack is because that step ("Building production JavaScript and CSS bundles") is blocking but the reporter already starts counting for "run queries" so the time spent in "run queries" includes the webpack time entirely.)

12:02:07 AM: success Building production JavaScript and CSS bundles - 155.857s
12:02:09 AM: success run queries - 157.743s - 223/223 1.41/s

11:22:19 AM: success Building production JavaScript and CSS bundles - 765.522s
11:22:21 AM: success run queries - 768.023s - 223/223 0.29/s

So this is why "run queries" is not the source of the problem here.

[Kikobeats]

I confirm this is fixed; build time is back to be reasonable.

Total build time took 119.87s.

Thanks for that!

[rgraber]

This does not appear to be solved for my team. Our webpack builds are still about 4x slower than they used to be, and we are on gatsby 2.21.13.

[pvdz]

@rgraber if you see significant regression in the current version of Gatsby please open a new issue and let us know the repo to test it with as well as the last version known to you to be fast. Tag me in that issue. But please first check to confirm that it's still a problem on the current version. That includes other dependencies besides gatsby.