Gatsby: Unable to notice any difference in subsequent build times using incremental build approach

[jay-coditas]

Hi team,
I'm facing issue using the new incremental build feature,
I've set the GATSBY_EXPERIMENTAL_PAGE_BUILD_ON_DATA_CHANGES=true in .env file
And then gatsby build
I don't see any time difference among the subsequent builds even while not updating any file, content, or be it a data source
Using it in POWERSHELL, $env:GATSBY_EXPERIMENTAL_PAGE_BUILD_ON_DATA_CHANGES="true"; gatsby build,
it still takes the same time as it does during the initial build
Incremental builds are supposed to reduce build times, right?
Am I using it in the wrong way?
Need help configuring the incremental build flag, the build time on every small update is very costly as of now
Current build time: 420 seconds

System

Windows 10 Pro
VScode terminal (PowerShell)
node -v: v12.16.2
npm -v: 6.14.4

package.json

{
  "name": "gatsby-starter-default",
  "private": true,
  "description": "A simple starter to get up and developing quickly with Gatsby",
  "version": "0.1.0",
  "author": "Kyle Mathews <[email protected]>",
  "dependencies": {
    "@mdx-js/mdx": "^1.5.8",
    "@mdx-js/react": "^1.5.8",
    "axios": "^0.19.2",
    "dotenv": "^8.2.0",
    "gatsby": "^2.19.45",
    "gatsby-image": "^2.2.44",
    "gatsby-plugin-catch-links": "^2.2.1",
    "gatsby-plugin-manifest": "^2.2.48",
    "gatsby-plugin-mdx": "^1.1.4",
    "gatsby-plugin-offline": "^3.0.41",
    "gatsby-plugin-react-helmet": "^3.1.24",
    "gatsby-plugin-sass": "^2.2.1",
    "gatsby-plugin-sharp": "^2.4.13",
    "gatsby-source-filesystem": "^2.2.2",
    "gatsby-source-git": "^1.0.2",
    "gatsby-source-mongodb": "^2.2.1",
    "gatsby-transformer-remark": "^2.7.1",
    "gatsby-transformer-sharp": "^2.3.19",
    "http-server": "^0.12.1",
    "monaco-editor": "^0.20.0",
    "monaco-editor-webpack-plugin": "^1.9.0",
    "node-sass": "^4.13.1",
    "prop-types": "^15.7.2",
    "react": "^16.12.0",
    "react-dom": "^16.12.0",
    "react-helmet": "^5.2.1",
    "react-monaco-editor": "^0.35.0"
  },
  "devDependencies": {
    "prettier": "^1.19.1"
  },
  "keywords": [
    "gatsby"
  ],
  "license": "MIT",
  "scripts": {
    "build": "npm run clean && gatsby build",
    "develop": "gatsby develop",
    "format": "prettier --write \"**/*.{js,jsx,json,md}\"",
    "start": "npm run clean && npm run develop",
    "serve": "npm run clean && gatsby build && gatsby serve",
    "clean": "gatsby clean",
    "test": "echo \"Write tests! -> https://gatsby.dev/unit-testing\" && exit 1"
  },
  "repository": {
    "type": "git",
    "url": "https://github.com/gatsbyjs/gatsby-starter-default"
  },
  "bugs": {
    "url": "https://github.com/gatsbyjs/gatsby/issues"
  }
}

[StuartRayson]

@jay-coditas 👋

If there are no changes from the data sources (mdx, CMS, etc..), the time saving comes from skipping the static pages that don't need to be rebuilt.
I'm not sure how large your project is in terms of its data sources, but if its relativity small you may not see the benefits compared to projects with a large data sources (10000+).

Using the argument --log-pages you can see if pages are being rebuilt. It is also important that you are't switching between an "incremental build" and a standard build, as the cache is cleared when this happens.

One of the biggest time constraints is the Webpack bundling of the JS assets. This does take a lot time during the build step, but might be improved in the next version of Webpack. 🤞

The JS bundling step is important tho as it returns a "webpack compilation hash" after the bundle. This hash is used to track if there are any code or static queries data changes and this will trigger a full build.

[jay-coditas]

@StuartRayson Thanks for pointing out about not so major time difference when it comes to small projects. As of now, I've less content to work with, which later on won't be the case.

And, yes, I was trying this new feature (incremental build) locally, which I just got to know that one needs to have a GatsbyCloud account and that's where the incremental build magic works, so yes trying out the GatsbyCloud platform.

When using the argument --log-pages, it did show the only files that were updated/added. I didn't clear cache, so as to go with the "incremental build"

Oh, I just got to know, the trial account which I signed up for in GatsbyCloud, it's of "Professional" type. And to witness the "incremental build" in action, I'll probably need an "Enterprise" account which out of question for now

Thanks

[ascorbic]

I think the confusion here is that GATSBY_EXPERIMENTAL_PAGE_BUILD_ON_DATA_CHANGES=true is a different thing from the incremental builds on Gatsby Cloud, so you shouldn't expect the same results. As @StuartRayson points out, page building is only one part of the build that can take a long time, so unfortunately even if that is quicker you won't be able to get sub 10-second builds like on Gatsby Cloud. (FWIW, you don't need an Enterpise account to get them.)

[StuartRayson]

@ascorbic 👋

Might be worth updating the documentation for the usage of GATSBY_EXPERIMENTAL_PAGE_BUILD_ON_DATA_CHANGES to make it clear that this is not the same solution to Gatsby Cloud incremental build.

As i've seen another issue where users are expecting the same build times as Gatsby Cloud using just this environment variable.
https://github.com/gatsbyjs/gatsby/issues/23531

i'm happy to do this?

[ascorbic]

That's a good idea, yes.