Gatsby: Why does 'source and transform nodes' take so long.

[Steveeeie]

Our build seems to get through converting all of the end points OK but seems to hang on a final 'source and transform nodes' forever.

Can somebody please explain what is going on here and if there is anything that can be done to alleviate this?

The dependencies i'm using:

  "dependencies": {
    "gatsby": "^2.0.0-beta.22",
    "gatsby-link": "^2.0.0-beta.4",
    "gatsby-paginate": "^1.0.16",
    "gatsby-plugin-react-helmet": "next",
    "gatsby-react-router-scroll": "^2.0.0-beta.2",
    "gatsby-source-filesystem": "^2.0.1-beta.3",
    "gatsby-source-wordpress": "^3.0.0-beta.5",
    "lodash": "^4.17.10",
    "react": "^16.4.1",
    "react-dom": "^16.4.1",
    "react-helmet": "^5.2.0"
  },

[pieh]

With wordpress it usually means downloading media library - so in your case 316 images (this will only happen on first run or after you change plugins/edit gatsby-node.js, which will invalidate cache).

[Steveeeie]

Thanks for the response @pieh but this is happening every time I run npm run develop

[tryzniak]

I ran gatsby's official site that is under www in this repo with gatsby develop, and source and transform nodes never finished. Then I ran gatsby develop again and it completed in like 5 seconds. Strange.

node version: v8.9.4.
os: MacOS 10.12.

[m-allanson]

It'd be great if we can work out what's happening here.

Gatsby has just added support for tracing builds which might be useful? I haven't tried it yet, but if anyone wants to investigate you can check out the new docs page here: https://next.gatsbyjs.org/docs/performance-tracing/

[tryzniak]

I'm pretty sure it hangs on this line. And I also noticed that the bug is present when a .cache folder haven't been created.

[gatsbot[bot]]

Old issues will be closed after 30 days of inactivity. This issue has been quiet for 20 days and is being marked as stale. Reply here or add the label "not stale" to keep this issue open!

[gatsbot[bot]]

This issue is being closed due to inactivity. Is this a mistake? Please re-open this issue or create a new issue.

[r1q]

I know its closed but In my case when i tried all solutions and nothing seems to fix it .. the only thing that worked was:

Resizing the Terminal window !! when it gets stuck.

[ivanvgarcia]

I know its closed but In my case when i tried all solutions and nothing seems to fix it .. the only thing that worked was:

Resizing the Terminal window !! when it gets stuck.

This worked for me too.

[Madgeniusblink]

I know its closed but In my case when i tried all solutions and nothing seems to fix it .. the only thing that worked was:

Resizing the Terminal window !! when it gets stuck.

Wow thank you! I had no idea that actually helped it!! I spend the past month building a small web app on gatsby and I was getting so mad with this loading issue...! but thats a nice trick around it

[JaKXz]

Of my ~163s bootstrap time, _~153s_ (or ~93%) is source and transform nodes (in development). I don't think this issue should be closed or stale, and I am using the latest versions of almost all my gatsby deps:

npx gatsby info

  System:
    OS: macOS 10.14.6
    CPU: (4) x64 Intel(R) Core(TM) i5-5257U CPU @ 2.70GHz
    Shell: 5.7.1 - /usr/local/bin/zsh
  Binaries:
    Node: 12.10.0 - /usr/local/bin/node
    Yarn: 1.17.3 - /usr/local/bin/yarn
    npm: 6.11.3 - /usr/local/bin/npm
  Languages:
    Python: 3.7.4 - /usr/local/opt/python/libexec/bin/python
  Browsers:
    Chrome: 77.0.3865.75
    Safari: 12.1.2
  npmPackages:
    gatsby: ^2.15.15 => 2.15.15
    gatsby-image: ^2.2.19 => 2.2.19
    gatsby-plugin-create-client-paths: ^2.1.7 => 2.1.7
    gatsby-plugin-import: ^2.1.5 => 2.1.5
    gatsby-plugin-lodash: ^3.1.7 => 3.1.7
    gatsby-plugin-manifest: ^2.2.16 => 2.2.16
    gatsby-plugin-offline: ^3.0.6 => 3.0.6
    gatsby-plugin-react-helmet: ^3.1.7 => 3.1.7
    gatsby-plugin-sharp: ^2.2.22 => 2.2.22
    gatsby-plugin-styled-components: ^3.1.5 => 3.1.5
    gatsby-source-filesystem: ^2.1.22 => 2.1.22
    gatsby-transformer-sharp: ^2.2.14 => 2.2.14

[Js-Brecht]

Has anybody tried increasing node's Worker Pool size? Some expensive IO operations (Filesystem IO/certain network requests) are offloaded to separate worker threads. Node defaults to 4, and when those fill up, IO pretty much stops. Might be an issue in some circumstances :man_shrugging:.

Try using environment variable UV_THREADPOOL_SIZE. Max is 128.