Gatsby: "source and transform nodes" hangs system with gatsby-source-wordpress

[ehibun]

Description

When using the WordPress source in Gatsby on a large WordPress site (6200 posts, 3196 images), the build process hangs my machine (a Late 2016 MacBook Pro running macOS 10.13.3) entirely, even if left overnight. This is probably related to #4293, except in my case the process never actually finishes.

With Activity Monitor running at the same time, it doesn't look like there's much CPU or network activity, but that might just be Activity Monitor silently crashing.

Steps to reproduce

• Using the Gatsby blog starter, install the gatsby-source-wordpress plugin and configure it to point at a large WordPress installation, with or without hostingWPCOM enabled.
• Replace gatsby-node.js with https://github.com/gatsbyjs/gatsby/blob/master/examples/using-wordpress/gatsby-node.js, and copy the page.js and post.js templates from that same repo.
• Attempt to run gatsby build or gatsby develop
• Observe that the system becomes unresponsive, and later hangs.

Expected result

Gatsby should import all WordPress posts, pages, and images successfully.

Actual result

The system hangs and does not complete the build process.

Environment

• Gatsby version: [email protected]
• gatsby-cli version: 1.1.47
• Node.js version: v9.8.0
• Operating System: macOS High Sierra 10.13.3

File contents (if changed):

gatsby-config.js (relevant part):

    {
      resolve: 'gatsby-source-wordpress',
      options: {
        baseUrl: 'large-self-hosted-wordpress-site.com',
        protocol: 'https',
        hostingWPCOM: false,
        useACF: false,
        verboseOutput: false,
      },
    },

package.json: Identical to gatsby-starter-blog, but with "gatsby-source-wordpress": "^2.0.69", added.
gatsby-node.js: Identical to https://github.com/gatsbyjs/gatsby/blob/master/examples/using-wordpress/gatsby-node.js
gatsby-browser.js: Unchanged
gatsby-ssr.js: Unchanged

[pieh]

When it get stuck on that step after checking all wordpress endpoints it usually waits to fetch all images but currently it tries to download them all at once (so doing over 3000 request more or less at once for your images), which is not good (but soon-ish we should fix that - https://github.com/gatsbyjs/gatsby/pull/4616).

[lightstrike]

I'm having related issues with a large WordPress site.

What causes the process to hang? Where can we trigger an explicit failure?

[kellymears]

i'm also getting this error.

additional info: my site is running in a wordpress network configuration, and it hangs whether i query a fresh site or a large one. don't know if multisite is compatible..

[pieh]

I'll close this as duplicate of https://github.com/gatsbyjs/gatsby/issues/6654 - that issue has reproduction repository

[r1q]

I know its closed but In my case when i tried all solutions and nothing seems to fix it .. the only thing that worked was:

Resizing the Terminal window !! when it gets stuck.

[lucashfreitas]

Same here @r1q. I just resized my termina window and it worked.

[michelegera]

OMG, it actually worked 😳

[cyrusdavid]

Resizing the terminal window worked for me as well 🤯

[stldo]

Resizing the terminal window worked here too.

[kangarang]

oh my goodness, i can't believe this actually worked