Gatsby: Cannot build locally after upgrade

[immbudden]

Description

Having not worked on a client project for a few months I jumped back in for site updates, upgrading a few dependencies as instructed as well as gatsby 2.7.19 to 2.15.28.

After trying to rebuild locally after the updates I have been unsuccessful for a few hours, with the site getting stuck on the source and transform nodes process within gatsby-source-prismic when trying to download images for the news page of the site. No matter what I try, the site will not build locally.

I noticed that gatsby-source-prismic migrated to a new version of the API (v3), however applying these updates didn't do anything, so I have reverted to the previous version.

After pushing my updates to github I have noticed that netlify persists with processing the images and builds the site, however it now takes 19 minutes to publish the site, rather than 3 minutes in previous successful build before any updates.

A few things to note...

• This is a related issue to https://github.com/angeloashmore/gatsby-source-prismic/issues/73 however the usual suggestions of "resizing the terminal" or "restarting" don't work.

• I had previously encountered and resolved the same issue within netlify (didn't occur locally) when pushing to build by using https://github.com/gatsbyjs/gatsby/tree/master/packages/gatsby-plugin-netlify

Thank you in advance for any help, this was my first gatsby site and I am still figuring a lot out!

Steps to reproduce

run gatsby-develop on latest deployment

You can see the project at : https://github.com/immbudden/Lanyon

Expected result

Site should generate all news images using childImageSharp and static site should be rendered

Actual result

Firstly, after all of the updates, I get the below error, which I _think_ is unrelated but am not sure...

ERROR

(node:40989) ExperimentalWarning: The WHATWG Encoding Standard implementation is an experimental API. It should not yet be used in production applications.

Then after gatsby begins to query files from prismic...

source and transform nodes hangs around the 95% mark.

Environment

Run gatsby info --clipboard in your project directory and paste the output here.

System:
OS: macOS 10.14.3
CPU: (8) x64 Intel(R) Core(TM) i7-6700HQ CPU @ 2.60GHz
Shell: 3.2.57 - /bin/bash
Binaries:
Node: 8.6.0 - ~/.nvm/versions/node/v8.6.0/bin/node
Yarn: 1.12.3 - /usr/local/bin/yarn
npm: 6.9.0 - ~/.nvm/versions/node/v8.6.0/bin/npm
Languages:
Python: 2.7.10 - /usr/bin/python
Browsers:
Chrome: 77.0.3865.90
Safari: 12.0.3
npmPackages:
gatsby: ^2.3.17 => 2.15.28
gatsby-image: ^2.0.0-rc.4 => 2.2.23
gatsby-paginate: ^1.0.17 => 1.1.1
gatsby-plugin-emotion: ^3.0.1 => 3.0.1
gatsby-plugin-favicon: ^3.1.5 => 3.1.6
gatsby-plugin-google-analytics: ^2.0.17 => 2.1.19
gatsby-plugin-netlify-cache: ^1.1.0 => 1.2.0
gatsby-plugin-react-helmet: ^3.0.12 => 3.1.10
gatsby-plugin-robots-txt: ^1.4.0 => 1.5.0
gatsby-plugin-sharp: ^2.0.33 => 2.2.27
gatsby-plugin-sitemap: ^2.1.0 => 2.2.16
gatsby-plugin-styled-components: ^3.0.4 => 3.1.8
gatsby-plugin-typography: ^2.2.2 => 2.3.10
gatsby-remark-responsive-iframe: ^2.0.8 => 2.2.20
gatsby-source-filesystem: ^2.0.25 => 2.1.28
gatsby-source-prismic: ^2.2.0 => 2.2.0
gatsby-transformer-remark: ^2.1.15 => 2.6.26
gatsby-transformer-sharp: ^2.1.9 => 2.2.19
npmGlobalPackages:
gatsby-cli: 2.7.53

[LekoArts]

Have you tried deleting your .cache and public folder and re-run gatsby build? Also same for clearing cache on Netlify.

[immbudden]

Unfortunately I have @LekoArts, I tried it a few times to no avail. I've ran gatsby clean a couple of times to make sure nothing was left behind but it didn't impact it either

[Js-Brecht]

@immbudden Care to give PR #17938 a whirl? It may not be related to your specific use case, but it fixes a nearly identical issue. I keep seeing this issue come up with MacOS users, and most of what you described fits the criteria. I don't see how it would effect Netlify, however, unless using a MacOS image (which it looks like is possible now).

[immbudden]

All sounds very familiar @Js-Brecht - going to give it a go now, I'll report back...

[immbudden]

@Js-Brecht I tried it but it didn't do anything for me, however I want to check whether I followed the steps correctly, so I have a question (I am relatively inexperienced with command line so apologies if they are stupid/obvious).

For step 2, this can be done anywhere within my tree, right?

git clone --single-branch -b macos-fsevents-fix https://github.com/Js-Brecht/gatsby
cd gatsby
yarn bootstrap
# Set the target repo path for gatsby-dev
gatsby-dev -p "${PWD}"

After I complete the steps and run gatsby develop I get 4 x Error: Cannot find module '@emotion/babel-preset-css-prop' errors. After this, the process still hangs at source and transform nodes

This all could be because I haven't followed the steps properly, so apologies in advance if that is the case!

Edit
I did just reinstall npm install gatsby-plugin-emotion to get rid of that error but I thought it was worth noting.

[Js-Brecht]

Yes, you can place that repo wherever you want to, as long as you have read/write access. I'm assuming you do, or you'd get errors and not be able to complete the step.

Line 1, git clone, is going to download the repository to a directory named gatsby, relative to your current location. You can name it whatever you want by putting the name after the url. This directory is key. You just need to go to the top of that directory and run yarn bootstrap. You should see a lot off output, packages being downloaded, builds being run.

Then, gatsby-dev needs to be pointed to that same directory (using an absolute path) where you downloaded the repo. That's where gatsby-dev -p "${PWD}" comes in... you can use a full path in place of "${PWD}" (and it might actually be recommended). I put that there for ease of use. Just replace "${PWD}" with whatever your full path is, starting from root. Running echo $PWD should show you what your current working directory is, or at least it should. That's if bash is getting the same environment variables in MacOS as it does in linux. If there are spaces, they should be escaped (with a backslash in front of them, like /path/to/gatsby\ with\ space), or the whole string should be surrounded by quotes.

Every so often, I run into a problem with gatsby-dev not getting all of the dependencies from the dev repo for some reason. Usually, I have to manually add them to the site after I run gatsby-dev. So, in the site where I'm running gatsby-dev, I would run yarn add @emotion/babel-preset-css-prop. Usually that causes the build to abort, so it's a little strange that it continued for you. Sometimes, I am able to fix it by running gatsby-dev -C --force-install, which would be gatsby-dev -C --force-install --scan-once in this case.

When you run gatsby-dev --scan-once in the desired site, you should also see a lot of output, as it loads up a Verdaccio server, uploads gatsby packages to it, and then installs them in the local repo

[Js-Brecht]

By the way, I downloaded your repository and hooked it all up, and was able to complete the build successfully. source and transform takes about 30 seconds to complete, Fetch Prismic data is about 15 seconds of that. Several errors on run static queries, something about Input file is missing and a rather long file name in the cache, but it continues anyway.

Site loads fine, too

So, this definitely looks like something that's going on with MacOS

[immbudden]

You're a gent, thanks for the detailed rundown of that!

30secs sounds about right for running it with all the image queries and resizes, some of them being fetched from prismic are quite big.

I'll give this a go again and report back if it fixes it.

[immbudden]

@Js-Brecht unfortunately after numerous retrying it doesn't work.

I thought it might have been something to do with my environment after all the updates that I had made before it all stopped working, however having cleaned NPM, node and Gatsby out, nothing seems to work.

[Js-Brecht]

That's too bad. Would have been nice if it were as simple as that.

Can you try making the following change in your site's node_modules, run gatsby develop again, and post the output up to where it hangs? Could help with debugging. I don't know how much time I'll be able to spend on this, but I'm sure it can be useful for anybody else that comes along to see where it is failing. Please verify that it always hangs in the same place, too.

node_modules/gatsby/dist/utils/api-runner-node.js

  345     let pluginName = plugin.name === `default-site-plugin` ? `gatsby-node.js` : plugin.name;
  346     return new Promise(resolve => {
+ 347       reporter.log(`${api}\t\t${plugin.name}`);
  348       resolve(runAPI(plugin, api, Object.assign({}, args, {
  349         parentSpan: apiSpan
  350       })));
  351     }).catch(err => {

Additionally, does it also hang when you run gatsby build?

[Js-Brecht]

One more question: are you able to build successfully without using gatsby-source-prismic?

[immbudden]

It seems to always hang around the 88% mark (120/136 nodes), I will post the log in the next comment.

It also hangs during gatsby build, I can post this log also if you think it is worthwhile - though when it is deployed to netlify, logs show it takes 17mins to generate all the thumbnails (3195).

In regards to building without gatsby-source-prismic - I haven't tried as all of the data within the news pages relies on that data, would this be worthwhile looking at?

[immbudden]

Gatsby-issues-17940.txt

[immbudden]

Probably also worth noting that I tried disabling the downloading of images in gatsby-configlike suggested in #448089987 however it will always fail to build as I am using localFile to process the images.

[Js-Brecht]

Just wanted to drop a cleaned up copy of the log.
Gatsby-issues-17940.txt

[Js-Brecht]

Nothing really remarkable there.

Can you try running it with tracing (performance tracing). Maybe that will provide some insight.

It might be helpful to try it without gatsby-source-prismic. It would help eliminate the network layer.
Would it be possible to download the content, and process using gatsby-source-filesystem? I haven't really examined your project to see what it's doing yet... I suppose it could take some extensive work. Just trying to eliminate some possible sources for the issue.

Maybe it would be better to start from the bottom. Start with a barebones project, and start adding disparate layers, until you can get it to fail?

[Js-Brecht]

I don't suppose you'd be able to get a full trace if it never finishes. 🤦‍♂

[immbudden]

I don't suppose you'd be able to get a full trace if it never finishes. 🤦‍♂

Ha - unfortunately not! I am going to look into processing a local file and see if that builds out ok, I suspect it will and the issue is with the image processing, it caused issues when I was building that part of the project initially and seemingly it continues too. I'll report back on that shortly.

I am also wondering whether it is to do with #6654 - #14819 suggests tweaking the concurrency so I am looking at that as a possible solution too.

[immbudden]

Aha - it worked, updating the concurrency in gatsby-config did the trick and now all the images can be downloaded.

To anyone who stumbles upon this, add this to gatsby-config.js

{
        resolve: `gatsby-source-filesystem`,
        options: {
          name: `images`,
          path: `${__dirname}/src/images`,
          GATSBY_CONCURRENT_DOWNLOAD: 20,
        },
},

and ensure the same setting is applied to create-remote-file-node.js

const queue = new Queue(pushToQueue, {
  id: `url`,
  merge: (old, _, cb) => cb(old),
  concurrent: process.env.GATSBY_CONCURRENT_DOWNLOAD || 20
}); // when the queue is empty we stop the progressbar

New error now, think it's related to the static queries you mentioned @Js-Brecht

⠋ run static queries
dyld: lazy symbol binding failed: Symbol not found: __ZN4node19GetCurrentEventLoopEPN2v87IsolateE
  Referenced from: /Users/immbudden/Development/Lanyon/app/node_modules/gatsby-plugin-sharp/node_modules/sharp/build/Release/sharp.node
  Expected in: flat namespace

dyld: Symbol not found: __ZN4node19GetCurrentEventLoopEPN2v87IsolateE
  Referenced from: /Users/immbudden/Development/Lanyon/app/node_modules/gatsby-plugin-sharp/node_modules/sharp/build/Release/sharp.node
  Expected in: flat namespace

Killed: 9

[Js-Brecht]

I'm not entirely sure if the two are related, but I suppose they could be; could just be the different operating systems are handling the same error in different ways.

Can you try using a newer version of Node? I see similar issues (not gatsby related) with some node interfaces out there.

[Js-Brecht]

If you do decide to install a new version of node, you will need to wipe out your node_modules, and reinstall all your packages

[immbudden]

Yea I think you are right, seems unrelated now and is something to do with my node setup. I was running node v12.10.0 when I got it to work there, got rid of my modules using rm -rf node_modules then ran npm install again.

Now it's reverted back to node 8.6.0 but I'm still getting those errors, will try a few more things...

[Js-Brecht]

Here's something interesting
https://spectrum.chat/gatsby-js/general/dyld-lazy-symbol-binding-failed-symbol-not-found~eef6ba21-b37e-4b36-88ec-0b321ed11475

How odd.

[immbudden]

Here's something interesting
https://spectrum.chat/gatsby-js/general/dyld-lazy-symbol-binding-failed-symbol-not-found~eef6ba21-b37e-4b36-88ec-0b321ed11475

How odd.

Hmmm - very odd. Trying to set mine to the same settings but it doesn't work :/

Edit

Restarted and reinstalled a few times, and we now have liftoff 🚀

Thanks for helping me work through it @Js-Brecht - here's a bottle of champagne to show my gratitude 🍾

[Js-Brecht]

Aha! I was just along for the ride on this one, I think 😁