Gatsby: Intermittent/inconsistent build/develop errors

Created on 16 Jan 2019  ·  20Comments  ·  Source: gatsbyjs/gatsby

Firstly really loving Gatsby, it's awesome!

Description

There a 2 different error messages I have been getting on my local development/production builds which appear to be intermittent and inconsistent. Firstly, if there is a preference to open an issue for each separate one I will be happy to do so. Just let me know:)

Steps to reproduce

Unfortunately I have no clue how to reproduce these issues for Error 1 and Error 2 , they seem to occur between code changes and re running gatsby develop or gatsby build

Here is the gatsby project for reference if need be.

Errors

On gatsby develop or gatsby build these errors may occur from time to time.

Error 1

  TypeError: Cannot read property 'children' of undefined

  - actions.js:50 children.concat.children.map.child
    [app-web]/[gatsby]/dist/redux/actions.js:50:39

  - Array.map

  - actions.js:49 findChildrenRecursively
    [app-web]/[gatsby]/dist/redux/actions.js:49:42

  - actions.js:301 actions.deleteNode.args
    [app-web]/[gatsby]/dist/redux/actions.js:301:29

  - redux.js:468
    [app-web]/[gatsby]/[redux]/lib/redux.js:468:35

  - source-nodes.js:77 staleNodes.forEach.node
    [app-web]/[gatsby]/dist/utils/source-nodes.js:77:34

  - Array.forEach

  - source-nodes.js:77
    [app-web]/[gatsby]/dist/utils/source-nodes.js:77:18

  - Generator.next

  - util.js:16 tryCatcher
    [app-web]/[bluebird]/js/release/util.js:16:23

  - promise.js:512 Promise._settlePromiseFromHandler
    [app-web]/[bluebird]/js/release/promise.js:512:31

  - promise.js:569 Promise._settlePromise
    [app-web]/[bluebird]/js/release/promise.js:569:18

  - promise.js:614 Promise._settlePromise0
    [app-web]/[bluebird]/js/release/promise.js:614:10

  - promise.js:694 Promise._settlePromises
    [app-web]/[bluebird]/js/release/promise.js:694:18

  - async.js:138 _drainQueueStep
    [app-web]/[bluebird]/js/release/async.js:138:12

  - async.js:131 _drainQueue
    [app-web]/[bluebird]/js/release/async.js:131:9

  - async.js:147 Async._drainQueues
    [app-web]/[bluebird]/js/release/async.js:147:5

  - async.js:17 Immediate.Async.drainQueues
    [app-web]/[bluebird]/js/release/async.js:17:14

Work Around

Deleting the .cache folder and rebuilding/ rerunning dev appears to fix this issue.

Error 2

error UNHANDLED REJECTION


  TypeError: Cannot read property 'internal' of undefined

  - build-node-types.js:67 _.flatMap.groupBy.node
    [app-web]/[gatsby]/dist/schema/build-node-types.js:67:84

  - lodash.js:497 arrayAggregator
    [app-web]/[lodash]/lodash.js:497:34

  - lodash.js:4829 Function.groupBy
    [app-web]/[lodash]/lodash.js:4829:16

  - lodash.js:4374
    [app-web]/[lodash]/lodash.js:4374:28

  - lodash.js:683 arrayReduce
    [app-web]/[lodash]/lodash.js:683:21

  - lodash.js:4373 baseWrapperValue
    [app-web]/[lodash]/lodash.js:4373:14

  - lodash.js:9052 LodashWrapper.wrapperValue
    [app-web]/[lodash]/lodash.js:9052:14

  - build-node-types.js:67 groupChildNodesByType
    [app-web]/[gatsby]/dist/schema/build-node-types.js:67:140

  - build-node-types.js:123 inferChildFields
    [app-web]/[gatsby]/dist/schema/build-node-types.js:123:28

  - build-node-types.js:137 inferFields
    [app-web]/[gatsby]/dist/schema/build-node-types.js:137:23

  - build-node-types.js:187 fields
    [app-web]/[gatsby]/dist/schema/build-node-types.js:187:19

  - Array.forEach

  - Array.forEach

Work Around

Again deleting the .cache folder and running the development server or rebuilding fixes this issue.

Expected result

To be able to run the development server consistently and or run a build

Actual result

Not those things
What happened.
Things broke :(

Environment

We run our app on openshift in production. For local dev, we run on docker which is pretty close to the openshift image.

local development

System:
    OS: Linux 4.9 Debian GNU/Linux 8 (jessie) 8 (jessie)
    CPU: (6) x64 Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
    Shell: 4.3.30 - /bin/bash
  Binaries:
    Node: 8.12.0 - /usr/local/bin/node
    Yarn: 1.9.4 - /usr/local/bin/yarn
    npm: 6.4.1 - /usr/local/bin/npm
  Languages:
    Python: 2.7.9 - /usr/bin/python
  npmPackages:
    gatsby: ^2.0.64 => 2.0.64
    gatsby-plugin-emotion: ^3.0.1 => 3.0.1
    gatsby-plugin-favicon: ^3.1.4 => 3.1.4
    gatsby-plugin-google-analytics: ^2.0.8 => 2.0.8
    gatsby-plugin-lunr: ^1.3.0 => 1.3.0
    gatsby-plugin-offline: ^2.0.18 => 2.0.19
    gatsby-plugin-postcss: ^2.0.2 => 2.0.2
    gatsby-plugin-react-helmet: ^3.0.4 => 3.0.4
    gatsby-plugin-sharp: ^2.0.14 => 2.0.15
    gatsby-plugin-styled-components: ^3.0.4 => 3.0.4
    gatsby-remark-autolink-headers: ^2.0.12 => 2.0.12
    gatsby-remark-images: ^3.0.1 => 3.0.1
    gatsby-remark-prismjs: ^3.1.4 => 3.1.4
    gatsby-source-filesystem: ^2.0.11 => 2.0.11
    gatsby-source-github-api: 0.0.3 => 0.0.3
    gatsby-transformer-json: ^2.1.6 => 2.1.6
    gatsby-transformer-remark: ^2.1.15 => 2.1.15
    gatsby-transformer-sharp: ^2.1.9 => 2.1.9
    gatsby-transformer-yaml: ^2.1.6 => 2.1.6
  npmGlobalPackages:
    gatsby-cli: 2.4.8

Even running on my local machine i still get the same issue

System:
    OS: macOS 10.14.2
    CPU: x64 Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
    Shell: 3.2.57 - /bin/bash
  Binaries:
    Node: 8.12.0 - /usr/local/bin/node
    npm: 6.5.0 - /usr/local/bin/npm
  Browsers:
    Chrome: 71.0.3578.98
    Safari: 12.0.2
  npmPackages:
    gatsby: ^2.0.64 => 2.0.91
    gatsby-plugin-emotion: ^3.0.1 => 3.0.1
    gatsby-plugin-favicon: ^3.1.4 => 3.1.5
    gatsby-plugin-google-analytics: ^2.0.8 => 2.0.9
    gatsby-plugin-lunr: ^1.3.0 => 1.3.0
    gatsby-plugin-offline: ^2.0.18 => 2.0.21
    gatsby-plugin-postcss: ^2.0.2 => 2.0.2
    gatsby-plugin-react-helmet: ^3.0.4 => 3.0.5
    gatsby-plugin-sharp: ^2.0.14 => 2.0.17
    gatsby-plugin-styled-components: ^3.0.4 => 3.0.4
    gatsby-remark-autolink-headers: ^2.0.12 => 2.0.12
    gatsby-remark-images: ^3.0.1 => 3.0.1
    gatsby-remark-prismjs: ^3.1.4 => 3.2.0
    gatsby-source-filesystem: ^2.0.11 => 2.0.16
    gatsby-source-github-api: 0.0.3 => 0.0.3
    gatsby-transformer-json: ^2.1.6 => 2.1.7
    gatsby-transformer-remark: ^2.1.15 => 2.2.0
    gatsby-transformer-sharp: ^2.1.9 => 2.1.10
    gatsby-transformer-yaml: ^2.1.6 => 2.1.7
  npmGlobalPackages:
    gatsby-cli: 2.4.1
    gatsby-remark-code-titles: 1.0.2

I'd think that this would have to be a caching issue somewhere. I know for a fact in production, in our openshift environment we create a new image for each PR (as well as any new code changes pushed to the PR) and so the call to gatsby build in production never fails. I don't claim to be an expert, it could very well be something in our codebase but I don't think I am modifying any of the Gatsby build/develop objects in anyway.

And again to clarify this issue has been only happening in development when running gatsby build or gatsby develop

picture patricksimonian

Most helpful comment

Nothing so far, i still get it inconsistently. One thought is that it is perhaps tied to switching branches in git. The .cache folder is ignored and so perhaps mixing those up is causing this issue. As i mentioned before, however, my work around of deleting the cache has been __working great__ so far.

picture patricksimonian on 15 Feb 2019
👍3

All 20 comments

Are you using remote assets at all? I've been using docker for development with Gatsby and get similar errors, these tend to be due to my poor/unstable connection in China+VPN where downloading my remotely sourced images can fail(either no image at all, or partially downloaded image(which avoids the error).

In my code, , I'm using the graphql data result in my props, if the image failed to download, it's childImageSharp data is empty/null, thus errors occur, I'm able to avoid it by filtering these out prior, though obviously not ideal.

Could it be that you're experiencing an issue similar to me with local development due to connection failing to fetch some remote assets? (I deploy to production with netlify which works fine) If so, then perhaps Gatsby needs to throw some sort of error message in CLI output about these processing/query failures upfront(and perhaps try again). For my data on a project it can take about 10 minutes or longer to reprocess again just because of a failure like this(prior to my filter hack to avoid the issue in development).

polarathene picture polarathene on 17 Jan 2019

I'm getting error 2 as well! Just started randomly happening...

skrapkam picture skrapkam on 17 Jan 2019

Are you using remote assets at all? I've been using docker for development with Gatsby and get similar errors, these tend to be due to my poor/unstable connection in China+VPN where downloading my remotely sourced images can fail(either no image at all, or partially downloaded image(which avoids the error).

In my code, , I'm using the graphql data result in my props, if the image failed to download, it's childImageSharp data is empty/null, thus errors occur, I'm able to avoid it by filtering these out prior, though obviously not ideal.

Could it be that you're experiencing an issue similar to me with local development due to connection failing to fetch some remote assets? (I deploy to production with netlify which works fine) If so, then perhaps Gatsby needs to throw some sort of error message in CLI output about these processing/query failures upfront(and perhaps try again). For my data on a project it can take about 10 minutes or longer to reprocess again just because of a failure like this(prior to my filter hack to avoid the issue in development).

I am using remotely sourced assets via a locally authored source plugin, however these two errors quit gatsby at build time and not run time. They both appear to be happening during node creation. Which comes to mind... perhaps I should try to reproduce the results while not running my local source plugin I'll come back with my results on that.

patricksimonian picture patricksimonian on 17 Jan 2019

I am using remotely sourced assets via a _locally authored source plugin_, however these two errors quit gatsby at build time and not run time. They both appear to be happening during _node creation_. Which comes to mind... perhaps I should try to reproduce the results while not running my local source plugin I'll come back with my results on that.

Disabling my locally source plugin seems to resolve the error

skrapkam picture skrapkam on 17 Jan 2019

I am using remotely sourced assets via a _locally authored source plugin_, however these two errors quit gatsby at build time and not run time. They both appear to be happening during _node creation_. Which comes to mind... perhaps I should try to reproduce the results while not running my local source plugin I'll come back with my results on that.

Disabling my locally source plugin seems to resolve the error

can you provide a link to your plugin so i can check it out? Maybe I'm doing too much magic in mine...

patricksimonian picture patricksimonian on 17 Jan 2019

I am using remotely sourced assets via a _locally authored source plugin_, however these two errors quit gatsby at build time and not run time. They both appear to be happening during _node creation_. Which comes to mind... perhaps I should try to reproduce the results while not running my local source plugin I'll come back with my results on that.

Disabling my locally source plugin seems to resolve the error

can you provide a link to your plugin so i can check it out? Maybe I'm doing too much magic in mine...

Yeah, here's the plugin.

skrapkam picture skrapkam on 17 Jan 2019

took a quick look. The only suspect i'd see from your code is https://github.com/MantasMikal/gatsby-source-instagram-all/blob/master/src/sourceNodes.js#L13

where you delete a property from a configuration object. Other than that it looks pretty normal. My local plugin is quite the behemoth but at the end of the day it does the same thing as all source plugins which is generate nodes by calling the createNode method.

patricksimonian picture patricksimonian on 17 Jan 2019
😕1

I think I have a similar plugin to what you two are referring as local plugin? I added offline/local support to gatsby-source-contentful via createRemoteNode(), iirc I'd get some build errors sometimes that were fixed by clearing out the cache.

polarathene picture polarathene on 18 Jan 2019

yeah that's my solution to my build errors.

patricksimonian picture patricksimonian on 22 Jan 2019

Any updates or ideas on how to fix this?

skrapkam picture skrapkam on 30 Jan 2019

Nothing so far, i still get it inconsistently. One thought is that it is perhaps tied to switching branches in git. The .cache folder is ignored and so perhaps mixing those up is causing this issue. As i mentioned before, however, my work around of deleting the cache has been __working great__ so far.

patricksimonian picture patricksimonian on 15 Feb 2019
👍3

(Closed by accident—Sorry!)

danoc picture danoc on 15 Feb 2019

Hi! I'm also running into this issue.

In my case it fails when calling findChildrenRecursively (actions.js) on the children of this node:

{ id: 'e7b3d69d-4834-54d1-9edb-52c18ea6a40c',
  parent: '1671edea-3cd1-5d88-ac40-e58084c964ef',
  children: [ '2cbff358-3084-58b5-98cd-32480f02c8f2' ],
  text: 'Text that describes the current status and is only visible to\nscreenreaders.',
  internal:
   { type: 'ComponentDescription',
     mediaType: 'text/markdown',
     content: 'Text that describes the current status and is only visible to\nscreenreaders.',
     contentDigest: 'f6d656b03e614d805f851c70135e935c',
     owner: 'gatsby-transformer-react-docgen' } }

The children of that node is from gatsby-mdx.

{ id: '2cbff358-3084-58b5-98cd-32480f02c8f2',
  children: [],
  parent: 'e7b3d69d-4834-54d1-9edb-52c18ea6a40c',
  internal:
   { content: 'Text that describes the current status and is only visible to\nscreenreaders.',
     type: 'Mdx',
     contentDigest: '8b6d3163f67bce5882615be6d2b1aaf2',
     owner: 'gatsby-mdx' },
  frontmatter: { title: '', _PARENT: 'e7b3d69d-4834-54d1-9edb-52c18ea6a40c' },
  exports: {},
  rawBody: 'Text that describes the current status and is only visible to\nscreenreaders.' }

Gatsby fails on getNode(child).children on the gatsby-mdx node (2cbff358-3084-58b5-98cd-32480f02c8f2) because getNode(child) is undefined. It is undefined because it is deleted from the Redux store.

I'm not sure if this is normal, but I noticed that deleteNode (actions.js) and getNode (nodes.js) are called a bunch of times on that same node:

success open and validate gatsby-configs — 0.154 s
success load plugins — 0.180 s
success onPreInit — 0.384 s
success delete html and css files from previous builds — 0.034 s
success initialize cache — 0.004 s
success copy gatsby files — 0.103 s
success onPreBootstrap — 0.013 s
⠠ source and transform nodes
getNode 2cbff358-3084-58b5-98cd-32480f02c8f2
getNode 2cbff358-3084-58b5-98cd-32480f02c8f2
getNode 2cbff358-3084-58b5-98cd-32480f02c8f2
getNode 2cbff358-3084-58b5-98cd-32480f02c8f2
getNode 2cbff358-3084-58b5-98cd-32480f02c8f2
deleteNode 2cbff358-3084-58b5-98cd-32480f02c8f2
getNode 2cbff358-3084-58b5-98cd-32480f02c8f2
deleteNode 2cbff358-3084-58b5-98cd-32480f02c8f2
getNode 2cbff358-3084-58b5-98cd-32480f02c8f2
getNode 2cbff358-3084-58b5-98cd-32480f02c8f2
deleteNode 2cbff358-3084-58b5-98cd-32480f02c8f2
getNode 2cbff358-3084-58b5-98cd-32480f02c8f2
deleteNode 2cbff358-3084-58b5-98cd-32480f02c8f2
getNode 2cbff358-3084-58b5-98cd-32480f02c8f2
getNode 2cbff358-3084-58b5-98cd-32480f02c8f2
getNode 2cbff358-3084-58b5-98cd-32480f02c8f2
deleteNode 2cbff358-3084-58b5-98cd-32480f02c8f2
getNode 2cbff358-3084-58b5-98cd-32480f02c8f2
deleteNode 2cbff358-3084-58b5-98cd-32480f02c8f2
getNode 2cbff358-3084-58b5-98cd-32480f02c8f2
getNode 2cbff358-3084-58b5-98cd-32480f02c8f2
deleteNode 2cbff358-3084-58b5-98cd-32480f02c8f2
getNode 2cbff358-3084-58b5-98cd-32480f02c8f2
deleteNode 2cbff358-3084-58b5-98cd-32480f02c8f2
getNode 2cbff358-3084-58b5-98cd-32480f02c8f2
error Cannot read property 'children' of undefined

  TypeError: Cannot read property 'children' of undefined

  - actions.js:57 children.concat.children.map.child
    [thumbprint]/[gatsby]/dist/redux/actions.js:57:47

  - Array.map

  - actions.js:52 findChildrenRecursively
    [thumbprint]/[gatsby]/dist/redux/actions.js:52:21

  - actions.js:338 actions.deleteNode.args
    [thumbprint]/[gatsby]/dist/redux/actions.js:338:33

Can't debug any longer and will do rm -rf .cache to workaround. I've seen this bug pop up a few times (over the course of months) although I'm not sure how to reliably reproduce.

danoc picture danoc on 15 Feb 2019

~Was able to reproduce this again today. I'm not 100% sure about this, but it seems to happen after I go a certain amount of time (over a day?) without working in the codebase.~

~Do nodes have some sort of time-based expiration that causes Gatsby to mark them as "stale"?~

Never-mind, happened again after just a few minutes of not using the repo. 🤷‍♂️

danoc picture danoc on 20 Feb 2019

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 30 days of inactivity. It’s been at least 20 days since the last update here.

If we missed this issue or if you want to keep it open, please reply here. You can also add the label "not stale" to keep this issue open!

Thanks for being a part of the Gatsby community! 💪💜

gatsbot[bot] picture gatsbot[bot] on 13 Mar 2019

I ran into the same issue as well. I did delete a folder within /pages (a page and a css file), while gatsby develop was running. When I restarted gatbsy develop, I got TypeError: Cannot read property 'children' of undefined. Deleting rm -rf .cache and restarting gatsby develop solved the issue.

gvocale picture gvocale on 22 Mar 2019
👍2

I keep running into this issue as well. No matter of debugging has gotten me anywhere. It seems related to trying to create a child-parent link on something that's undefined, but no amount of guarding seems to resolve the issue.

cbetta picture cbetta on 5 May 2019

I have this issue too. Deleting rm -rf .cache and restarting gatsby develop solved the issue. But almost every time I restarted the dev server I ran into this issue.

satouriko picture satouriko on 21 Aug 2019

Hi!

Sorry to hear you're running into an issue. To help us best begin debugging the underlying cause, it is incredibly helpful if you're able to create a minimal reproduction. This is a simplified example of the issue that makes it clear and obvious what the issue is and how we can begin to debug it.

If you're up for it, we'd very much appreciate if you could provide a minimal reproduction and we'll be able to take another look.

I wasn't able to run the project listed in first comment.

Thanks for using Gatsby! 💜

wardpeet picture wardpeet on 4 Sep 2019

Feel free to open a new issue if you can clearly reproduce the error. Until then I'll close this issue as it's not actionable.

LekoArts picture LekoArts on 25 May 2020
Was this page helpful?
0 / 5 - 0 ratings

Related issues

signalwerk picture signalwerk  ·  3Comments
magicly picture magicly  ·  3Comments
andykais picture andykais  ·  3Comments
dustinhorton picture dustinhorton  ·  3Comments
benstr picture benstr  ·  3Comments