Gatsby: CircleCI build Error: spawn ENOMEM

[andreivmaksimov]

Description

CircleCI builds failed with the following error:

  Error: spawn ENOMEM

  - child_process.js:372 ChildProcess.spawn
    internal/child_process.js:372:11

  - worker.js:80 exports.default._initialize
    [src]/[jest-worker]/build/worker.js:80:69

  - worker.js:206 exports.default._exit
    [src]/[jest-worker]/build/worker.js:206:12

  - child_process.js:254 Process.ChildProcess._handle.onexit
    internal/child_process.js:254:12

Steps to reproduce

Slightly modify f74d723f037ca55068b456dc808ee3dd8ec563cc and send new pull request.

Expected result

Gatsby build should pass

Actual result

Build is not passing

Environment

System:
OS: Linux 4.15 Debian GNU/Linux 9 (stretch) 9 (stretch)
CPU: (36) x64 Intel(R) Xeon(R) Platinum 8124M CPU @ 3.00GHz
Shell: 4.4.12 - /bin/bash
Binaries:
Node: 11.5.0 - /usr/local/bin/node
Yarn: 1.12.3 - /usr/local/bin/yarn
npm: 6.4.1 - /usr/local/bin/npm
Languages:
Python: 2.7.13 - /usr/bin/python
npmPackages:
gatsby: ^2.3.31 => 2.3.31
gatsby-image: ^2.0.37 => 2.0.37
gatsby-paginate: ^1.0.17 => 1.0.17
gatsby-plugin-catch-links: ^2.0.13 => 2.0.13
gatsby-plugin-favicon: ^3.1.6 => 3.1.6
gatsby-plugin-google-tagmanager: ^2.0.12 => 2.0.12
gatsby-plugin-manifest: ^2.0.26 => 2.0.26
gatsby-plugin-netlify: ^2.0.16 => 2.0.16
gatsby-plugin-offline: ^2.1.0 => 2.1.0
gatsby-plugin-react-helmet: ^3.0.12 => 3.0.12
gatsby-plugin-sentry: ^1.0.1 => 1.0.1
gatsby-plugin-sharp: ^2.0.32 => 2.0.32
gatsby-plugin-sitemap: ^2.0.11 => 2.0.11
gatsby-plugin-typography: ^2.2.10 => 2.2.10
gatsby-remark-images: ^3.0.10 => 3.0.10
gatsby-remark-prismjs: ^3.2.7 => 3.2.7
gatsby-source-filesystem: ^2.0.28 => 2.0.28
gatsby-transformer-json: ^2.1.11 => 2.1.11
gatsby-transformer-remark: ^2.3.8 => 2.3.8
gatsby-transformer-sharp: ^2.1.17 => 2.1.17

[pieh]

Can you add GATSBY_CPU_COUNT=2 env var to your build command in CircleCi and see if this helps?

-command: npx gatsby build
+command: GATSBY_CPU_COUNT=2 npx gatsby build

[andreivmaksimov]

@pieh, thanks for advice, but it was not helpful (https://circleci.com/gh/hands-on-cloud/hands-on.cloud/23). I've already tried to play with GATSBY_CPU_COUNT={1,2} and --max-old-space-size={1024,2048,4096}.

I've also tried to use Node 12.2.0. Still getting this error.

This generate images part of deployment is crucial part of whole framework. With each blog post deployment it takes more resources because of growing images count. Right now I have 20 posts and 160 images with mean size 250k. Probably we have memory leak somewhere, but I'm not strong Node dev yet to prove\fix it by myself.

[andreivmaksimov]

@pieh, probably, guys from next.js found the reason of the problem: https://github.com/zeit/next.js/issues/6353.

They added experimental CPU configuration option through CIRCLE_NODE_TOTAL env var: https://github.com/zeit/next.js/pull/6604/files.

[DSchau]

This may be fixed in #14624. Sharp wasn't actually _using_ this environment variable, so the images may have been using too many CPUs as a result!

[andreivmaksimov]

@DSchau, thanks for reply, but that was not helpful. Just updated gatsby and its plugins:

    "gatsby": "^2.9.4",
    "gatsby-image": "^2.1.4",
    "gatsby-paginate": "^1.1.0",
    "gatsby-plugin-catch-links": "^2.0.15",
    "gatsby-plugin-favicon": "^3.1.6",
    "gatsby-plugin-google-tagmanager": "^2.0.15",
    "gatsby-plugin-manifest": "^2.1.1",
    "gatsby-plugin-netlify": "^2.0.17",
    "gatsby-plugin-offline": "^2.1.3",
    "gatsby-plugin-react-helmet": "^3.0.12",
    "gatsby-plugin-sentry": "^1.0.1",
    "gatsby-plugin-sharp": "^2.1.5",
    "gatsby-plugin-sitemap": "^2.1.0",
    "gatsby-plugin-typography": "^2.2.13",
    "gatsby-remark-images": "^3.0.16",
    "gatsby-remark-images-contentful": "^2.0.13",
    "gatsby-remark-prismjs": "^3.2.11",
    "gatsby-source-contentful": "^2.0.71",
    "gatsby-source-filesystem": "^2.0.39",
    "gatsby-transformer-json": "^2.1.11",
    "gatsby-transformer-remark": "^2.4.0",
    "gatsby-transformer-sharp": "^2.1.21",
    "gatsby-transformer-sqip": "^2.0.42",

Played again with GATSBY_CPU_COUNT={1,2} and --max-old-space-size={1024,2048,4096}, but was not able to make it work even on node versions 8, 10, 12.

Well, I moved away from CircleCI to CodeBuild 😕

[dannyphillips]

I also recently had this issue, and no success with any of these fixes.

[crofty]

We were running into issues that I think were related to this. Our builds were happening within a docker container on CircleCI. The issues went away when we switched away from using docker containers and building using the machine executor.
Could be worth a try for anyone else blocked by this issue.

[dannyphillips]

@KyleAMathews are you able to help? We're currently blocked by this with no solution.

[henriquea]

Just a heads-up. We had the same issue here today and in our case we had a few images larger than 4000px and resizing them to 1920px solved the issue without having to touch circle config.

[gatsbot[bot]]

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 30 days of inactivity. It’s been at least 20 days since the last update here.

If we missed this issue or if you want to keep it open, please reply here. You can also add the label "not stale" to keep this issue open!

As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contributefor more information about opening PRs, triaging issues, and contributing!

Thanks for being a part of the Gatsby community! 💪💜

[gatsbot[bot]]

Hey again!

It’s been 30 days since anything happened on this issue, so our friendly neighborhood robot (that’s me!) is going to close it.

Please keep in mind that I’m only a robot, so if I’ve closed this issue in error, I’m HUMAN_EMOTION_SORRY. Please feel free to reopen this issue or create a new one if you need anything else.

As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks again for being part of the Gatsby community!

[derakhshanfar]

same issue for me

[ruucm]

same here

[hendotcat]

Hey @derakhshanfar and @ruucm it might be worth trying the parallel = false trick in this comment:
https://github.com/webpack-contrib/terser-webpack-plugin/issues/143#issuecomment-570539870

[leonfs]

To all having this problem, check how many worker processes you have when running a gatsby build in CircleCI

I experienced a problem with:

• gatsby-plugin-iltorb: where the worker farm library uses require('os').cpus() to set the number of workers.
  
  
  
  • gatsby: the same as with the iltorb plugin, but you can override it with an environment variable. The problem was that a bug caused the environment variable to be ignored.
  
  

A good way to tests this is to ssh into the CircleCI job machine, and trigger a build while also looking at the processes (using ps or top).

Hope this helps.
Leo

[czabaj]

In our case setting env variable GATSBY_CPU_COUNT=1 helped (running [email protected])

Later we increased circle.ci resource_class to xlarge which solved ths issue. Apparently 4GB of RAM is not enough. Beware the resource_class feature might not be allowed in your plan, find out in your profile or contact circle.ci support.

[papistan]

start: GATSBY_CPU_COUNT=2 yarn start totally worked.

For us it was failing contstantly when running cypress, so added it in .circleci/config.yml like:

workflows:
  build:
    jobs:
      - cypress/run:
          start: GATSBY_CPU_COUNT=2 yarn start
          wait-on: "http://localhost:8000"