Nx: Using affected:build in CI yields spawnSync error

[rpd10]

Expected Behavior

When running affected:build in CI, Nx should build the affected apps.

Current Behavior

The affected:build command fails before running any tests.

Failure Information (for bugs)

Build output:

$ ./node_modules/.bin/nx affected:build --base=origin/master --prod --build-optimizer

>  NX   ERROR  There was a critical error when running your command

  spawnSync /bin/sh ENOBUFS

error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
ERROR: Job failed: command terminated with exit code 1

Steps to Reproduce

We are using FROM node:12 as our base docker image in CI.

Context

Please provide any relevant information about your setup:

• version of Nx used: 8.5.1

[rpd10]

This repository has a large commit history. I tried the max_old_space_size trick for node, with no change:

$ node --max_old_space_size=8000 ./node_modules/.bin/nx affected:build --base=origin/master --prod

>  NX   ERROR  There was a critical error when running your command

  spawnSync /bin/sh ENOBUFS

[vsavkin]

@rpd10 I believe the issue is with the fact that some command exceed the buffer size, which is 200kb.

Could you run the following three command to measure the size of the output?

git diff --name-only --base=origin/master
git diff --name-only HEAD .
git ls-files --others --exclude-standard

[rpd10]

Indeed, git ls-files --others --exclude-standard has a huge output, so much so that GitLab bails out: "Job's log exceeded limit of 4194304 bytes".

We have CI setup to put the .yarn cache into the working directory (I believe for Cypress). I think newer versions of Cypress has better support for caching, I will look into using the default cache location for Cypress and yarn, and report back.

[rpd10]

Moving the yarn cache outside the working directory does fix this problem, affected:build, etc. works now.

I'm not sure whether there is anything to fix here from the Nx side (maybe a better error message?). I can close this issue unless you would like it to remain open.

[vsavkin]

One thing we can do is to bump up the size of the buffer to something like 10mb, and a better error message. Would you be interested in contributing the change? I can help.

[rpd10]

I've pushed up a PR for this.

[MichaelKaaden]

Hi,

today I ran into the same problem using nx affected in my Jenkins pipeline: Error: spawnSync /bin/sh ENOBUFS.

After some investigation, I saw that git ls-files --others --exclude-standard produced an output with a size of 11 MB because of Yarn's .cache directory inside the workspace. Well, that's definitely above rdp10's PR.

After some thought, I found a very simple solution: git ls-files --others --exclude-standard ignores everything that is in your .gitignore file. So, to get rid of this error, it's sufficient to put .cache inside your .gitignore file.

@vsavkin: Would perhaps be worth a hint in the documentation. ;-)

Hope that helps.

Best wishes,
Michael

[robinpellegrims]

Had the same issue today in my gitlab ci pipeline. Turned out my .gitignore file was ignored by my .dockerignore file, which made git ls-files --others --exclude-standard return the whole contents of node_modules.. Removing .gitignore from the .dockerignore file solved it.

Thanks for pointing me in the right direction @MichaelKaaden.

[namvhoang02]

@MichaelKaaden thanks alot, you save my day!
I have been stuck for 6 hours on this bug :(