Code-server: Server build task stuck forever

Created on 6 Apr 2019  路  12Comments  路  Source: cdr/code-server


  • code-server version: 1.691-vsc1.33.0

  • OS Version: CentOS Linux release 7.6.1810

  • node version: 8.10.0

  • gcc version: 8.2.0
  • yarn version: 1.15.2

This behavior also remains on the latest release 1.604-vsc1.32.0

Description

Running the server build task gets stuck. This is the log for a while now.

yarn run v1.15.2
$ ts-node -r tsconfig-paths/register build/tasks.ts build:server:binary
INFO  BUILD:SERVER:BINARY Starting...
INFO  VSCODE:INSTALL Starting...
INFO  VSCODE:INSTALL Completed! time=242716ms {"time":{"expected":9007199254740991,"ms":1554574909554}}
INFO  BUILD:BOOTSTRAP-FORK Starting...
INFO  BUILD:WEB Starting...
INFO  BUILD:SERVER:BUNDLE Starting...
INFO  BUILD:SERVER:BUNDLE Executing 'npm ["run","build"]' {"options":{"cwd":"/home/code-server/packages/server"}}
INFO  BUILD:APP:BROWSER Starting...
INFO  VSCODE:PATCH Starting...
INFO  BUILD:APP:BROWSER Executing 'npm ["run","build"]' {"options":{"cwd":"/home/code-server/packages/app/browser"}}
INFO  VSCODE:PATCH Executing 'git ["apply","--unidiff-zero","/home/code-server/scripts/vscode.patch"]' {"options":{"cwd":"/home/code-server/lib/vscode"}}
INFO  VSCODE:PATCH Completed! time=1993ms {"time":{"expected":9007199254740991,"ms":1554575152394}}
INFO  BUILD:BOOTSTRAP-FORK Executing 'npm ["run","build:bootstrap-fork"]' {"options":{"cwd":"/home/code-server/packages/vscode"}}
INFO  BUILD:WEB Executing 'npm ["run","build"]' {"options":{"cwd":"/home/code-server/packages/web"}}
INFO  BUILD:APP:BROWSER Completed! time=41679ms {"time":{"expected":9007199254740991,"ms":1554575152394}}
INFO  BUILD:SERVER:BINARY Still running...
INFO  BUILD:BOOTSTRAP-FORK Still running...
INFO  BUILD:WEB Still running...
INFO  BUILD:SERVER:BUNDLE Still running...
INFO  BUILD:SERVER:BINARY Still running...
INFO  BUILD:BOOTSTRAP-FORK Still running...
INFO  BUILD:WEB Still running...
INFO  BUILD:SERVER:BUNDLE Still running...
INFO  BUILD:SERVER:BINARY Still running...
INFO  BUILD:BOOTSTRAP-FORK Still running...
INFO  BUILD:WEB Still running...
INFO  BUILD:SERVER:BUNDLE Still running...
INFO  BUILD:SERVER:BINARY Still running...
INFO  BUILD:BOOTSTRAP-FORK Still running...
INFO  BUILD:WEB Still running...
INFO  BUILD:SERVER:BUNDLE Still running...
INFO  BUILD:SERVER:BINARY Still running...
INFO  BUILD:BOOTSTRAP-FORK Still running...
INFO  BUILD:WEB Still running...
INFO  BUILD:SERVER:BUNDLE Still running...
INFO  BUILD:SERVER:BINARY Still running...
INFO  BUILD:BOOTSTRAP-FORK Still running...
INFO  BUILD:WEB Still running...
INFO  BUILD:SERVER:BUNDLE Still running...
INFO  BUILD:SERVER:BINARY Still running...
INFO  BUILD:BOOTSTRAP-FORK Still running...
INFO  BUILD:WEB Still running...
INFO  BUILD:SERVER:BUNDLE Still running...
INFO  BUILD:SERVER:BINARY Still running...
INFO  BUILD:BOOTSTRAP-FORK Still running...
INFO  BUILD:WEB Still running...
INFO  BUILD:SERVER:BUNDLE Still running...
INFO  BUILD:SERVER:BINARY Still running...
INFO  BUILD:BOOTSTRAP-FORK Still running...
INFO  BUILD:WEB Still running...
INFO  BUILD:SERVER:BUNDLE Still running...
INFO  BUILD:SERVER:BINARY Still running...
INFO  BUILD:BOOTSTRAP-FORK Still running...
INFO  BUILD:WEB Still running...
INFO  BUILD:SERVER:BUNDLE Still running...
INFO  BUILD:SERVER:BINARY Still running...
INFO  BUILD:BOOTSTRAP-FORK Still running...
INFO  BUILD:WEB Still running...
INFO  BUILD:SERVER:BUNDLE Still running...
INFO  BUILD:SERVER:BINARY Still running...
INFO  BUILD:BOOTSTRAP-FORK Still running...
INFO  BUILD:WEB Still running...
INFO  BUILD:SERVER:BUNDLE Still running...
INFO  BUILD:SERVER:BINARY Still running...
INFO  BUILD:BOOTSTRAP-FORK Still running...
INFO  BUILD:WEB Still running...
INFO  BUILD:SERVER:BUNDLE Still running...
INFO  BUILD:SERVER:BINARY Still running...
INFO  BUILD:BOOTSTRAP-FORK Still running...
INFO  BUILD:WEB Still running...
INFO  BUILD:SERVER:BUNDLE Still running...

Steps to Reproduce

  1. Run yarn
  2. Run yarn task build:server:binary
bug waiting-for-info
Source
picture activatedgeek
👍1

Most helpful comment

AFAIK this happens when you don't have enough RAM available. I had set docker to use 2GB and it didn't work. After setting the RAM to 8GB it did. I did some more testing and the max seems to be between 2 and 3 GB of RAM.

picture lucacasonato on 17 Apr 2019
👍2

All 12 comments

Very interesting. What happens if you go into the packages/web directory and run npm run build?

kylecarbs picture kylecarbs on 7 Apr 2019

Here's the error log. This keeps repeating for a long time (I believe it logs for each of the 63 threads in a non-deterministic order).

> @coder/web@ build /home/code-server/packages/web
> ../../node_modules/.bin/cross-env UV_THREADPOOL_SIZE=100 node --max-old-space-size=32384 ../../node_modules/webpack/bin/webpack.js --config ./webpack.config.js

Happy[ts]: Version: 5.0.1. Threads: 63
Happy[ts]: All set; signaling webpack to proceed.
HappyThread[ts:32]: unable to send to worker! Error [ERR_IPC_CHANNEL_CLOSED]: channel closed
    at ChildProcess.target.send (internal/child_process.js:588:16)
    at processSend (/home/code-server/node_modules/happypack/lib/processSend.js:8:15)
    at Object.UnbufferedFd.send (/home/code-server/node_modules/happypack/lib/UnbufferedFd.js:9:3)
    at send (/home/code-server/node_modules/happypack/lib/HappyThread.js:38:20)
    at Object.compile (/home/code-server/node_modules/happypack/lib/HappyThread.js:146:7)
    at Object.compile (/home/code-server/node_modules/happypack/lib/HappyThreadPool.js:82:14)
    at HappyPlugin.compile (/home/code-server/node_modules/happypack/lib/HappyPlugin.js:194:14)
    at Object.HappyLoader (/home/code-server/node_modules/happypack/lib/HappyLoader.js:31:15)
    at LOADER_EXECUTION (/home/code-server/node_modules/loader-runner/lib/LoaderRunner.js:119:14)
    at runSyncOrAsync (/home/code-server/node_modules/loader-runner/lib/LoaderRunner.js:120:4)
    at iterateNormalLoaders (/home/code-server/node_modules/loader-runner/lib/LoaderRunner.js:229:2)
    at Array.<anonymous> (/home/code-server/node_modules/loader-runner/lib/LoaderRunner.js:202:4)
    at Storage.finished (/home/code-server/node_modules/enhanced-resolve/lib/CachedInputFileSystem.js:43:16)
    at provider (/home/code-server/node_modules/enhanced-resolve/lib/CachedInputFileSystem.js:79:9)
    at /home/code-server/node_modules/graceful-fs/graceful-fs.js:90:16
    at FSReqWrap.readFileAfterClose [as oncomplete] (fs.js:511:3)
HappyThread[ts:34]: unable to send to worker! Error [ERR_IPC_CHANNEL_CLOSED]: channel closed
    at ChildProcess.target.send (internal/child_process.js:588:16)
    at processSend (/home/code-server/node_modules/happypack/lib/processSend.js:8:15)
    at Object.UnbufferedFd.send (/home/code-server/node_modules/happypack/lib/UnbufferedFd.js:9:3)
    at send (/home/code-server/node_modules/happypack/lib/HappyThread.js:38:20)
    at Object.compile (/home/code-server/node_modules/happypack/lib/HappyThread.js:146:7)
    at Object.compile (/home/code-server/node_modules/happypack/lib/HappyThreadPool.js:82:14)
    at HappyPlugin.compile (/home/code-server/node_modules/happypack/lib/HappyPlugin.js:194:14)
    at Object.HappyLoader (/home/code-server/node_modules/happypack/lib/HappyLoader.js:31:15)
    at LOADER_EXECUTION (/home/code-server/node_modules/loader-runner/lib/LoaderRunner.js:119:14)
    at runSyncOrAsync (/home/code-server/node_modules/loader-runner/lib/LoaderRunner.js:120:4)
    at iterateNormalLoaders (/home/code-server/node_modules/loader-runner/lib/LoaderRunner.js:229:2)
    at Array.<anonymous> (/home/code-server/node_modules/loader-runner/lib/LoaderRunner.js:202:4)
    at Storage.finished (/home/code-server/node_modules/enhanced-resolve/lib/CachedInputFileSystem.js:43:16)
    at provider (/home/code-server/node_modules/enhanced-resolve/lib/CachedInputFileSystem.js:79:9)
    at /home/code-server/node_modules/graceful-fs/graceful-fs.js:90:16
    at FSReqWrap.readFileAfterClose [as oncomplete] (fs.js:511:3)
HappyThread[ts:35]: unable to send to worker! Error [ERR_IPC_CHANNEL_CLOSED]: channel closed
    at ChildProcess.target.send (internal/child_process.js:588:16)
    at processSend (/home/code-server/node_modules/happypack/lib/processSend.js:8:15)
    at Object.UnbufferedFd.send (/home/code-server/node_modules/happypack/lib/UnbufferedFd.js:9:3)
    at send (/home/code-server/node_modules/happypack/lib/HappyThread.js:38:20)
    at Object.compile (/home/code-server/node_modules/happypack/lib/HappyThread.js:146:7)
    at Object.compile (/home/code-server/node_modules/happypack/lib/HappyThreadPool.js:82:14)
    at HappyPlugin.compile (/home/code-server/node_modules/happypack/lib/HappyPlugin.js:194:14)
    at Object.HappyLoader (/home/code-server/node_modules/happypack/lib/HappyLoader.js:31:15)
    at LOADER_EXECUTION (/home/code-server/node_modules/loader-runner/lib/LoaderRunner.js:119:14)
    at runSyncOrAsync (/home/code-server/node_modules/loader-runner/lib/LoaderRunner.js:120:4)
    at iterateNormalLoaders (/home/code-server/node_modules/loader-runner/lib/LoaderRunner.js:229:2)
    at Array.<anonymous> (/home/code-server/node_modules/loader-runner/lib/LoaderRunner.js:202:4)
    at Storage.finished (/home/code-server/node_modules/enhanced-resolve/lib/CachedInputFileSystem.js:43:16)
    at provider (/home/code-server/node_modules/enhanced-resolve/lib/CachedInputFileSystem.js:79:9)
    at /home/code-server/node_modules/graceful-fs/graceful-fs.js:90:16
    at FSReqWrap.readFileAfterClose [as oncomplete] (fs.js:511:3)
activatedgeek picture activatedgeek on 7 Apr 2019
👍2

@kylecarbs any luck here? Thanks!

activatedgeek picture activatedgeek on 9 Apr 2019

@kylecarbs Ping!

activatedgeek picture activatedgeek on 16 Apr 2019

What happens if you remove --max-old-space-size=32384 from the package.json in packages/web?

nol166 picture nol166 on 16 Apr 2019

@nol166 The same error remains.

activatedgeek picture activatedgeek on 16 Apr 2019

screenshot

the same situation occurs on my mac when building image from the Dockfile

OS Version: macOS 10.13.2
Docker version: 17.09.1-ce-mac42

I have been waiting for a long time, but it seems never end

fengzilong picture fengzilong on 17 Apr 2019

AFAIK this happens when you don't have enough RAM available. I had set docker to use 2GB and it didn't work. After setting the RAM to 8GB it did. I did some more testing and the max seems to be between 2 and 3 GB of RAM.

lucacasonato picture lucacasonato on 17 Apr 2019
👍2

Yeah, you are right. After setting the RAM to 8GB, it doesn't stuck anymore

fengzilong picture fengzilong on 17 Apr 2019

It's very interesting that the process never seems to die if its out of mem...

kylecarbs picture kylecarbs on 17 Apr 2019
👍1

I have like >32G of free memory on my remote workstation.

activatedgeek picture activatedgeek on 18 Apr 2019

Fixed on latest tree. We already use a new build pipeline. This should take you no more than 2 minutes to build on a decent machine.

sr229 picture sr229 on 12 Sep 2019
Was this page helpful?
0 / 5 - 0 ratings

Related issues

pchecinski picture pchecinski  路  3Comments
lshamis picture lshamis  路  3Comments
sa7mon picture sa7mon  路  3Comments
justmao945 picture justmao945  路  3Comments
pchecinski picture pchecinski  路  3Comments