Build: Out-of-memory errors when using cache utils

[jlengstorf]

the cache utils run out of memory when creating hashes of the directory contents

example error:

3:33:44 PM: <--- Last few GCs --->
3:33:44 PM: [1842:0x2df1650]  1401592 ms: Mark-sweep 921.7 (1027.3) -> 921.7 (990.3) MB, 96.1 / 0.0 ms  (average mu = 0.872, current mu = 0.000) last resort GC in old space requested
3:33:44 PM: [1842:0x2df1650]  1401703 ms: Mark-sweep 921.7 (990.3) -> 921.7 (985.8) MB, 111.4 / 0.0 ms  (average mu = 0.725, current mu = 0.000) last resort GC in old space requested
3:33:44 PM: <--- JS stacktrace --->
3:33:44 PM: ==== JS stack trace =========================================
3:33:44 PM:     0: ExitFrame [pc: 0xb8cacddbe1d]
3:33:44 PM: Security context: 0x1936e820b479 <JSObject>
3:33:44 PM:     1: _transform [0x21eb52b37bd1] [internal/crypto/hash.js:43] [bytecode=0x2904968ff4b1 offset=13](this=0x39ed73d8a571 <Hash map = 0xca1c0a7d149>,chunk=0x39ed73d8a3a9 <Very long string[279812793]>,encoding=0x2bf365a862c9 <String[6]: buffer>,callback=0x39ed73d8ac59 <JSBoundFunction (BoundTargetFunction 0x10cd60649789)>)
3:33:44 PM:     2: _read [0xa752c7cf7f9] [_stream_t...
3:33:44 PM: FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
3:33:44 PM:  1: 0x8fa090 node::Abort() [/opt/buildhome/.nvm/versions/node/v10.19.0/bin/node]
3:33:44 PM:  2: 0x8fa0dc  [/opt/buildhome/.nvm/versions/node/v10.19.0/bin/node]
3:33:44 PM:  3: 0xb0052e v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/opt/buildhome/.nvm/versions/node/v10.19.0/bin/node]
3:33:44 PM:  4: 0xb00764 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/opt/buildhome/.nvm/versions/node/v10.19.0/bin/node]
3:33:44 PM:  5: 0xef4c72  [/opt/buildhome/.nvm/versions/node/v10.19.0/bin/node]
3:33:44 PM:  6: 0xf0448f v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [/opt/buildhome/.nvm/versions/node/v10.19.0/bin/node]
3:33:44 PM:  7: 0xed403b v8::internal::Factory::NewRawTwoByteString(int, v8::internal::PretenureFlag) [/opt/buildhome/.nvm/versions/node/v10.19.0/bin/node]
3:33:44 PM:  8: 0x101d8c3 v8::internal::String::SlowFlatten(v8::internal::Handle<v8::internal::ConsString>, v8::internal::PretenureFlag) [/opt/buildhome/.nvm/versions/node/v10.19.0/bin/node]
3:33:44 PM:  9: 0xafded4 v8::internal::String::Flatten(v8::internal::Handle<v8::internal::String>, v8::internal::PretenureFlag) [/opt/buildhome/.nvm/versions/node/v10.19.0/bin/node]
3:33:44 PM: 10: 0xb200b9 v8::String::WriteUtf8(v8::Isolate*, char*, int, int*, int) const [/opt/buildhome/.nvm/versions/node/v10.19.0/bin/node]
3:33:44 PM: 11: 0x9c5c2b node::StringBytes::Write(v8::Isolate*, char*, unsigned long, v8::Local<v8::Value>, node::encoding, int*) [/opt/buildhome/.nvm/versions/node/v10.19.0/bin/node]
3:33:44 PM: 12: 0x91d2c4 node::StringBytes::InlineDecoder::Decode(node::Environment*, v8::Local<v8::String>, v8::Local<v8::Value>, node::encoding) [/opt/buildhome/.nvm/versions/node/v10.19.0/bin/node]
3:33:44 PM: 13: 0xa1131b node::crypto::Hash::HashUpdate(v8::FunctionCallbackInfo<v8::Value> const&) [/opt/buildhome/.nvm/versions/node/v10.19.0/bin/node]
3:33:44 PM: 14: 0xb8e96f  [/opt/buildhome/.nvm/versions/node/v10.19.0/bin/node]
3:33:44 PM: 15: 0xb8f4d9 v8::internal::Builtin_HandleApiCall(int, v8::internal::Object**, v8::internal::Isolate*) [/opt/buildhome/.nvm/versions/node/v10.19.0/bin/node]
3:33:44 PM: 16: 0xb8cacddbe1d
3:33:44 PM: ​
3:33:44 PM: ┌─────────────────────────────────────────────────────┐
3:33:44 PM: │ Plugin "netlify-plugin-gatsby-cache" internal error │
3:33:44 PM: └─────────────────────────────────────────────────────┘
3:33:44 PM: ​
3:33:44 PM: ❯ Error message
3:33:44 PM:   Plugin exited with exit code null and signal SIGABRT.
3:33:44 PM:   Instead of calling process.exit(), plugin methods should either return (on success) or throw errors (on failure).​

[jasonLaster]

Here is the full log and an STR:
https://gist.github.com/jasonLaster/1415b1e27b215a17122985d56282c7d6

the issue is in this repo http://github.com/climatescape/climatescape.org when we build. If you need any environment variables ping me on twitter and i'd be happy to share https://twitter.com/jasonlaster11

[ehmicky]

Thanks for reporting this!
There's a PR at #1054 which should fix this problem.

[ehmicky]

@jasonLaster This should be fixed now.
The Netlify Build version shown in your build should show up as 0.1.111 or later.
Could you please try and confirm this works for you? Thanks!

[jasonLaster]

Thanks @ehmicky

We changed our build step to use lets photos. I can try and revert that
change later and see if it is fixed

On Thu, Mar 26, 2020 at 11:04 AM ehmicky [e-mail] wrote:

@jasonLaster https://github.com/jasonLaster This should be fixed now.
The Netlify Build version shown in your build should show up as 0.1.111
or later.
Could you please try and confirm this works for you? Thanks!

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/netlify/build/issues/1026#issuecomment-604587229, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAB6EYRWPKL6LJQZYIIXRGLRJOKKLANCNFSM4LSHU66A
.

--

--
Jason Laster
Software Engineer

moz://a