Nuxt.js: nuxt generate memory leak

[andrewharvey]

Versions

• nuxt: 2.14.1
• node: v14.7.0

Reproduction

I'm using nuxt generate with the memory benchmarking from https://github.com/nuxt/nuxt.js/issues/7412#issuecomment-643195374.

I've reduced the generate.concurrency down to 1 and I can see in the logs that after "Generating pages with full static mode" it start's going through my pages and the heap memory just keeps on growing.

After about 100 pages generated it runs out of heap memory FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory.

I'm using 3GB of heap memory with node --max_old_space_size=3072 ./node_modules/.bin/nuxt generate, sure I could increase this but with hundreds of pages it's not scalable.

This wasn't happening in the older version of nuxt 2.12.2

What is Expected?

I expect that generating pages would have roughly flat memory usage and not keep on growing with each generated page.

What is actually happening?

Memory usage keeps growing with each generated page.

[andrewharvey]

I discovered this only happens on the first run (but reliably on the first fresh run), if after the crash due to out of memory I run the same generate command again, heap usage hovers around 500MB and stays there throughout each generated route.

So my weird workaround is to let it crash and run again.

[pi0]

Hi @andrewharvey. Do you have a reproduction repo or if hard possibily privately sharing repo?

[andrewharvey]

Hi @andrewharvey. Do you have a reproduction repo or if hard possibily privately sharing repo?

I'll try to cut it down into something reproducible, but for a large application this isn't straightforward.

I tried running an allocation timeline profile, but because of the resources required just to capture it I could only capture a few seconds, and to be honest I can't really understand it

[andrewharvey]

this sampling profile seems to show it's mostly in an anonymous function in async function createApp(ssrContext, config = {}) {.

Not that this probably helps much without a minimal replication....

[donbot-dev]

Same problem! Nuxt generates 1/5 pages, freezes, eats up all memory and crashes. Last normally working version for me 2.12+

[Spreizu]

I also encountered this problem while first upgrading to Nuxt 2.13 (Javascript heap out of memory while generating pages). For me, this happened for 2 reasons:
1) A very large payload was manually specified to the route (~6 MB);
2) $route object was passed to the store commit call as a parameter inside page asyncData
After optimizing the payload size and refactoring the store mutation logic, this issue disappeared. However, the root cause is still unknown.

[AliasIO]

I'm running into this issue as well. My repository is public: https://github.com/AliasIO/wappalyzer.com

<--- Last few GCs --->

[54002:0x102409000]  1097925 ms: Mark-sweep 3879.9 (4117.9) -> 3874.1 (4111.7) MB, 2800.7 / 0.0 ms  (average mu = 0.138, current mu = 0.022) allocation failure scavenge might not succeed
[54002:0x102409000]  1098012 ms: Scavenge 3878.3 (4111.7) -> 3875.3 (4111.7) MB, 42.2 / 0.0 ms  (average mu = 0.138, current mu = 0.022) allocation failure 
[54002:0x102409000]  1098090 ms: Scavenge 3891.0 (4114.3) -> 3879.6 (4115.8) MB, 19.7 / 0.0 ms  (average mu = 0.138, current mu = 0.022) allocation failure 


<--- JS stacktrace --->

==== JS stack trace =========================================

    0: ExitFrame [pc: 0x10072e699]
Security context: 0x00a190b408a1 <JSObject>
    1: join [0xa190b51239](this=0x00a1e3304fe1 <JSArray[39]>,0x00a113300729 <String[#0]: >)
    2: intoTokens [0xa1effb7e09] [/Users/elbert/Sites/wappalyzer/v4/frontend/node_modules/clean-css/lib/tokenizer/tokenize.js:~80] [pc=0x3b7f26fab657](this=0x00a170d44cf1 <GlobalObject Object map = 0xa1ce51f2e1>,0x00a259d00111 <Very long string[386516]>,0x00a2579c9251 <O...

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory

Writing Node.js report to file: report.20200831.135051.54002.0.001.json
Node.js report completed
 1: 0x100b6a52a node::Abort() (.cold.1) [/usr/local/Cellar/node/12.12.0/bin/node]
 2: 0x100081f70 node::FatalError(char const*, char const*) [/usr/local/Cellar/node/12.12.0/bin/node]
 3: 0x100082098 node::OnFatalError(char const*, char const*) [/usr/local/Cellar/node/12.12.0/bin/node]
 4: 0x10017461d v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/usr/local/Cellar/node/12.12.0/bin/node]
 5: 0x1001745c7 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/usr/local/Cellar/node/12.12.0/bin/node]
 6: 0x10028a569 v8::internal::Heap::FatalProcessOutOfMemory(char const*) [/usr/local/Cellar/node/12.12.0/bin/node]
 7: 0x10028b8ee v8::internal::Heap::MarkCompactPrologue() [/usr/local/Cellar/node/12.12.0/bin/node]
 8: 0x1002894eb v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/usr/local/Cellar/node/12.12.0/bin/node]
 9: 0x100287f93 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/usr/local/Cellar/node/12.12.0/bin/node]
10: 0x10028f6da v8::internal::Heap::AllocateRawWithLightRetry(int, v8::internal::AllocationType, v8::internal::AllocationAlignment) [/usr/local/Cellar/node/12.12.0/bin/node]
11: 0x10028fbd0 v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationType, v8::internal::AllocationAlignment) [/usr/local/Cellar/node/12.12.0/bin/node]
12: 0x10026d8c6 v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationType) [/usr/local/Cellar/node/12.12.0/bin/node]
13: 0x1004bc4ee v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [/usr/local/Cellar/node/12.12.0/bin/node]
14: 0x10072e699 Builtins_CEntry_Return1_DontSaveFPRegs_ArgvOnStack_NoBuiltinExit [/usr/local/Cellar/node/12.12.0/bin/node]
15: 0x100742e9a Builtins_ArrayPrototypeJoin [/usr/local/Cellar/node/12.12.0/bin/node]
16: 0x3b7f26fab657 
17: 0x3b7f26fac5d4

[Atinux]

Hi, you could try disabling the crawler?

export default {
  generate: {
    crawler: false
  }
}

[AliasIO]

@Atinux That fixes it, no issue on 2.14 with crawler set to false. I can also confirm it was working fine on 2.12 without disabling the crawler.

[bukhalo]

Hi, you could try disabling the crawler?

@Atinux Unfortunately nothing has changed

UPD:

After all, there was an error in my code...

In my case, the problem arose because in one of the components, the name of the nested component and the name of the exported class coincided.

import { Component, Vue } from 'nuxt-property-decorator';

@Component({
  components: {
    Blablabla: () => import('@/components/Blablabla.vue'), // Here 
  },
})
export default class Blablabla extends Vue {} // And here 😁

I have no idea why it worked normally in on Nuxt 2.12+. Maybe it'll help somebody.

[AliasIO]

@Atinux I thought disabling the crawler fixed it but I'm having the same issue again. I downgraded to 2.12 for now.

[gabrielsze]

Having the same issue when i upgraded from 2.13 to 2.14, fails at nuxt generate with/without target static.

It seems to work for fully static if i cancel and run the command again (2 tries to work every time), but without full static (original legacy static) it doesn't seem to work and hangs.

Not sure if related but https://github.com/nuxt/nuxt.js/issues/5546,

Downgraded to 2.13 for now

[gryphonmyers]

I just spent a whole day trying to optimize what I thought was a leaky application. I'm increasingly certain that the problem lies with Nuxt, especially hearing I'm not the only one. Over the course of debugging memory consumption, I can see that even after the generate process has completed, our generate process leaves over 2GB sitting in heap.

Our application is fetching fairly large payloads in the fetch method, across most routes of a fairly large site (many hundreds of routes). If I limit the amount of data that is being fetched here, the problem is less noticeable. Thus the problem seems to scale both with the amount of data being fetched and the number of routes being generated.

[andrewharvey]

@pi0 I spent a few hours reducing my project to a minimally reproducing example.

https://github.com/andrewharvey/nuxt-bug-7855

Running yarn run generate the memory will keep growing for each page and never come down, all the way until running out of space. Increasing max_old_space_size might buy you a couple extra pages, but that's not scalable and doesn't address the root cause.

There are still a few pieces but I've hit the point where if I take something out I don't see the issue anymore, so it's hard to work out what exactly is causing it.

UPDATE: though if I replace https://github.com/andrewharvey/nuxt-bug-7855/blob/master/store/index.js#L15 with the similar bit from earlier in the file then looks like the issue goes away...

[stale[bot]]

Thanks for your contribution to Nuxt.js!
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.
If you would like this issue to remain open:

1. Verify that you can still reproduce the issue in the latest version of nuxt-edge
2. Comment the steps to reproduce it

Issues that are labeled as pending will not be automatically marked as stale.

[andrewharvey]

@stale I updated https://github.com/andrewharvey/nuxt-bug-7855 to use nuxt-edge and it's still reproducing as per https://github.com/nuxt/nuxt.js/issues/7855#issuecomment-698332583

[stale[bot]]

Thanks for your contribution to Nuxt.js!
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.
If you would like this issue to remain open:

1. Verify that you can still reproduce the issue in the latest version of nuxt-edge
2. Comment the steps to reproduce it

Issues that are labeled as pending will not be automatically marked as stale.

[andrewharvey]

@stale Yes I can still replicate, updated nuxt-edge to latest. However the workaround does seem to make a difference, so that might be helpful for anyone interested in looking at this.