Next.js: OOM on 9.3.6

[VinSpee]

Bug report

Describe the bug

on a next dev or next build, node runs out of memory.

To Reproduce

run yarn build
see this error:

$ next build
$ tsc --project tsconfig.server.json
Warning: Built-in CSS support is being disabled due to custom CSS configuration being detected.
See here for more info: https://err.sh/next.js/built-in-css-disabled

Creating an optimized production build .
<--- Last few GCs --->

[7951:0x108008000]    35387 ms: Mark-sweep 1952.3 (2088.8) -> 1936.1 (2088.0) MB, 313.2 / 0.0 ms  (average mu = 0.157, current mu = 0.070) allocation failure scavenge might not succeed
[7951:0x108008000]    35714 ms: Mark-sweep 1953.8 (2092.3) -> 1938.5 (2090.8) MB, 302.1 / 0.0 ms  (average mu = 0.124, current mu = 0.076) allocation failure scavenge might not succeed


<--- JS stacktrace --->

==== JS stack trace =========================================

    0: ExitFrame [pc: 0x100931399]
Security context: 0x3c879d1408a1 <JSObject>
    1: add [0x3c879d14ddb9](this=0x3c8743530c61 <Set map = 0x3c87b6fc23c1>,0x3c87663ba029 <Very long string[47196]>)
    2: new Set(aka Set) [0x3c879d14d8a9](this=0x3c8711c80589 <the_hole>,0x3c8793782321 <Set map = 0x3c87b6fc23c1>)
    3: ConstructFrame [pc: 0x1008ad37a]
    4: StubFrame [pc: 0x100991b00]
    5: doResolve [0x3c87f79a1a19] [/...

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0x10007f4b9 node::Abort() [/.fnm/node-versions/v12.15.0/installation/bin/node]
 2: 0x10007f63d node::OnFatalError(char const*, char const*) [/.fnm/node-versions/v12.15.0/installation/bin/node]
 3: 0x100176a27 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/.fnm/node-versions/v12.15.0/installation/bin/node]
 4: 0x1001769c3 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/.fnm/node-versions/v12.15.0/installation/bin/node]
 5: 0x1002fab75 v8::internal::Heap::FatalProcessOutOfMemory(char const*) [/.fnm/node-versions/v12.15.0/installation/bin/node]
 6: 0x1002fc244 v8::internal::Heap::RecomputeLimits(v8::internal::GarbageCollector) [/.fnm/node-versions/v12.15.0/installation/bin/node]
 7: 0x1002f9117 v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/.fnm/node-versions/v12.15.0/installation/bin/node]
 8: 0x1002f70fd v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/.fnm/node-versions/v12.15.0/installation/bin/node]
 9: 0x100302814 v8::internal::Heap::AllocateRawWithLightRetry(int, v8::internal::AllocationType, v8::internal::AllocationAlignment) [/.fnm/node-versions/v12.15.0/installation/bin/node]
10: 0x10030288f v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationType, v8::internal::AllocationAlignment) [/.fnm/node-versions/v12.15.0/installation/bin/node]
11: 0x1002ce269 v8::internal::Factory::NewFixedArrayWithFiller(v8::internal::RootIndex, int, v8::internal::Object, v8::internal::AllocationType) [/.fnm/node-versions/v12.15.0/installation/bin/node]
12: 0x100507bab v8::internal::OrderedHashTable<v8::internal::OrderedHashSet, 1>::Rehash(v8::internal::Isolate*, v8::internal::Handle<v8::internal::OrderedHashSet>, int) [/.fnm/node-versions/v12.15.0/installation/bin/node]
13: 0x1005df0f9 v8::internal::Runtime_SetGrow(int, unsigned long*, v8::internal::Isolate*) [/.fnm/node-versions/v12.15.0/installation/bin/node]
14: 0x100931399 Builtins_CEntry_Return1_DontSaveFPRegs_ArgvOnStack_NoBuiltinExit [/.fnm/node-versions/v12.15.0/installation/bin/node]
error Command failed with signal "SIGABRT".

Expected behavior

build runs successfully.

System information

• OS: MacOS, ubuntu
• Version of Next.js: 9.3.6
• Version of Node.js: 12.15.0

Additional context

builds fine with 9.3.5.

[timneutkens]

Hi we'd be happy to investigate. But it's hard to help on this if no reproduction is provided.

[VinSpee]

Understood. I thought it might be something more obvious because 9.3.5 is totally fine.

[zetoke]

I have the same problem. The build of my project have just started to have OOM error on CI (in the branch with next.js upgraded from 9.3.5 to 9.3.6)

[timneutkens]

@VinSpee @zetoke feel free to investigate this issue if you're not going to provide a reproduction. I'd start by downgrading through all canary versions between 9.3.5 and 9.3.6

[zetoke]

@timneutkens yeah, it's a good idea. I will try it out.
The challenge with a reproduction is that it's probably a problem for medium sized projects or bigger and I can't just opensource the whole project haha. So the way is just to create a lot of dummy pages/components. Probably.

[VinSpee]

@zetoke, a couple of questions which might help narrow things:

1. would you mind sharing your next.config.js?
2. are you using a custom server? is it using ts-node?

here's my next.config.js:

const withTM = require('next-transpile-modules')([
  'rambda',
  'react-lazy-load-image-component',
]);
const withPlugins = require('next-compose-plugins');
const dotenv = require('dotenv');
const withCSS = require('@zeit/next-css');

const SECRETS = dotenv.config({ path: './config/secrets.env' });
const CONFIG = dotenv.config({ path: './config/config.env' });

module.exports = withPlugins([withTM, withCSS], {
  poweredByHeader: false,
  webpack(config) {
    config.module.rules.push({
      test: /\.svg$/,
      use: {
        loader: '@svgr/webpack',
        options: {
          dimensions: false,
        },
      },
    });
    config.module.rules.push({
      test: /\.po/,
      use: [
        {
          loader: '@lingui/loader',
        },
      ],
    });
    config.module.rules.push({
      test: /\.gql$/,
      exclude: /node_modules/,
      use: { loader: 'graphql-tag/loader' },
    });

    return config;
  },
  publicRuntimeConfig: {…},
  env: {…},
});

[BjoernRave]

I have the same issue and reported it earlier: https://github.com/zeit/next.js/issues/12102

[VinSpee]

@BjoernRave would you mind answering these questions too?

[duvholt]

Looks like the issue was introduced in [email protected].
[email protected] builds fine for me.

Unfortunately the project I can reproduce this on is proprietary.

[BjoernRave]

@VinSpee sure.
1.

const withBundleAnalyzer = require('@next/bundle-analyzer')({
  enabled: process.env.ANALYZE === 'true',
})
require('dotenv').config()

module.exports = withBundleAnalyzer({
  target: 'serverless',
  env: {
    SENTRY_FRONTEND_DSN: process.env.SENTRY_FRONTEND_DSN,
    BACKEND_URL: process.env.BACKEND_URL,
    WEB_AUTH0_CLIENT_ID: process.env.WEB_AUTH0_CLIENT_ID,
    WEB_AUTH0_CLIENT_SECRET: process.env.WEB_AUTH0_CLIENT_SECRET,
    AUTH_SECRET: process.env.AUTH_SECRET,
    AUTH0_DOMAIN: process.env.AUTH0_DOMAIN,
    API_AUTH0_AUDIENCE: process.env.API_AUTH0_AUDIENCE,
  },
})

1. I am not using a custom server.

My project is also rather big and I can't share the source code...

[zetoke]

Looks like i found a point where it started to fail.
It was turning on paths support by default in 9.3.6-canary.3
I don't know why but I had node_modules in tsconfig paths and it was a problem.

So this change made everything work as expected

@VinSpee sorry for not answering questions

1. Yeah, I can if it's still actual
2. Yeah. But I use ts-node only for development purposes (so only next build fails in my case).

[zetoke]

So I think the workaround/fix for people can be looking into paths in tsconfig or jsconfig and making sure that it doesn't have weird stuff as in a previous message.

[VinSpee]

@zetoke interesting!

I had this in mine:

    "paths": {
      "*": ["*"],
      …

which, when i changed to

    "paths": {
      "src/*": ["*"],

is now no longer failing. Good eye!

[brianlovin]

Same issue on my end this morning when bumping from 9.3.5 => 9.3.6

next.config.js:

const path = require('path')

module.exports = {
  webpack: (config) => {
    config.resolve.alias['~'] = path.resolve('./src')
    return config
  },
}

tsconfig.json has this paths key:

{
  "compilerOptions": {
    // ...
    "paths": {
      "~/*": [
        "src/*"
      ],
      "*": [
        "node_modules/*"
      ]
    },
  },
}

Edit (resolved):

Removing this from my tsconfig.json paths fixed the OOM issue:

"*": [
  "node_modules/*"
]

[BjoernRave]

after cleaning up "paths" in my tsconfig.json it works fine. So pretty sure it's related to that

[Tylerian]

I also have this problem when upgrading to 9.3.6.

I had the line below on my paths in tsconfig.json

"*": [
  "node_modules/*"
]

Removing it as said by @zetoke solved it too for me

[timneutkens]

I'm super curious why you all had those lines there to begin with, doesn't seem like it does anything to improve your application.

We'll figure out a way to make it work though.

[VinSpee]

@timneutkens I _think_ it's an attempt to make relative paths automatic, ex: "any file/folder in this directory can be referenced without relative paths"

[timneutkens]

Opened a PR to fix it: #12318

[Tylerian]

I'm super curious why you all had those lines there to begin with, doesn't seem like it does anything to improve your application.

We'll figure out a way to make it work though.

@timneutkens I think It was an attempt to tell ts compiler to find all modules without a relative import to resolve from node_modules, not being aware of the fact that ts compiler node resolution algorithm was already doing this for us.

Perhaps this should be marked as malpractice and an error should be thrown if attempted?