Webpack-cli: RangeError: Invalid string length

[kubijo]

Describe the bug

I've started the migration to webpack@5 and from what
I can summise it seems that the compilation is able to
pretty much finish, but than it fails with following error.

By what the linked sources are about I've tried
to set stats: false, however nothing changed.

What is the current behavior?

[webpack-cli] Promise rejection: RangeError: Invalid string length
[webpack-cli] RangeError: Invalid string length
    at JSON.stringify (<anonymous>)
    at Compiler.compilerCallback (.yarn/$$virtual/webpack-cli-virtual-6b0393a6f7/0/cache/webpack-cli-npm-4.0.0-ccb32dab5d-6c693d4cc9.zip/node_modules/webpack-cli/lib/utils/Compiler.js:91:36)
    at .yarn/$$virtual/webpack-cli-virtual-6b0393a6f7/0/cache/webpack-cli-npm-4.0.0-ccb32dab5d-6c693d4cc9.zip/node_modules/webpack-cli/lib/utils/Compiler.js:110:46
    at .yarn/cache/neo-async-npm-2.6.2-75d6902586-34a8f53091.zip/node_modules/neo-async/async.js:2830:7
    at done (.yarn/cache/neo-async-npm-2.6.2-75d6902586-34a8f53091.zip/node_modules/neo-async/async.js:2865:11)
    at .yarn/cache/neo-async-npm-2.6.2-75d6902586-34a8f53091.zip/node_modules/neo-async/async.js:2818:7
    at .yarn/$$virtual/webpack-virtual-de8dd232be/0/cache/webpack-npm-5.0.0-72465b5b59-bbd9df6389.zip/node_modules/webpack/lib/HookWebpackError.js:69:3
    at Hook.eval [as callAsync] (eval at create (.yarn/cache/tapable-npm-2.0.0-1b1ecde87d-e6321280f9.zip/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:9:1)
    at Hook.CALL_ASYNC_DELEGATE [as _callAsync] (.yarn/cache/tapable-npm-2.0.0-1b1ecde87d-e6321280f9.zip/node_modules/tapable/lib/Hook.js:18:14)
    at Cache.shutdown (.yarn/$$virtual/webpack-virtual-de8dd232be/0/cache/webpack-npm-5.0.0-72465b5b59-bbd9df6389.zip/node_modules/webpack/lib/Cache.js:150:23)

To Reproduce

Hard to say really, but I'll gladly collect and provide any further information when given pointers.

Please paste the results of webpack-cli info here, and mention other relevant information

  System:
    OS: Linux 5.4 Ubuntu 20.04.1 LTS (Focal Fossa)
    CPU: (12) x64 Intel(R) Core(TM) i7-8700K CPU @ 3.70GHz
    Memory: 7.94 GB / 31.29 GB
  Binaries:
    Node: 13.12.0 - /tmp/xfs-eeeb95f6/node
    Yarn: 2.1.1 - /tmp/xfs-eeeb95f6/yarn
    npm: 6.14.4 - ~/.virtualenvs/pool/bin/npm
  Browsers:
    Chrome: 84.0.4147.125
    Firefox: 81.0

[alexander-akait]

Without reproducible steps hard to fix it

[kubijo]

Without reproducible steps hard to fix it

I'm well aware that this not a good bug report and I'm ready to collect as much traces as needed, but I need something to get going.

These are the last two trace locations:

/**
 * @param {CallbackCache<void>} callback signals when the call finishes
 * @returns {void}
 */
endIdle(callback) {
    this.hooks.endIdle.callAsync(
        makeWebpackErrorCallback(callback, "Cache.hooks.endIdle")
    );
}

    compilerCallback(err, stats, lastHash, options, outputOptions) {
        const statsErrors = [];

        if (!outputOptions.watch || err) {
            // Do not keep cache anymore
            this.compiler.purgeInputFileSystem();
        }
        if (err) {
            lastHash = null;
            logger.error(err.stack || err);
            process.exit(1);
        }
        if (!outputOptions.watch && stats.hasErrors()) {
            process.exitCode = 1;
        }
        if (outputOptions.json === true) {
            process.stdout.write(JSON.stringify(stats.toJson(outputOptions), null, 2) + '\n');
        } else if (stats.hash !== lastHash) {
            lastHash = stats.hash;
            if (stats.compilation && stats.compilation.errors.length !== 0) {
                const errors = stats.compilation.errors;
                errors.forEach((statErr) => {
                    const errLoc = statErr.module ? statErr.module.resource : null;
                    statsErrors.push({ name: statErr.message, loc: errLoc });
                });
            }

            //
            // THIS IS THE EXCEPTION LOCATION
            //
            const JSONStats = JSON.stringify(stats.toJson(outputOptions), null, 2);
            if (typeof outputOptions.json === 'string') {
                try {
                    writeFileSync(outputOptions.json, JSONStats);
                    logger.success(`stats are successfully stored as json to ${outputOptions.json}`);
                } catch (err) {
                    logger.error(err);
                }
            }
            return this.generateOutput(outputOptions, stats, statsErrors);
        }
    }

[alexander-akait]

Can you try to use it without PnP?

[kubijo]

Can you try to use it without PnP?

I'm trying to… it doesn't seem to fail on the reported error, but fails differently (some errors are related to my migration to new dependencies versions, some by disabling PnP), so bare with me on this....

[alexander-akait]

I think it is bug on our side, so I want to get better error reporting to search the problem

[kubijo]

I think it is bug on our side, so I want to get better error reporting to search the problem

I think so as well, but the project is pretty big, everything takes a long time & I want to first remove our bugs … so yeah, give me some time and I'll get there ;-)

[kubijo]

The compilation is VERY slow now ... but that may be related to the fact, that now I'm running the production build instead of development watch one (you already have a bug in that reported elsewhere)

[kubijo]

Ok… so it seems the only trailing issues I have now with disabled PnP are

• the CLI output is not colored anymore
• Module build failed (from ./node_modules/mini-css-extract-plugin/dist/loader.js): Error: Didn't get a result from child compiler

Neither of which seem to really be related to this… so I guess that what's to do now is to wait for your linked PR to land & build again with PnP enabled right?

[alexander-akait]

the CLI output is not colored anymore

Already in tracker

Module build failed (from ./node_modules/mini-css-extract-plugin/dist/loader.js): Error: Didn't get a result from child compiler

Do you use latest mini-css-extract-plugin version?

[kubijo]

$ yarn why mini-css-extract-plugin
└─ root-workspace-0b6124@workspace:.
   └─ mini-css-extract-plugin@npm:1.0.0 [dc3fc] (via npm:^1.0.0 [dc3fc])

[alexander-akait]

@kubijo Maybe you can create reproducible test repo? Error: Didn't get a result from child compiler is not related to webpack-cli, I think something wrong in mini-css-extract-plugin options

[alexander-akait]

I think it is fixed in master, but I want more information about mini-css-extract-plugin

[kubijo]

And I'll try to whip-up something in the morning... It's just that that the
project is a complex tangle and it's pretty late here...

[kubijo]

So…

• the good one is that I've somehow managed to make a very minimal setup
• the bad one is that the error is not there :facepalm:

On with the archeology then…

[kubijo]

So I've tried a watch mode to se if there will be a difference and… not sure what is the correct place, so I'll drop it here

Compilation A starting…
Compilation B starting…
Compilation C starting…
Compilation D starting…

(node:38481) [DEP_WEBPACK_WATCH_WITHOUT_CALLBACK] DeprecationWarning: A 'callback' argument need to be provided to the 'webpack(options, callback)' function when the 'watch' option is set. There is no way to handle the 'watch' option without a callback.
    at webpack (node_modules/webpack/lib/webpack.js:145:5)
    at f (node_modules/webpack/lib/index.js:35:15)
    at Compiler.createCompiler (node_modules/webpack-cli/lib/utils/Compiler.js:129:35)
    at WebpackCLI.run (node_modules/webpack-cli/lib/webpack-cli.js:220:32)
    at async runCLI (node_modules/webpack-cli/lib/bootstrap.js:74:24)

Compilation  finished
Compilation  finished

Compilation C starting…
Compilation D starting…

Compilation  finished
Compilation B starting…

Compilation  finished

<--- Last few GCs --->

[38481:0x4bf7290]   305780 ms: Scavenge 3297.3 (4115.0) -> 3295.4 (4130.3) MB, 20.2 / 0.0 ms  (average mu = 0.064, current mu = 0.019) allocation failure 
[38481:0x4bf7290]   306437 ms: Mark-sweep 3310.8 (4130.3) -> 3310.7 (4119.3) MB, 643.2 / 0.0 ms  (average mu = 0.058, current mu = 0.051) allocation failure scavenge might not succeed


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x3040d5c40921 <JSObject>
    0: builtin exit frame: stringify(this=0x3040d5c5efd9 <Object map = 0x11d1f7443641>,2,0x395ddb8c01c1 <null>,0x0053fa3c1341 <Object map = 0x28a43f01f8e1>,0x3040d5c5efd9 <Object map = 0x11d1f7443641>)

    1: compilerCallback [0x19b34e9f2b9] [node_modules/webpack-cli/lib/utils/Compiler.js:91] [bytecode=0x35cd1ccba161 offset=314](this=0x10859d6197d9 <Compiler map = 0...

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0xa0c020 node::Abort() [webpack]
 2: 0xa0c42d node::OnFatalError(char const*, char const*) [webpack]
 3: 0xb8130e v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [webpack]
 4: 0xb81689 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [webpack]
 5: 0xd2fb95  [webpack]
 6: 0xd30226 v8::internal::Heap::RecomputeLimits(v8::internal::GarbageCollector) [webpack]
 7: 0xd3ea99 v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [webpack]
 8: 0xd3f8d5 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [webpack]
 9: 0xd423ac v8::internal::Heap::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [webpack]
10: 0xd07b5d  [webpack]
11: 0xd19abd v8::internal::Factory::NewRawTwoByteString(int, v8::internal::AllocationType) [webpack]
12: 0x10d501b v8::internal::IncrementalStringBuilder::Extend() [webpack]
13: 0xe2753b v8::internal::JsonStringifier::Result v8::internal::JsonStringifier::Serialize_<true>(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8::internal::Object>) [webpack]
14: 0xe2af21 v8::internal::JsonStringifier::Result v8::internal::JsonStringifier::Serialize_<false>(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8::internal::Object>) [webpack]
15: 0xe28970 v8::internal::JsonStringifier::Result v8::internal::JsonStringifier::Serialize_<true>(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8::internal::Object>) [webpack]
16: 0xe28da5 v8::internal::JsonStringifier::SerializeJSReceiverSlow(v8::internal::Handle<v8::internal::JSReceiver>) [webpack]
17: 0xe2a4f2 v8::internal::JsonStringifier::Result v8::internal::JsonStringifier::Serialize_<false>(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8::internal::Object>) [webpack]
18: 0xe2b503 v8::internal::JsonStringifier::SerializeArrayLikeSlow(v8::internal::Handle<v8::internal::JSReceiver>, unsigned int, unsigned int) [webpack]
19: 0xe27a77 v8::internal::JsonStringifier::Result v8::internal::JsonStringifier::Serialize_<true>(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8::internal::Object>) [webpack]
20: 0xe2af21 v8::internal::JsonStringifier::Result v8::internal::JsonStringifier::Serialize_<false>(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8::internal::Object>) [webpack]
21: 0xe28970 v8::internal::JsonStringifier::Result v8::internal::JsonStringifier::Serialize_<true>(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8::internal::Object>) [webpack]
22: 0xe2af21 v8::internal::JsonStringifier::Result v8::internal::JsonStringifier::Serialize_<false>(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8::internal::Object>) [webpack]
23: 0xe2b8d7 v8::internal::JsonStringify(v8::internal::Isolate*, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>) [webpack]
24: 0xc349ab v8::internal::Builtin_JsonStringify(int, unsigned long*, v8::internal::Isolate*) [webpack]
25: 0x13d8c59  [webpack]

not an improvement to say the least (I know that the watch error is aleady being fixed, but I've included it for completness)

[kubijo]

also one nitpick… it seems that the Compilation finished should have a name as well, but there are just two spaces

[alexander-akait]

DeprecationWarning: A 'callback' argument need to be provided to the 'webpack(options, callback)' function when the 'watch' option is set. There is no way to handle the 'watch' option without a callback.

Already reported, WIP

[kubijo]

@evilebottnawi … it seems to me that the problem with css must be either in sass-loader (last one)…

{
    loader: 'sass-loader',
    options: {
        // eslint-disable-next-line global-require
        implementation: require('sass'),
        // webpackImporter: typeof conf.sassWebpackImporter === 'boolean' ? conf.sassWebpackImporter : true,
        sassOptions: {
            // eslint-disable-next-line global-require
            fiber: require('fibers'),
            sourceMap: !isProduction,
        },
    },
}

…or css-loader (one before sass-loader; normally I have a postcss-loader between them, but when I comment it out, the error persists)

{
    loader: 'css-loader',
    options: {
        url: false, // URL calls are already handled by PostCSS
        import: true,
        modules: !withModules
            ? false
            : {
                  localIdentContext: paths.styles,
                  // - In production, short (yet still unique) CSS identifiers are the goal.
                  // - In Developmnet we want the classes to be be readable.
                  localIdentName:
                      isProduction && !inline ? '[md5:hash:base64:8]' : '[name]_[local]_[md5:hash:base64:3]',
              },
        sourceMap: !isProduction,

        // Required because we have postcss-loader pre-processing
        importLoaders: 1,
    },
}

Would you be able to assist me with this somehow?

[alexander-akait]

@kubijo sorry, no, without full reproducible test repo I can't help

[mizchi]

I have same problem with webpack 5.

[webpack-cli] Uncaught exception: RangeError: Invalid string length
[webpack-cli] RangeError: Invalid string length
    at JSON.stringify (<anonymous>:null:null)
    at Compiler.compilerCallback (<my repo root>/node_modules/webpack-cli/lib/utils/Compiler.js:91:36)

that code is here.

// lib/utils/Compiler.js

            const JSONStats = JSON.stringify(stats.toJson(outputOptions)); // here
            if (typeof outputOptions.json === 'string') {
                try {
                    writeFileSync(outputOptions.json, JSONStats);
                    logger.success(`stats are successfully stored as json to ${outputOptions.json}`);
                } catch (err) {
                    logger.error(err);
                }
            }

Turning it off will work by edit this directly.

I have huge size project (total size of chunks have <100MB on prod). I doubt something wrong on stats.toJson() in this case.

[alexander-akait]

Fixed in master, hope release will be today, WIP on other issues

[kubijo]

@kubijo Maybe you can create reproducible test repo? Error: Didn't get a result from child compiler is not related to webpack-cli, I think something wrong in mini-css-extract-plugin options

I've already burnt two days on this and I'm just not able to create a minimal repro, so I'm droping this for now as need to do something usefull for the company…

Thanks for the other fix though and I'll possibly revisit this at some point…

[francescocaveglia]

Thanks for the fix @evilebottnawi ! Will be released soon?

[alexander-akait]

@francescocaveglia yes, today/tomorrow

[alexander-akait]

I want to close this issue, because original problem was solved, anyway @kubijo Can you provide more information to help your migration?