Jszip: Extremely slow performance of generateAsync with 3.2.X JSZip versions

[TestPolygon]

generateAsync works ~50 times slower in JSZip 3.2.X than in JSZip 3.1.5!
I use JSZip in a Chromium 76 based browser.

Here is a code to test it:

function addBlobToZipMultipleTimes(blob, times) {

    console.log("Input blob.size: ", formatBytes(blob.size));

    const zip = new JSZip();

    for(let i = 0; i < times; i++) {
        zip.file("file_" + i, blob);
    }

    console.log("generateAsync...");
    console.time("generateAsync");
    zip.generateAsync({type:"blob"})
        .then(function(blob) {
            console.timeEnd("generateAsync");
            console.log("Result zip.size: ", formatBytes(blob.size));

            // downloadBlob(blob); // some work with the result zip
        });
}

// https://stackoverflow.com/a/18650828/11468937
function formatBytes(bytes, decimals = 2) {
    if (bytes === 0) return '0 Bytes';

    const k = 1024;
    const dm = decimals < 0 ? 0 : decimals;
    const sizes = ['Bytes', 'KB', 'MB', 'GB', 'TB', 'PB', 'EB', 'ZB', 'YB'];

    const i = Math.floor(Math.log(bytes) / Math.log(k));

    return parseFloat((bytes / Math.pow(k, i)).toFixed(dm)) + ' ' + sizes[i];
}

The example

For addBlobToZipMultipleTimes(blob, 1):
JSZIP 3.1.5:

Input blob.size: 2.75 MB
generateAsync...
generateAsync: 22.086181640625ms
Result zip.size: 2.75 MB

JSZIP 3.2.1:

Input blob.size: 2.75 MB
generateAsync...
generateAsync: 926.528076171875ms
Result zip.size: 2.75 MB

JSZIP 3.2.0, 3.2.2 have the same speed.

22 ms vs 926 ms!

---

Some more examples

JSZIP 3.1.5:

generateAsync: 97.862060546875ms
Result zip.size: 13.76 MB

JSZIP 3.2.1:

generateAsync: 4944.34423828125ms
Result zip.size: 13.76 M

97 ms vs 4944 ms

---

For 41.29 MB:

JSZIP 3.1.5:

287.92626953125ms

JSZIP 3.2.1:

14723.44287109375ms

287 ms vs 14723 ms

---

For 137.65 MB:

JSZIP 3.1.5:

858.16796875ms

JSZIP 3.2.1:

49275.451904296875ms

0.8 sec vs 49 sec!

[maximerety]

Hi @TestPolygon,

Based on your code, I added a live demo here: https://jsbin.com/kuxutodini/edit?html,console

From the output of your test, I assumed you were using a 2.75MB blob, added respectively 1, 10 and 50 times to a zip.

I managed to reproduce your performance issue (tested both Chrome 76 and Firefox 69), with version 3.2.5 taking from 10 to 35 times longer than 3.1.5 to generate a zipped output.

I hope this can help debugging the issue.

[tfcardoso]

I used this lib on a personal project today and was ready to give up when I found this issue. With 3.1.5 its quite fast generating one large zip that I need (+/- 10s) and in latest it takes above 10m and thats not an amount of time acceptable to download a file.

This to say that (in my opinion) this issue is a blocker and having a PR prepared, this should be top priority.

[caxton]

I used this lib on a personal project today and was ready to give up when I found this issue. With 3.1.5 its quite fast generating one large zip that I need (+/- 10s) and in latest it takes above 10m and thats not an amount of time acceptable to download a file.

This to say that (in my opinion) this issue is a blocker and having a PR prepared, this should be top priority.

+1

[xqdoo00o]

Hi, @TestPolygon , I think the PR #702 may resolve this issue. Could you plz test with it ?

[TestPolygon]

I can only check the first 2 commits. Using 64k block size is definitely the better thing, even ReadableStream uses such chunk size (or bigger).
And using the properly implemented setimmediate is the must have thing, it's the main reason of the performance issue.

I don't undetstand why the rep owner @Stuk ignores this issue. Maybe someone ping him on other platforms?

[Maxime-J]

Thank you @xqdoo00o !
Performance is way better with your commits
(went from more than 1 min to ~2 s for a huge archive)

[101arrowz]

You might want to consider UZIP or fflate for higher performance since the maintainer of this repo seems to dropped support for it. Check #720 for more info.