Zstd: potential type mismatch on i386 and 32-bit arch

[jrmarino]

Hi Yann,
I noticed something that could be a potential (albeit rare in practice) issue.

The prototype for ZSTD_getDecompressedSize is

unsigned long long ZSTD_getDecompressedSize(const void* src, size_t srcSize);

unsigned long long is always a 64-bit integer IIUC.

The output of this function feeds into dstCapacity argument of ZSTD_decompress typically:

size_t ZSTD_decompress( void* dst, size_t dstCapacity,
                              const void* src, size_t compressedSize);

The issue is that while size_t is a 64-bit unsigned integer on x86_64, it's only a 32-bit unsigned integer on i386. So presumably this effectively limits decompression to 4Gb on 32-bit architectures, after which ZSTD_decompress will likely fail I think. Is my analysis correct?

[jrmarino]

(I understand a normal i386 machine has less than 4Gb memory)

[Cyan4973]

Yes, you are totally correct @jrmarino .

I should add a note that, when decompressed size is too large, it's not possible to decode it "in a single pass" depending on local system limitations, and typically 32-bits systems will be limited.

In which case, it's necessary to use a "streaming API", to progress through the stream step by step.

[jrmarino]

Okay, feel free to close the issue.
I'm working on bindings for the Ada language and Ada caught this. I'll submit a PR to update your documentation when it's done.

[Cyan4973]

Comments clarified within v0.8.0.
Please let me know if this is not enough.

[jrmarino]

Thanks! I think the clarifications are sufficient.