Lz4: A heap-buffer-overflow vulnerability in v1.9.1

[cxht]

Hi!
When I run your decompression function of v1.9.1, I use ASAN to run the program and it reports a heap-buffer-overflow bug in lz4.c:1729 memcpy(op, match, 8);

And here is the information printed by ASAN:

==9677==ERROR:AddressSanitizer:heap-buffer-overflow on address 0x7f78cb4f3e21 at pc 0x00000046e1b2 bp 0x7ffc41c17ae0 sp 0x7ffc41c17ad0
READ of size 8 at 0x7f78cb4f3e21 thread T0
0x46e1b1 in LZ4_decompress_generic /home/cx/workspace/target_group/lz4/lib/lz4.c:1729
0x46e1b1 in LZ4_decompress_safe      /home/cx/workspace/target_group/lz4/lib/lz4.c:1990
0x5190e8 in LZ4IO_decodeLegacyStream /home/cx/workspace/target_group/lz4/programs/lz4io.c:925
0x5190e8 in selectDecoder /home/cx/workspace/target_group/lz4/programs/lz4io.c:1123
0x5190e8 in LZ4IO_decompressSrcFile /home/cx/workspace/target_group/lz4/programs/lz4io.c:1168
0x521ea7 in LZ4IO_decompressDstFile /home/cx/workspace/target_group/lz4/programs/lz4io.c:1202
0x521ea7 in LZ4IO_decompressFilename /home/cx/workspace/target_group/lz4/programs/lz4io.c:1223
0x405397 in main /home/cx/workspace/target_group/lz4/programs/lz4cli.c:741
0x7f78cf53a82f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)
0x409178 in _start (/home/cx/workspace/target_group/lz4/lz4+0x409178)
0x7f78cb4f3e21 is located 2527 bytes to the left of 8388608-byte region [0x7f78cb4f4800,0x7f78cbcf4800)
allocated by thread T0 here:
0x7f78cf97c602 in malloc (/usr/lib/x86_64-linux-gnu/libasan.so.2+0x98602)
0x518df6 in LZ4IO_decodeLegacyStream /home/cx/workspace/target_group/lz4/programs/lz4io.c:902
0x518df6 in selectDecoder /home/cx/workspace/target_group/lz4/programs/lz4io.c:1123
0x518df6 in LZ4IO_decompressSrcFile /home/cx/workspace/target_group/lz4/programs/lz4io.c:1168
SUMMARY: AddressSanitizer: heap-buffer-overflow /home/cx/workspace/target_group/lz4/lib/lz4.c:1729 LZ4_decompress_generic

[terrelln]

Can you provide the input that causes this bug?

[cxht]

The command is lz4 -d input.zip out, and the input file(input.zip) is attached below.
input.zip

[Cyan4973]

I've tried to decode this file multiple times with lz4, both v1.9.1 and dev, with ASAN enabled, and I'm unable to generate this issue

compilers tested :

cc -v
Apple LLVM version 10.0.1 (clang-1001.0.46.4)
Target: x86_64-apple-darwin18.6.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

gcc-9 -v
(...)
Thread model: posix
gcc version 9.1.0 (Homebrew GCC 9.1.0)

Some results :

lz4 --list input.zip
    Frames           Type Block  Compressed  Uncompressed     Ratio   Filename
         1    LegacyFrame     -       1.48K             -         -   input.zip

lz4 -v --list input.zip
*** LZ4 command line interface 64-bits v1.9.1, by Yann Collet ***
input.zip(1/1)
     Frame           Type Block Checksum           Compressed         Uncompressed     Ratio
         1    LegacyFrame     -        -           3284387302                    -         -

(note : the compressed size is weird in this mode, there is likely some bug to fix here)

./lz4 -vv -t input.zip
*** LZ4 command line interface 64-bits v1.9.1, by Yann Collet ***
_POSIX_VERSION defined: 200112L
PLATFORM_POSIX_VERSION defined: 200112L
Detected : Legacy format
Error 53 : Decoding Failed ! Corrupted input detected !

./lz4 -vv -d input.zip -f tmp
*** LZ4 command line interface 64-bits v1.9.1, by Yann Collet ***
_POSIX_VERSION defined: 200112L
PLATFORM_POSIX_VERSION defined: 200112L
Detected : Legacy format
Error 53 : Decoding Failed ! Corrupted input detected !

See, decoding fails, but properly, as there is no asan warning anywhere.

However, @terrelln was able to reproduce it.
As we both use macOS, same source files, same compilers, we are currently struggling to understand what's the difference...

It's also notable that input.zip seems to act as a zip bomb, reproducing itself ad infinitum. But this should not matter for lz4 decoder.

[Cyan4973]

Thanks @cxht , this is now reliably reproduced.
We'll get a fix soon,
but as importantly, we want to understand why our current test suite missed it, and make sure to upgrade it.

[cxht]

Thank you!

[terrelln]

FYI this was caused by PR #719, and was not present in any release.

Still, it showed valuable weaknesses in our test infrastructure that we are patching.