Lz4: More ASAN issues in the new decompression loop in 1.9.0

[danlark1]

Hi!

We tried to update lz4 in our codebase at Yandex (e.g. here https://github.com/yandex/ClickHouse/issues/5013) and investigated some ASAN issues. We tried the released branch and current branch where some of the ASAN issues were fixed.

We used a simple code like this with argv[1] is the file attached (test_file.txt should be exactly 92 bytes).

#include <lz4.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <cstdlib>

const size_t com_size = 92;
const size_t dec_size = 342;

int main(int, char** argv) {
    char* input = (char*)malloc(com_size);
    int fd = open(argv[1], O_RDONLY);
    read(fd, input, com_size);
    char* out = (char*)malloc(dec_size);
    LZ4_decompress_safe(input, out, com_size, dec_size);
    close(fd);
    free(input);
    free(out);
}

test_file.txt

The actual report is

==780396==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60800000007c at pc 0x00000032fa16 bp 0x7ffd03659e90 sp 0x7ffd03659640
READ of size 16 at 0x60800000007c thread T0
    #0 0x32fa15 in __asan_memcpy asan_interceptors_memintrinsics.cc:23:3
    #1 0x45161a in LZ4_wildCopy32 /place/home/contrib/libs/lz4/lz4.c:358:26
    #2 0x42fa92 in LZ4_decompress_generic /place/home/contrib/libs/lz4/lz4.c:1675:21
    #3 0x42fa92 in LZ4_decompress_safe /place/home/contrib/libs/lz4/lz4.c:1993
    #4 0x362bd9 in main /place/home/test/main.cpp:16:5
    #5 0x7f465deca82f in __libc_start_main /build/glibc-Cl5G7W/glibc-2.23/csu/../csu/libc-start.c:291

[Cyan4973]

Thanks for report @danlark1 ,
it's an important issue, I'll look into it.

[Cyan4973]

Bug confirmed, I can reliably reproduce it.
Onto fix ...

[Cyan4973]

fixed in dev branch

[danlark1]

Great! Thank you.