Ripgrep: Out of memory

[andschwa]

I managed to break ripgrep somehow.

$ git clone https://github.com/docker/docker.git
$ cd docker

$ rg --ignore-file vendor tar
fatal runtime error: out of memory
^C^C^C^CIllegal instruction (core dumped)

$ rg --version
ripgrep 0.2.8

$ uname -r
4.8.6-1.el7.elrepo.x86_64

This is on CentOS 7 virtualized with Hyper-V.

I have since realized that --ignore-file was not what I wanted (I'm trying to exclude files). It's actual intention is a file which lists what to be ignored. However, by giving it a directory (any directory) I manage to crash ripgrep pretty easily.

[andschwa]

@BurntSushi is it intended that --ignore-file accepts and walks a directory for ignore files? If it is not, I think I can fix it rather simply by denying the use of directories.

[BurntSushi]

This is an awesome bug, thanks for the report!

The issue was that I got a bit overzealous with partial error reporting. Namely, when processing a gitignore file, we should try to use every pattern even if some patterns are invalid globs (e.g., a**b). In the process, I applied the same logic to I/O errors. In this case, it manifest by attempting to read lines from a directory, which appears to yield Results forever, where each Result is an error of the form "you can't read from a directory silly." Since I treated it as a partial error, ripgrep was just spinning and accruing each error in memory, which caused the OOM killer to kick in.

[andschwa]

Yay! This is why I love ripgrep. It's already fixed! Thanks @BurntSushi.