Exa: Git ignore filter

[jacwah]

Exa already has some options for filtering files. I think filtering out files ignored in .gitignore files, as well as global exclude files, would be a great addition. Output from exa can become littered by uninteresting files like build directory contents or node_modules/. This is especially true when using the --tree option, which can have hundreds of lines of output.

Here is a draft of the help text for the option I'm proposing:

-g, --git-ignore   don't list files ignored by git

Libgit2, which is already used by Exa, has a function for determining whether a files should be excluded or not. See the documentation for status_should_ignore.

What do you think?

[ogham]

If there's a nice easy libgit2 function to do it, then I don't really have much of a reason not do, do I?

The -g option is already taken, but --git-ignore is free.

[ogham]

I tried to use the status_should_ignore function, but I couldn’t get it to work properly. I think that exa is querying the repository for paths that have the wrong prefix. Turns out this isn’t as easy as I thought it would be :(

My attempt is at 277bc0e3b7fbd714f6722d3ece6c0f0eaeb20f2f.

[lilyball]

It might be worth just reimplementing ignore functionality. ripgrep already has this functionality so maybe just look at what it does. In fact, I believe this particular functionality was extracted into the ignore crate.

[ogham]

Oh, I didn't know it was in its own crate! That's going to make things much easier.

[hoodie]

there is a crate for everything :D

[ogham]

Unfortunately it wasn't as easy to integrate the ignore crate as I hoped; it has its own DirWalker that gathers up all the ignore rules and automatically applies them for you, but exa has its own directory-walking implementation already, and I'd have to combine the two somehow. So I'm bumping it a release to give me more time.

Its other crate, globset, looks worth using instead of the "vector of glob patterns" exa currently has, especially if it's going to eventually use the ignore crate anyway.

[jacwah]

@ogham I remember having problems with status_should_ignore as well, so I looked through my code and found this comment: https://github.com/jacwah/oak/blob/master/src/filters.rs#L165. Maybe you are experiencing the same issue?

My fix was to call path.canonicalize before passing it to libgit2, because it seems to be very picky about how the path is formatted.

[ogham]

@jacwah Quite possibly — thanks for the tip. exa doesn't call canonicalize anywhere, but it does get the repository's working directory and tacks the relative paths onto the end of that before comparing them, which seems like it would result in an absolute path... which, yeah, it wouldn't accept as being the same path.

[jacwah]

@ogham I found the explanation. The problem is probably that the paths anchored at ./ -- it should work if you just remove this prefix.

You cannot add ., .. or .git to a git repository, they are not valid repository paths. Thus, these paths are ignored when trying to add them.

Do recall that the path is rooted at the repo's workdir. ./ignore.c is not a valid path in the repository, it's a working directory path.

https://www.bountysource.com/issues/38818799-status_should_ignore-returns-true-for-any-path-starting-with

[wldcordeiro]

I feel like the --git option should automatically ignore the things in .gitignore and other excludes.

[hoodie]

another proposition:
at least the --tree option should automatically obey ignore files

[lilyball]

I strongly disagree. Git ignore rules should always be an opt-in behavior. It would be incredibly surprising to me for a tool that lists files to silently skip a bunch of files just because there happens to be a file named .gitignore in the current directory.

[wldcordeiro]

Let me modify my original opinion. I think files and dirs that are in .gitignore/.othervcsignore should be displayed in a way that makes them distinct from not ignored content.

[ogham]

So I've been working on this on-and-off for the past two days, as it's the last real issue I wanted to get done before 0.8.0, and I have something that, well, 90% works!

In the ignoreds-again branch (which I'm going to merge after giving it a look-over on the train home) I've added code that checks .gitignore, turns it into globs, and ignores files that match those globs. Thing is, I'm a little worried about having such a different implementation:

• I considered using the ignore crate, which is fantastic, but unfortunately it does too many things that exa does already. However, its sister crate globset looks like a more featureful replacement for glob.
• I was going to use the git2 crate's ignore functionality, but our Repository values don't stick around that long, and it wouldn't work with .ignore files later.
• My current code doesn't seem to work with globs beginning with a /, so that's something already.

PS: I'm going to stick with not obeying ignore files by default, even with --git, because it adds a lot of extra stat calls.

[benzvan]

Would it be possible to add some kind of visual indication that a file is in gitignore? Or would that be a different issue? (didn't want to open a new issue for something so closely related)

[ghost]

I noticed that it doesn't ignore files in the global gitignore set by core.excludesfile

[boojum]

I'm hesitant to open a bug report about --git-ignore as it's marked for v0.9.0, but perhaps it should be removed as a option from v0.8.0? I was might confused as to why it is not working before I turn to github.
Explanation for various git states (N, M etc etc) in the manpage would be nice as well :-)

[kbd]

I noticed that it doesn't ignore files in the global gitignore set by core.excludesfile

Came here to report this. Was wondering why exa wasn't respecting my .gitignore. I noticed it just looks for files named ".gitignore" and doesn't respect git's global ignore settings. FWIW, GitHub recommends naming your global ignore file .gitignore_global, which is what I've personally followed.

[lilyball]

Why the heck does GitHub recommend that? The default of ~/.config/git/ignore is so much better.

[kbd]

Why the heck does GitHub recommend that? The default of ~/.config/git/ignore is so much better.

Great question! I agree with you. It's even in git's documentation. I just set mine up that way years ago at GitHub's recommendation and have left it since. I appreciate you pointing out that git respects the xdg_config standard; I'll switch to that when I get a chance.

So, to correctly respect git's ignores, exa needs to respect repo ignores and global ignores (accounting for core.excludesfile), and finally use the default global configuration locations of [$XDG_CONFIG_HOME | $HOME/.config]/git/ignore, etc.

[tueda]

As of v0.8.0, when both the --git-ignore and --all options are used, exa shows dot files but does not show Git-ignored files. This sounds more or less reasonable, but I would pefer that when both the options are used exa ignores --git-ignore and shows all the files. This is because, to me, Git-ignored files are similar to dot files in the sense that they should be hidden. So, my use case would be to hide Git-ignored files by default via alias exa='exa --git-ignored' and want exa -a to show all the files.

Maybe adding another option meaning like --git-ignore-unless-all-is-specified would be OK for this purpose (this option name is not good, though).

[lilyball]

If possible, I'd err on the side of respecting whichever flag came later in the arguments list. That means exa --git-ignored -a would show ignored files, and exa -a --git-ignored would not.

[PicoSushi]

Does --git-ignore work with exa v0.9.0? It seems not work at all...

$ exa --version
exa v0.9.0
$ touch /tmp/some.log
$ ls /tmp
some.log
$ exa --git-ignore /tmp
some.log

note: I have *.log line in my $HOME/.gitignore file...

[lilyball]

$HOME/.gitignore doesn't affect anything.

That said, testing right now, it looks like exa only obeys .gitignore in the listed directory, rather than following git's actual logic for this. It's also not applying the patterns the same way as git does when doing a recursive listing (e.g. a will only hide a file a in the current directory and not in any nested directory when doing a tree listing).

[schroederc]

Can exa have an additional flag (like rg --ignore-file=<path>) to include additional ignore files? That would help my specific use-case and might help those that want to use global ignore files.