Fd: Add `--ignore-file` flag

[alok]

Basically identical to rg's same flag.

The benefit of this is that sometimes you don't want to ignore every gitignore file found in the search path, but you do want to specifically ignore some files, so this flag comes in handy. Also, if I'm searching from ~, I don't expect the gitignore file several levels down in some project to get picked up by default.

[sharkdp]

Thank you for the feedback.

Basically identical to rg's same flag.

The benefit of this is that sometimes you don't want to ignore every gitignore file found in the search path, but you do want to specifically ignore some files, so this flag comes in handy.

If I understand correctly, rgs flag is used to specify an additional ignore file (with low precedence), so this doesn't solve the use case you descibed(?).

Have you seen fds --exclude option? It allows you to directly ignore a certain pattern:

> fd --exclude '*.o' --exclude '/etc/secret-files' ...

Would this work for you?

Also, if I'm searching from ~, I don't expect the gitignore file several levels down in some project to get picked up by default.

If you don't expect them to be picked up, you can use the --no-ignore option.

[alok]

I think I may not have explained this clearly.

Say I have an ignore file at ~/.ignore and another one at ~/(several levels down)/.gitignore. My working directory is ~.

If I run fd --hidden --type directory ., I expect the files in ~/.ignore to be ignored, but not the ones in ~/(several levels down)/.gitignore. But fd seems to pick up every ignore file in the tree rather than jus the top level one, which is unexpected behavior (for me at least).

--no-ignore would work if I could then specify a specific ignore file, hence the request for the new flag `--ignore-file.

[tmccombs]

But fd seems to pick up every ignore file in the tree rather than jus the top level one, which is unexpected behavior (for me at least)

This is actually the behaviour I expect. If I am in a git repository I would expect fd to behave similar to git ls-files | grep. Usually, if I don't want to include files that are ignored by .gitignore files in their repository.

As @sharkdp mentioned the --include-file option in rg doesn't behave the way you describe. It adds ignore rules _in addition to_ the ignore rules from .ignore and .gitignore files.

[alok]

I'm not in a git repo though. Hmm, I'll test something when I get in to
work tomorrow
On Wed, Nov 1, 2017 at 9:49 PM Thayne McCombs [e-mail]
wrote:

But fd seems to pick up every ignore file in the tree rather than jus the
top level one, which is unexpected behavior (for me at least)

This is actually the behaviour I expect. If I am in a git repository I
would expect fd to behave similar to git ls-files | grep. Usually, if I
don't want to include files that are ignored by .gitignore files in their
repository.

As @sharkdp https://github.com/sharkdp mentioned the --include-file
option in rg doesn't behave the way you describe. It adds ignore rules in
addition to the ignore rules from .ignore and .gitignore files.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/sharkdp/fd/issues/156#issuecomment-341317021, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AH8KTOCyxGHGFCsOikyXt-MiLlMwdc9nks5syUn0gaJpZM4QMFAH
.

>

• Alok

[alok]

My bad, its behavior is identical to rg. I still think the option to explicitly pass a set of ignore files and to only use those files for ignore patterns isn't a bad one, but I understand that's something extra.

[rlue]

I'd like to chime in in support of a command line option like this. ag and rg both honor _both_ .gitignore and a separate .ignore file.

I'm currently using fzf to quick-find files in vim, supported by rg --files as my DEFAULT_FZF_COMMAND. Naturally, I have ripgrep set up to ignore all kinds of binary files (like images) that I would never want to open in vim, but which don't belong in .gitignore.

While I certainly wouldn't want fd to honor my .ignore file all the time (or even by default), it would be nice to have the option to.

[sharkdp]

I'd like to chime in in support of a command line option like this. ag and rg both honor both .gitignore and a separate .ignore file.

fd does that, too.

While I certainly wouldn't want fd to honor my .ignore file all the time (or even by default), it would be nice to have the option to.

Okay, so you would really like to have the --ignore-file flag of ripgrep (with the exact same semantics)? In this case, let's re-open this.

[rlue]

fd does that, too.

So it does! My mistake. I think the bigger issue then is simply that it's not mentioned in the man page or the README.

Okay, so you would really like to have the --ignore-file flag of ripgrep (with the exact same semantics)? In this case, let's re-open this.

Thanks; it'd definitely be nice to be able to rely on predefined ignore files to specify large sets of non-default ignore patterns.

[rlue]

After some more experimentation, it appears that fd also honors .rgignore by default (but not .agignore or .fdignore).

IMHO it would be saner to honor .gitignore and .ignore only, and leave .<xx>ignore files to apply to their respective utilities. If I want my rgignore rules to apply to fd, I'll either put them in .ignore or hardlink .rgignore and .fdignore to each other.

On a separate note, the manpage also contains an error (it says to use --type s for symlinks, where the actual option is --type l).

(If it would save you some trouble, I'd be happy to amend the documentation myself to clarify this point once I have a clear understand of how things work, and once they work the way you intend.)

[sharkdp]

After some more experimentation, it appears that fd also honors .rgignore by default (but not .agignore or .fdignore).

This is due to our usage of the ignore crate, that is also used by ripgrep. This crate has both .ignore and .rgignore hardcoded:

https://github.com/BurntSushi/ripgrep/blob/c4e194538472de2cd74664a9a016b9c25c0e800b/ignore/src/dir.rs#L213

However, if my research is correct, the .rgignore files (as well as the .agignore files for ag) are deprecated and were superseded by .ignore. So .rgignore only exists for legacy reasons and should not be used anymore.

However, you have a fair point, since users might very well want to ignore different files for fd as compared to rg/ag. If .ignore is really meant for search-in-file tools, we should probably not respect these files and introduce .fdignore files instead?

On a separate note, the manpage also contains an error (it says to use --type s for symlinks, where the actual option is --type l).

Thanks, fixed!

(If it would save you some trouble, I'd be happy to amend the documentation myself to clarify this point once I have a clear understand of how things work, and once they work the way you intend.)

That would be great! Any contributions are always very much appreciated.

[rlue]

However, if my research is correct, the .rgignore files (as well as the .agignore files for _ag_) are deprecated and were superseded by .ignore.

Yep, this is definitely true. From @burntsushi himself:

Please use .ignore. .rgignore is deprecated.

---

If .ignore is really meant for search-in-file tools...

I can't say to a certainty what .ignore is for. AFAIK, it's a new convention that has been established by _ag_ and _rg._ In my opinion, it's not a very good one — there are many programs that can be configured with ignorefiles: not just _rg,_ _ag,_ and _fd,_ but git, syncthing, stow, and others. Why should a couple of popular grep utilities get an exemption on namespacing here?

As to how _fd_ should handle this problem, I think the question should really be, _how much deviation from find can you justify in the name of sane defaults?_ IMO, ignoring hidden files and git-ignored files is reasonable, but lumping in _rg_ and _ag_ ignores is pushing it. And I think the remainder of use cases could be reasonably resolved by supporting ~/.fdignore (ideally with an XDG-compatible alternative, like ~/.config/fdignore), which, again, could be hardlinked to ignore if users really want them to be the same.

But you're the boss here, so what do you think?

[BurntSushi]

I'd just like to chime in and say that I'd support simple changes to the ignore crate to iron out this UX, if y'all so choose.

[sharkdp]

IMO, ignoring hidden files and git-ignored files is reasonable, but lumping in rg and ag ignores is pushing it. And I think the remainder of use cases could be reasonably resolved by supporting ~/.fdignore (ideally with an XDG-compatible alternative, like ~/.config/fdignore), which, again, could be hardlinked to ignore if users really want them to be the same.

Agreed. Let's change this. So there are a few action points here:

• [x] Disable support for .ignore and .rgignore. I believe we simply have to remove the .ignore(..) call on the WalkBuilder. We should also carefully review the other options in WalkBuilder.
• [x] Add support for .fdignore. I think we should copy the behavior of ripgrep here (just with .fdignore instead of .ignore).
• [x] Add an --ignore-file option for users that would like to add a specific ignore file (again, in analogy to ripgrep).
• [x] Add integrations tests for this!
• [x] Document the new behavior (man page and README)

[ptzz]

Currently the fd man page says

-I, --no-ignore
              Do not respect any gitignore(5) files.

The description is wrong since it also ignores the .rgignore file.

It would be useful to have an option --no-ignore-vcs specifically targeting git/vcs ignore files (same as ripgrep has).

[ptzz]

Disable support for .ignore and .rgignore. I believe we simply have to remove the .ignore(..) call on the WalkBuilder. We should also carefully review the other options in WalkBuilder.
Add support for .fdignore. I think we should copy the behavior of ripgrep here (just with .fdignore instead of .ignore).

I gave this a try but I think it's hard to get right without modifying the hardcoded .ignore and .rgignorenames in the ignore crate.

My attempt was to add .fdignore as an explicit ignore using add_ignore(), and disabling .ignore/.rgignore using ignore(false). That 1) causes precedence issues (.fdignore lower than .gitignore), and 2) add_ignore() only adds a global ignore file, whereas .ignore is respected in any traversed directory.

I think making the names of the default ignore files (currently static IG_NAMES) configurable in the ignore crate API is the best option. @BurntSushi does this sound like a reasonable change?

[BurntSushi]

@ptzz Please see the latter half of this comment: https://github.com/BurntSushi/ripgrep/issues/673#issuecomment-343673015

TL;DR - While I believe that making IG_NAMES configurable is the shortest path to your destination, I would actually like to keep .ignore as a standard name used by the ignore crate. If we stick to that, then I think the way to solve your problem is to add an additional layer of ignore files that has higher precedence than .ignore but uses a custom (probably application specific) name. For example, ripgrep might use .rgignore where as fd might use .fdignore, but both tools could still respect .ignore if ignore(true) is set.

[ptzz]

@BurntSushi That sounds like a great approach!

Btw., when trying the workarounds above I failed to find a way to respect only the additional ignore file. Using

ignore(false) git_ignore(false) add_ignore(<fdignore_abspath>)

Did not work. But with git_ignore(true), fdignore was also respected. I did however not investigate why due the other reasons this approach would not work. Maybe it’s expected behavior?

[BurntSushi]

If that's true that sounds like a bug to me.

On Nov 28, 2017 2:00 AM, "ptzz" [e-mail] wrote:

@BurntSushi https://github.com/burntsushi That sounds like a great
approach!

Btw., when trying the workarounds above I failed to find a way to respect
only the additional ignore file. Using

ignore(false)
git_ignore(false)
add_ignore()

Did not work. But with git_ignore(true), fdignore was also respected. I
did however not investigate why due the other reasons this approach would
not work. Maybe it’s expected behavior?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/sharkdp/fd/issues/156#issuecomment-347431913, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAb34tHZqFBwcZyGWkzMpQG019DyQ43eks5s669-gaJpZM4QMFAH
.

[BurntSushi]

Thanks to @ptzz's hard work, https://github.com/BurntSushi/ripgrep/pull/706 got merged which will be in the next release of ignore!

[sharkdp]

@BurntSushi Thank you for the update. Looking forward to the next ignore release.

@ptzz Great work!

[sharkdp]

ignore-0.4.0 has been released (thanks!), but it's currently not possible to implement this due to https://github.com/BurntSushi/ripgrep/issues/800

[sharkdp]

Added support for .fdignore via #241.

[sharkdp]

We now have:

• --exclude/-E to exclude specific glob patterns.
• Support for .fdignore files.
• --no-ignore-vcs to show .gitignored files (but not the .fdignored ones).

All this has been documented and we have integration tests.

I currently don't see any need for the additional --ignore-file flag, but if someone has a strong argument for it, I am happy to discuss this again (and reopen the ticket). Let me know what you think!

[ptzz]

I'm currently using fzf to quick-find files in vim, supported by rg --files as my DEFAULT_FZF_COMMAND. Naturally, I have ripgrep set up to ignore all kinds of binary files (like images) that I would never want to open in vim, but which don't belong in .gitignore.

@sharkdp I have a similar setup. With support for --ignore-file, fd could be used instead of rg.

I.e. by default I want fd to find any file type, but it would be useful with a simple way to have it return only textual files.

[sharkdp]

@ptzz I see. That's a valid use case... a .fdignore file would always be present.

I'm reopening this ticket to track the addition of --ignore-file.

[sharkdp]

Implemented in #271

[sharkdp]

Both .fdignore support as well as --ignore-file are part of fd v7.0.0

[mbrubeck]

Just as feedback, I miss having a shared .ignore file that works out of the box with both fd and rg. Now I have to create duplicate files for all my projects, or remember to pass extra arguments every time I call fd. (And when finding across multiple directory trees at once, there's no single argument that will pull in all the .ignore files, which used to happen automatically.)

Maybe someday there could be a configuration file that adds the old behavior as an option...

[BurntSushi]

@mbrubeck Oh, does fd not support .ignore? I might be misremembering, but I thought the plan was for both ripgrep and fd to support .ignore, while also supporting application specific files with higher precedence (e.g., .rgignore and .fdignore, respectively).

[sharkdp]

Oh, does fd not support .ignore?

It does not, anymore. We removed it (and added .fdignore) because some people argued that they would typically put more ignore-patterns into the .ignore file (which is intended for ripgrep-like tools, if I understand correctly) as compared to the .fdignore file. For example, you would maybe .ignore PDF files because you do not want to search through them, but you would still like to be able to find those files.

If I got this wrong or if there is a strong support for adding .ignore support back into fd, I'm happy to discuss this again.

[BurntSushi]

For example, you would maybe .ignore PDF files because you do not want to search through them, but you would still like to be able to find those files.

Right. You could achieve this by putting *.pdf in your .rgignore but not your .fdignore. Alternatively, if fd supported .ignore and .fdignore and *.pdf was in your .ignore, then you could put !*.pdf in your .fdignore, which would override the *.pdf in your .ignore.

I think I thought the idea was that .ignore could be shared across applications, but that the ignore crate now also permits application specific ignore files so that users could share as many rules as they like while also being able to override rules on a per-app basis. It gives the most flexibility, anyway.

The downside I can see is that if someone did treat .ignore as ripgrep specific rules, then they might get frustrated if fd also respected that if they didn't know how to override it (or if they didn't know about .rgignore).

[sharkdp]

I think I thought the idea was that .ignore could be shared across applications, but that the ignore crate now also permits application specific ignore files so that users could share as many rules as they like while also being able to override rules on a per-app basis. It gives the most flexibility, anyway.

If this is the idea, I'm all for it! My previous understanding was that .ignore files were only meant to be used by the "search-in-file tools". Personally, I also share your and @mbrubeck​s opinion and would like to be able to just put my rules into a .ignore file (if it doesn't fit into .gitignore, of course).

The downside I can see is that if someone did treat .ignore as ripgrep specific rules, then they might get frustrated if fd also respected that if they didn't know how to override it (or if they didn't know about .rgignore).

I'm okay with this. We can add a detailed explanation of how .ignore and .rgignore/.fdignore are meant to be used in fds README.

[ptzz]

I also thought .ignore would be shared across applications. My current workaround is to have .fdignore contain what should by ignored by both rg and fd, and alias rg to rg --ignore-file .fdignore. Currently I don't use .ignore at all, only .rgignore and .fdignore.

If .ignore would have been shared, my .fdignore would have been called .ignore and there would be no need for the rg alias.

[sharkdp]

Let's add back support for .ignore then :+1:

[sharkdp]

Support for .ignore has been re-enabled in fd-7.2.0.