Ripgrep: How to **whitelist** specific file extensions for listing with '--files'

[n00bmind]

Hi.
I'm using ripgrep primarily inside vim/ctrlp.
I'm currently working on a very big repository (tens of thousands of files) with a, let's say "relaxed" policy about what should be in version control and what should not, meaning I have to filter out a lot of binaries, build byproducts and whatnot.
I'm of course using a .ignore file at the root, which helps with speeding up the listings, but I need an additional layer of filtering to weed out so much "noise", so I'm trying to find an adequate syntax to use for the 'ctrlp_user_command' variable in vim which would _whitelist_ just what I'm interested in (only certain types of source files).
So far, I've obtained the best results by using the 'type-add' switch using the curly braces notation to include several file extensions, like this:
rg . --files --type-add "source:*.{h,cs,c}" -tsource

This _almost_ works, I can see most of the files listed have the extensions I specified, but there's also many unwanted files, like these two for instance:
tools\webscarab\src\org\owasp\webscarab\plugin\scripted\script.bsh
tools\win32\python27\Lib\site-packages\data\themes\tools\icons48.code.tga

For the first one I have no explanation. The second one would seem as if the '.code' part at the end was matching my '.c'.. Wild speculation, of course.

Any ideas?

[BurntSushi]

What happens if you try:

rg . --files --type-add "source:*.{h,cs,c}" -tsource --debug

and

$ rg . --files -g '*.{h,cs,c}' --debug

[BurntSushi]

Note that the --debug flag might cause a lot of output to stderr. If you can stick that in a gist or a pastebin, that would be great.

[n00bmind]

I don't feel too comfortable sharing details about this particular project, so I'll try to find a suitable example and provide the info you requested..

[n00bmind]

Ok, I'm using the Android SDK for this..
https://gist.github.com/chopsueysensei/bd0d7908f8d1b628cbb47b33a9b551b5

(I hope you can see all of it, it's pretty long)
I can see that it whitelists several things like images, jars and other things.
I'd say that paths that include a '.' somewhere are causing some trouble..

[BurntSushi]

@chopsueysensei Could you please include the command you ran? Could you also tell me how to clone the repo you're searching?

[BurntSushi]

I need enough information to reproduce the problem.

[n00bmind]

The commands are the ones you asked me to run.
"rg_g_debug.txt" contains all the output from the command rg . --files -g '*.{h,cs,c}' --debug, while "rg_t_debug.txt" contains all the output from rg . --files --type-add "source:*.{h,cs,c}" -tsource --debug.

The tree is just my current Android SDK folder.

[BurntSushi]

The tree is just my current Android SDK folder.

Could you please tell me how to get it?

[n00bmind]

Just install Android SDK anywhere in your HD.. maybe also open "SDK Manager.exe" located in the root folder and download a couple platform versions / optional components to add some more content to it..

[BurntSushi]

@chocolateboy I'm not on Windows. I've never used the Android SDK before. Can you please link me to some instructions on how to acquire it? I need to be able to reproduce your problem.

[n00bmind]

https://developer.android.com/studio/index.html#downloads

However, if you're not on windows, you probably won't get the same output right?

[n00bmind]

Download the one under 'get just the command line tools'.
It should be a matter of unzipping then running.

[cheater]

The android sdk is a red herring. You should be able to just create a file called foo.code.tga which is eg an ascii file with C inside it (for example) and it should be able to instruct rg to not find it.

[cheater]

(by C i mean C source of course)

imo the perfect resolution would be to add something like gnu find syntax for specifying file names. At least -path, -name, -ipath, and -iname as well as -not, -o, -a, -(, and -).

Or at least -ipath and -iname for starters.

[okdana]

• rg's --glob and --iglob are effectively the same thing as find's -name/-path and -iname/-ipath (though the semantics* are slightly different obviously)
• !-prefixed patterns are effectively the same thing as -not
• as in find, multiple glob patterns are combined with an implicit -a by default

* The pattern functionality is mostly as described here: https://git-scm.com/docs/gitignore

[cheater]

Then I guess the issue can be closed as resolved? With the caveat that the original reporter should be able to reopen it if they are unhappy with the features rg provides.

[n00bmind]

Is --glob the same as -g?
In that case, I already tried that as commented in my original post, and it still had some issues. It almost worked, but some files gave false positives..
My intention when opening this was more in the direction of bug catching & fixing.. I since have not used vim again in large codebases, so cannot attest as to how rg behaves currently in that scenario.

[BurntSushi]

I am going to close this because it's not reproducible. If someone can come up with a contained example that uses something more accessible than the entire Android code base, then I can take a look and re-open this.

[alper]

Thte manpage says:

Only search files matching TYPE.

That does not help me to figure out what TYPE can be. I figured out in my case I have to say -tgo but that's fairly counterintuitive.

[BurntSushi]

@alper How is -tgo counter intuitive? Please consider reading the guide's section on filtering with file types.

[alper]

Oh cool. The guide has examples so that makes it a lot easier. The man page does not.

Values concatenated onto the flag is not something I see a lot? I would expect: -t go but that could be just me.

[BurntSushi]

Values concatenated onto the flag is not something I see a lot? I would expect: -t go but that could be just me.

It's standard and idiomatic in UNIX command line since... forever. I don't know precisely when the convention started, but it is specified by POSIX, which likely suggests the convention pre-dated POSIX. So it's probably been around for at least 32 years.

And -t go works as well.