Fd: Add option: stop search after 'n' hits

[warpsprung]

Would make it very useful e.g. in scripts

[sharkdp]

Hi, thank you for your feedback.

You can use head to achieve this:

> fd test | head -n 20

[warpsprung]

Using head will not stop fd from further trying to find something. In a script it's important that this search is quick and fast (and immediately over once it found what it was searching for).

[sharkdp]

Using head will not stop fd from further trying to find something

Actually, it will stop fd. Once the pipe is broken, fd will exit.

There is one special thing to consider for fd: It actually buffers results for a certain time until it will start streaming results to the console (by default, 100 ms). You can disable this using:

> fd --max-buffer-time 0 test

Check the timings:

Without using head, it takes a long time (5 s):

> time fd test / > /dev/null
fd test / > /dev/null  3,90s user 3,78s system 152% cpu 5,041 total

Using head, it finishes early. It takes around 100 ms due to the buffering:

> time fd test / | head -n 10 > /dev/null                      
head -n 10 > /dev/null  0,00s user 0,00s system 4% cpu 0,107 total

With the buffering disabled, it finishes almost immediately:

> time fd --max-buffer-time 0 test / | head -n 10 > /dev/null 
head -n 10 > /dev/null  0,00s user 0,00s system 22% cpu 0,019 total

[warpsprung]

Maybe Windows 10 is causing some confusion here, with this OS 'head' doesn't stop 'fd' (at least not the head command that comes with a msys installation). So at least for the Windows platform such an option would make sense.
Also 'fd' on Windows might still need some optimizations as the builtin 'where' command takes 44s on my 512 GB SSD to search for certain filenames while 'fd' takes 55 seconds. But 'fd' is much faster finding the first hits.
Another thing b.t.w. is that colored output on the Windows command line seems to have a problem:
←[1;34m←[0m←[1;34mC:←[0m←[1;34m\←[0m←[1;34m\←[0m←[1;34m\←[0m←[1;34mWindows←[0m←[1;34m\←[0m←[1;34mSystem32←[0m←[1;34m\←[0mcmd.exe

[sharkdp]

Maybe Windows 10 is causing some confusion here, with this OS 'head' doesn't stop 'fd' (at least not the head command that comes with a msys installation). So at least for the Windows platform such an option would make sense.

I see, thank you for the clarification. Could you try PowerShell's select command? I.e., something like

fd.exe test | select -first 1

Also 'fd' might still need some optimizations as the builtin 'where' command takes 44s on my 512 GB SSD to search for certain filenames while 'fd' takes 55 seconds. But 'fd' is much faster finding the first hits.

I'm not exactly sure how where.exe works (if it maybe has access to some search index?), but it looks like it does glob-matching instead of regex searches. Could you show the two commands that you used to benchmark? Also, did you run them once or more often?

Another thing b.t.w. is that colored output on the Windows command line seems to have a problem:
←[1;34m←[0m←[1;34mC:←[0m←[1;34m\←[0m←[1;34m\←[0m←[1;34m\←[0m←[1;34mWindows←[0m←[1;34m\←[0m←[1;34mSystem32←[0m←[1;34m\←[0mcmd.exe

It should work on PowerShell (starting from Windows 10), but it currently does not work on cmd.exe and older versions of PowerShell, see #29.

[warpsprung]

I can confirm select -first 1 works in PowerShell. The colored output doesn't (PSVersion 5.0.10240.17609).
Within my cmd shell I used these two commands:
where /R C:\ cmd.exe
fd -t f cmd.exe C:\

[danielpclark]

Actually, it will stop fd. Once the pipe is broken, fd will exit.

Wow. Can I hop in on this question and ask if that's a feature implemented from heads side or fd? This intrigues me. If it's fd where might I see the code that does that.

[sharkdp]

Actually, it will stop fd. Once the pipe is broken, fd will exit.

Wow. Can I hop in on this question and ask if that's a feature implemented from heads side or fd? This intrigues me. If it's fd where might I see the code that does that.

Sure!

For the most part, this is a feature of unix pipes. head closes its stdin stream once it has read n lines. This, in turn, closes the stdout stream of fd. So all we have to do is to detect the closed output pipe. The relevant code is here:
https://github.com/sharkdp/fd/blob/8fc3a83d928f43ce37750ab3ff3f1433fc5dd6a1/src/output.rs#L103-L108

[sharkdp]

I'm going to close this, as it seems this can be solved by using head on Unix and select on Windows.

For the colored output on Windows, we should open a new ticket. In PowerShell, it works for me with a recent version of Windows 10 (see screenshot in #70).

[alains-trend]

The "head" solution does not work when you want to use fd -print0 with xargs -0
Should I be opening a separate new issue?

[sharkdp]

In the meantime, fd has a --max-results N and a -1 option.