Fd: -X should batch the number of passed files to the maximum supported by the shell

[lespea]

It appears that if you run getconf ARG_MAX it returns the maximum length that the command string can be. Possibly include a command to artificially limit the number of arguments as well?

$ fd -IH . -tf -X wc -l
[fd error]: Problem while executing command: Argument list too long (os error 7)

[sharkdp]

Thank you for reporting this. That was a known limitation when we first implemented --exec-batch, but we should definitely try to fix this.

Thank you for the information about getconf ARG_MAX. Looks like this should work on all POSIX systems. We will have to check how to get that information on Windows.

[tavianator]

Note that actually counting the size of your arguments is not straightforward: https://github.com/tavianator/bfs/blob/master/exec.c#L61

On Linux at least, you have to count the total length of the command line arguments and environment variables (and auxiliary vector), including NUL terminators, plus the length of the argv/environ pointer arrays themselves (including the final NULL elements). The kernel actually allocates these a page at a time, so you have to round up to the nearest page size. And POSIX recommends you leave an additional 2048 bytes just in case.

This is all finicky enough that I also implemented code in bfs to detect and recover from E2BIG by trying fewer and fewer arguments until it works, just in case the ARG_MAX accounting is wrong or some other platform counts them differently. I don't know if that's feasible in rust.

[Artoria2e5]

It could be easier to implement on Windows, as the accounting only involves the size of the ''lpCommandLine'' parameter. However, it is still non-trivial since the escapes performed by the rust std will increase the size of the command-line. Since we are going to break the std::process::Command abstraction either way, it might make sense to just ask rust-lang to make such a thing available.

[tavianator]

See https://github.com/rust-lang/rust/issues/40384