Aws-cli: aws s3 ls --recursive --max-depth

[letalvoj]

Add option --max-depth to limit the depth of recurse.

Let's say that I have a bunch of folders with daily dumps structured as /bucket/source/year/month/day/.... For each day there might be hundreds of files which I am not interested in. I just need to check out for each source which days I have the data available.

Currently I have like 60k of items in the bucket. The --recursive query takes like 40 seconds. By being able to specify the depth both the runtime the number of queries could reduce greatly.

[stealthycoin]

I don't think that would help anything because there aren't really directories in s3 its all just a flat keyspace with prefixes that have slashes in them. So when we do s3 ls --recursive prefix we are really just doing a ListObjects API call and specifying the prefix parameter. So I don't think there is a way to do this without getting the results client side and then throwing away some of the results, which won't save you any time.

[kyprifog]

Could you at least add a limit? In some cases it would be really convenient to not pull down a full directory (even at one level) but instead just pull a sample of it, without having to do multiple calls.