Crates.io: Stats: Ignore downloads made by bots

[pczarn]

Crates are often downloaded by crater, travis and other automatic tools. This doesn't reflect real use. Beginners might be confused by rising download counts on their newly uploaded crates.

It's better to ignore these downloads. Alternatively, display separate stats categories for both kinds of downloads.

[tshepang]

What about automatic dependencies (i.e. those not listed in Cargo.toml)?

[Eh2406]

cc# https://github.com/rust-lang/crates.io/issues/110#issuecomment-281489171

[carols10cents]

I don't know of a way we can reliably detect the difference between a legitimate download and bot downloads. In addition to download numbers inflated by bots, download numbers are likely also deflated by caches. The download number is always going to be inherently fuzzy, so I'm giving this a close.

[pczarn]

To detect the difference, bots could be modified to say they're bots while communicating with crates.io. However, you may be right about caches.

[tshepang]

It is better to have a lower number (caches) than an inflated number (bots), so having bots say they are bots is one thing, but even better would be if cargo can detect if it's being run manually.

[Mubelotix]

It's necessary

[jhg]

Maybe downloads done by bots are legitimate in some cases and "manual" not.

One can automate downloads to inflate #. Also an CD (with cache) can download it and maybe I don't download it manually. I'm not sure if that example is less real than if I do manually. Also other devs download it, or same dev in different machines. It's not 1:1 "crate x.y.z download":"crate using it".

If requests identify itself for not count twice, or more, maybe. Or show it separately. It's only IMHO.

EDIT: Dependent crates # could help to know usage, but only by published crates.