Crates.io: Change the front page "most downloaded" list to "most directly depended upon"

[reem]

I think the latter list is likely to be full of projects which new users are more likely to want to play with, rather than just the "base" dependency packages of the ecosystem.

I think it's important this happens before we get a massive influx of new users from 1.0, who will want to find highly usable libraries rather than just those which are important to the ecosystem (I'm thinking of things like gcc, openssl-sys, and newly libz-sys, which aren't needed directly by >90% of users but are currently on the front page) and I would be happy to work on making it happen if we agree this is desirable.

[tomaka]

I don't think "most directly depended upon" is really more useful.
Libraries which are depended upon are more likely to be building blocks for other libraries. "End-products", for example a high-level framework, would have very few or even no reverse dependencies because it would only be used by private projects that are not published on crates.io.

It would also advantage crates that are split into a core and multiple sub-crates, and put "big bundles" at a disadvantage.

As far as I know, npm determines which packages are the most popular by counting the number of npm install command run on this package. That's probably the best solution, but would require adding this command to Cargo.

[reem]

Hmm, that's a good point. I didn't consider that most uses of a directly-used library will be by applications never uploaded to crates.io. Is there a way we could track "downloads by a direct reverse dependent"? That seems most similar to what @tokama is suggesting.

[alexcrichton]

I think for now that sorting by dependencies may be better than downloads. I think the rough metric we want to shoot for is the "most used" package and it looks like downloads aren't the best way to measure that.

I think for now our best way to measure this metric is "most times written in a Cargo.toml" which is the number of direct dependents. I do find npm's strategy interesting though, so we may wish to do some more research!

[reem]

@alexcrichton Would "number of times written in a Cargo.toml" count stuff never uploaded to crates.io? My concern is that packages which are of use for library authors will be listed in many uploaded Cargo.tomls but packages which are of use to application/executable authors will not.

[alexcrichton]

Ah that is a good point, I hadn't considered crates not on crates.io. Right now we can basically construct a huge dependency graph on crates.io, and I was thinking of sorting by the number of incoming edges to any node (dependencies-on).

If your crate is only depended upon by crates not published though, we wouldn't have that information. We could tweak the interface to downloading though and include information like "why am I downloading this" so we can track it.

[reem]

The highest reward:effort ratio action item here it to just look at stuff on crates.io like you suggested, then possibly do additional work if we don't think the resulting package list is representative.

[Eh2406]

I think that there are 3 kinds of downloads:

1. Real uses:
   
   
   
   • I get a thing on crates.io because I want it, like cargo instal ripgrp
   
   
   • I used it as a dependency in something I am making/installing (not on crates.io).
   
   
2. Dependency uses: I downloaded a thing because it is a dependency of my real need.
3. Automatic uses: A program downloaded a thing because of automated testing. I.E. crator, cargo bomb, docs.rs, and arguably travis.

Now the challenge is to infer the label for each historical download. Not that I have bean thinking about it for a long time, that is not why I want an api for historical download history, or anything. :-0

[carols10cents]

Counting the number of dependencies is an expensive query, so I don't think we're going to do this.

[jhg]

I read this issue now after replying at https://github.com/rust-lang/crates.io/issues/386#issuecomment-751109167 The point of @Eh2406 is interesting and I almost agree.

Sometimes an automatic usage could be also real. The example I told in the other issue; I don't download it because I build something that's my real need and use it as dependnecy using CD. The example could be second sub-item of first point and also third point of what @Eh2406 told.

Maybe the way to know usage is only to ask to each one downloading it each time. But maybe that's annoying for users.