Github-readme-stats: [BREAKING CHANGE] New Top Language Detection Method

[anuraghazra]

As you all might know there are various bugs/issues regarding the top languages calculation.

The problem

The main issue i see is that people often get confused by how the calculations are done.

Currently the top languages are calculated based on how much code in bytes you have in a particular language and then we choose the top languages.
This method is the main reason people are confused about the calculations, because normally users perceive how much they code in languages by how many repositories they have with that particular language.

Quirks with the current calculation method

• Confusion about the calculations. (https://github.com/anuraghazra/github-readme-stats/issues/136#issuecomment-665172181)
• Repositories might have vendor code or auto generated code which would make the calculations wrong. (#153)
• If some language have exaggerated code bytes then it becomes the dominant language. (https://github.com/anuraghazra/github-readme-stats/issues/358#issuecomment-673327755)
• Users aren't satisfied with the method.

The Solution

The most straight forward solution I see is that instead of calculating how much code they have, we can calculate how many repositories they have with the languages.

Related issues

432 #403 #270 #136 #358

[saurabhdaware]

Not sure but the current methods seems right to me. Imagine having 2 HTML, JS repositories with more 51% HTML in both.

According to the new proposed method, I wouldn't know JavaScript.

So in my opinion, the current method makes more sense.

[anuraghazra]

@saurabhdaware We would not just take the primary language of the individual repo, we would also calculate top 10 langs of individual repos too.

That is we are already doing :-

https://github.com/anuraghazra/github-readme-stats/blob/6e73a0035dfebf7411f4763b7ce49df2f8737e36/src/fetchers/top-languages-fetcher.js#L14

So it would look like -:

HTML ---------- some%
Javascript ------ some%

[saurabhdaware]

I am not sure if I understood. Wouldn't calculating top 10 languages of each repository same as calculating how much code the user has in bytes? that's how GitHub calculates the percentage as well no?

[anuraghazra]

No we would just "count" them and in current method we get the "language.size" reduce and sum it up and then sort it.

[saurabhdaware]

Oh ok so in this example

Imagine having 2 HTML, JS repositories with more 51% HTML in both.

I would have
51% HTML
49% JavaScript
right?

[anuraghazra]

Repo 1 - JS x1 & HTML x1
Repo 2 - JS x1 & HTML x1

HTML - 50%
JS - 50%

We would just count them.

[saurabhdaware]

Oh cool. Seems good to me then.

[DenverCoder1]

I think I like this new way better. I did one very large C# project earlier this year and now it says C# is my top language (over 52%) even though it is definitely _not_ what I do the most of.

[DenverCoder1]

If there are people that are more happy with the current way, there could possibly be an additional parameter that will switch between the different calculation modes?

[anuraghazra]

If there are people that are more happy with the current way, there could possibly be an additional parameter that will switch between the different calculation modes?

Unfortunately we cannot do that, it would make the logic complex & we would have two different statistics. it would hamper the consistency.

[anuraghazra]

I will firstly publish experimental query param to enable this and then if people likes it i would make it default.

[Bas950]

I personally wouldnt use this new one, I think the current one is better.

I think I like this new way better. I did one very large C# project earlier this year and now it says C# is my top language (over 52%) even though it is definitely _not_ what I do the most of.

You could exclude the language, but in my PR (https://github.com/anuraghazra/github-readme-stats/pull/480) I am making it so you can just exclude a repo. Which is probs better.

[anuraghazra]

@Bas950 yeah I can understand what you are saying, but the main reason why soo many people uses github-readme-stats is because of it's simplicity and ease of use. Of course we can add "exclude_repo" options and make it better but the thing is that not many people have the time/patience to go through all of their repositories and check which one has some vendor code and exclude them one by one, not to mention this is impractical for users who have lot of repos.

So this is why i'm considering this new approach which would mitigate these issues.

[crazy-max]

@anuraghazra Check https://github.com/github/linguist

[mchelen-gov]

Quirks with the current calculation method

• Confusion about the calculations. (#136 (comment))
• Repositories might have vendor code or auto generated code which would make the calculations wrong. (#153)
• If some language have exaggerated code bytes then it becomes the dominant language. (#358 (comment))
• Users aren't satisfied with the method.

@anuraghazra Does the current method only look at repos owned by the user or does it include other repos the user has contributed to?

[MehdiRazaBukhari]

Consider i have five repositories.

1) C++ 50k line Machine Learning Library i wrote.
2) html page with hello
3) html page with hello
4) html page with hello
5) html page with hello

Now repo approcah should give my top language as HTML! right??

[anuraghazra]

Consider i have five repositories.

1. C++ 50k line Machine Learning Library i wrote.
2. html page with hello
3. html page with hello
4. html page with hello
5. html page with hello

Now repo approcah should give my top language as HTML! right??

[Bas950]

Quirks with the current calculation method

• Confusion about the calculations. (#136 (comment))
• Repositories might have vendor code or auto generated code which would make the calculations wrong. (#153)
• If some language have exaggerated code bytes then it becomes the dominant language. (#358 (comment))
• Users aren't satisfied with the method.

@anuraghazra Does the current method only look at repos owned by the user or does it include other repos the user has contributed to?

I will be add a feature soon in PR tho that will allow you to opt-in to forks or opt-out of using forks, depends of what @anuraghazra wants to use.

[Dev-CasperTheGhost]

Kinda off-topic question: Does the stats-card calculate private repos aswell? Thankyou.

[Vicellken]

Kinda off-topic question: Does the stats-card calculate private repos aswell? Thankyou.

looking forward to this feature as well!

[BenStigsen]

Will this make it support other languages, or should I make a seperate PR for this? Currently I can't see GDScript on my stats.

[MehdiRazaBukhari]

Don't think so.

On Wed, 21 Oct 2020, 6:33 PM Benjamin Stigsen, [e-mail]
wrote:

Will this make it support other languages, or should I make a seperate PR
for this? Currently I can't see GDScript on my stats.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/anuraghazra/github-readme-stats/issues/481#issuecomment-713577455,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AL7VWCKYDQVDU4QPHEUD2U3SL3PJDANCNFSM4RTPUQVQ
.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.