Nugetgallery: [Speculative bug] Package.DownloadCount is going to eventually overflow

[pranavkm]

Describe the bug

We were admiring Json.NET's almost 1 billion download count today. I was curious about the data type used to store this value in the gallery. It looks like Package.DownloadCount is a int32 value and will eventually overflow to negative values. Might be prudent to update that property and everything that touches it to use a 64-bit integer.

[JamesNK]

Newtonsoft.Json is getting 74 million downloads every 6 weeks. If it stays constant, in about 80 weeks it will break Int32.MaxValue (2.1 billion)

[loic-sharma]

Thanks for reporting this! It looks like a few areas are affected:

Gallery database

https://github.com/NuGet/NuGetGallery/blob/be38792c3d3d327765113a32d82cd5b9b68cd1b0/src/NuGet.Services.Entities/Package.cs#L60

https://github.com/NuGet/NuGetGallery/blob/be38792c3d3d327765113a32d82cd5b9b68cd1b0/src/NuGet.Services.Entities/PackageRegistration.cs#L26

Statistics database

https://github.com/NuGet/NuGet.Jobs/blob/376ac06e6e07d4ee8d1e28f6b2346e3891487496/src/Stats.Warehouse/Tables/dbo.Fact_Download.sql#L12

https://github.com/NuGet/NuGet.Jobs/blob/376ac06e6e07d4ee8d1e28f6b2346e3891487496/src/Stats.Warehouse/Tables/dbo.Fact_Dist_Download.sql#L9

https://github.com/NuGet/NuGet.Jobs/blob/376ac06e6e07d4ee8d1e28f6b2346e3891487496/src/Stats.Warehouse/Programmability/Stored%20Procedures/dbo.RollUpDownloadFacts.sql#L17

https://github.com/NuGet/NuGet.Jobs/blob/376ac06e6e07d4ee8d1e28f6b2346e3891487496/src/Stats.AggregateCdnDownloadsInGallery/AggregateCdnDownloadsJob.cs#L38

V2 Protocol

This needs verification, but the V2 protocol may be affected as well: https://joelverhagen.github.io/NuGetUndocs/#package-entity

[JamesNK]

I think it is important to test what happens to clients. There will likely be some older clients out there that are no longer updated. What happens when they get int32.maxvalue+1?

[joelverhagen]

Thanks @pranavkm and @JamesNK for bringing this to our attention.

We've been loosely following the numbers and have fixed some problems in this area already. For example it's possible that our front page total download count stopped updating one time suspiciously close to int.MaxValue several years back 🙄. But a full analysis should be done since this will be the first time a _single_ package exceeds int.MaxValue.

I think overflowing to negative is a better case than, you know, VS UI crashing 😨. We need to dig through server and client side.

Thanks @loic-sharma for the initial analysis!

[ymwymw]

Just a suggestion, maybe manually move the download count down as it's moving up until you know everything is fixed. or have it as a backup plan on D-Day.