Nugetgallery: Make "Validation" Faster

[bchavez]

Previously, publishing on NuGet was easy-peasy and fast.

Now I have to wait over an hour for my "nuget" package to publish. The consequence is now others, and I, have to stop working on an issue for at least a few hours until my NuGet package is published.

• Why does it take so long?
• Can you please make this faster?

This new validation is annoying and irritably getting in the way of being productive.

Also, this new janky "validation" gets worse, sometimes validation doesn't complete. I unlisted v21.0.1 few days ago because it took too long and never completed in a timely manner. I thought maybe something got stuck in your validation pipeline. So I had to republish the same exact package with a different version number 21.0.2.

Again, the overall goal should be to make NuGet more accessible, not less.

My NuGet package is Bogus listed here: https://www.nuget.org/packages/Bogus/

Thanks,
Brian

[jskeet]

4 hours 13 minutes and counting here :(

[chrisdunelm]

This is causing us significant pain.

We've just published over 100 Google.Apis.* packages at v1.32.0 (e.g. Google.Apis.Storage.v1), but they all depend on Google.Apis v1.32.0, Google.Apis.Auth v1.32.0 and Google.Apis.Core v1.32.0, all three of which are still validating.

Please resolve this as soon as possible.

[chenriksson]

We are currently experiencing high latency in validating and indexing new packages pushed to NuGet.org. We will post an update to https://status.nuget.org when the issue has been resolved. Thank you for your patience.

[chenriksson]

The issue causing the high latency has been resolved, and an update posted to https://status.nuget.org.

[chrisdunelm]

Thanks for the update :)

[bchavez]

Fantastic. Published Bogus v21.0.5 and only took 10 minutes for validation. Feels like 10 minutes is a good maximum threshold. :ok_hand:

Also, the Bogus v21.0.1 validation from 6 days ago finally validated. :checkered_flag: :turtle:

Thanks a bunch @chenriksson for fixing the problem and making NuGet work again. :+1:

[dazinator]

Over 2 hours and still waiting. https://status.nuget.org/ says everything is fine?

[fractos]

Over 3 hours for a few packages :( Might as well go to bed.

[UweKeim]

Just found this thread, cause I was wondering how long it would take. Took 8 Minutes for my small package.

Maybe the size of the package has influence on the duration, too.

Update 1:

When trying to download from within Visual Studio 2017's Package Manager, it lists my package, but still says:

Package 'ZetaLongPaths 1.0.0.24' is not found in the following primary source(s): 'https://api.nuget.org/v3/index.json'. Please verify all your online package sources are available (OR) package id, version are specified correctly.

So this seems to take longer.

Update 2:

Did a restart of Visual Studio 2017, now I was able to install.

Maybe some caching involved, or I've waited long enough.

[loic-sharma]

@UweKeim That issue you ran into is after validation has completed. There's some lag between NuGet's v2 and v3 pipelines, it sounds like you searched for your package using v2 but then tried to restore the package using v3 before it had been indexed.

There are many factors that influence how long validations take, and the size of a package is indeed one of them. We're working on improving our validation pipeline's speed.

[mwpowellhtx]

What is the issue with Validating that it is taking so long? Also, to be clear, I'm not sure how anyone can "up vote" this taking longer. Validation should be improved.

[anangaur]

@mwpowellhtx We are working on making the validation faster. You can read up about what happens during validations and work that we are doing in this regard in this blog post published earlier this year. We have already made some progress to reduce the validation time.
Which packages did you see long validation times for and what was the duration for the validation delays roughly?

[mwpowellhtx]

@anangaur You're missing the point; validation is hanging up to the point of failure. This never used to happen before.

[anangaur]

@mwpowellhtx We understand the pain. The key change from the times this was not a problem is that we have made the essential validations like malware scanning a blocking operation to publish a package. We are trying to make this step as fast as possible.

What is an acceptable delay beyond which you term it a failure, according to you?

[mwpowellhtx]

@anangaur I'm not sure you do understand; whatever was pushed into production broke validation to the point of failure. If you are going to experiment, then kindly do so offline in a test server. Now my packages are not publishing, validation is timing out, and for no fault of my own.

[anangaur]

I just checked with the team and it seems we had a live site incident causing the validation delays leading to validation failures. We have fixed the problem now and also initiated re-flow for all the packages to validate again. Expect this to get fixed soon.

/cc: @skofman1

[mwpowellhtx]

@anangaur @skofman1 Appreciate y'all looking into that; or if it was a different specific issue.

Incidentally, at least as far as I was concern, if the "time" can be improved great, but the time wasn't that big a deal for me before, really and truly. It didn't take hours, and it didn't stop due to failure, etc, to be clear.

[bsoja]

I have a really small library (https://www.nuget.org/packages/PolishVehicleRegistrationCertificateDecoder) and I encountered the same problem. There is still "Validating" status for 3 days now for v1.0.1. For the other versions v1.0.2 / v1.0.3 validation time was less than 5 minutes. Now I just uploaded v1.0.4 which still is "Validating" for half an hour now. Why is it taking so much time sometimes and is my v1.0.1 will be finally validated?

[anangaur]

/cc: @skofman1 1.0.1 is still Validating.

[bsoja]

@anangaur @skofman1 I see that after your response v1.0.1 was finally published. Can you do the same for the newest one (v1.0.5)? This version is in Validating stage for 12h now and probably is locked the same way v1.0.1 was.
EDIT: Thank you, all versions are validated now.

[apobekiaris]

not fixed for me, all my build depend on this the VersionConverter package :(

https://www.nuget.org/packages/Xpand.VersionConverter/