Nugetgallery: Uploading bigger packages fails

[bhuvak]

With the support for native packages, we need to start supporting the upload of bigger packages.
Packages of size up to 150 MB is getting up uploaded properly from commandline. But for package bigger than that nuget.exe gives an error "The request was aborted. "
It works fine from UI.
The stack trace points to
System.IO.InvalidDataException: End of Central Directory record could not be found.
at System.IO.Compression.ZipArchive.ReadEndOfCentralDirectory()

Download huge packages from : https://qa.nugettest.org/packages/hugerimage ( various versions of this will be of size 75, 150 and 220 MBs).

[bhuvak]

Linking to #1301 and updating the title.
While looking at it, make sure #1301 is also fixed.

[falahati]

I have the same problem with a 100MB package. I think problem is not size, it is about upload speed. My upload speed is really low, around 50KBps (512Kbps) so If I want to upload this file I am going to wait about 30Min which is OK with me but probably not for NuGet. It usually close the connection after 5 minutes (300 seconds).
I tried both command line and website and no success.

I believe this is a security thing in IIS to prevent slow uploaders fill all possible connections to server. Yet it is not very acceptable for a uploading site/file gallery.

 PUT https://www.nuget.org/api/v2/package/
 System.Net.WebException: The request was aborted: The request was canceled.
 at System.Net.ConnectStream.InternalWrite(Boolean async, Byte[] buffer, Int32
 offset, Int32 size, AsyncCallback callback, Object state)
 at System.Net.ConnectStream.Write(Byte[] buffer, Int32 offset, Int32 size)
 at NuGet.MultipartWebRequest.CreateMultipartRequest(WebRequest request)
 at NuGet.PackageServer.<>c__DisplayClass6.<PushPackageToServer>b__5(Object sender, WebRequestEventArgs e)
 at System.EventHandler`1.Invoke(Object sender, TEventArgs e)
 at NuGet.HttpClient.RaiseSendingRequest(WebRequest webRequest)
 at NuGet.RequestHelper.GetResponse(Func`1 createRequest, Action`1 prepareRequest, IProxyCache ProxyCache, ICredentialCache credentialCache, ICredentialProvider credentialProvider)
 at NuGet.HttpClient.GetResponse()
 at NuGet.PackageServer.EnsureSuccessfulResponse(HttpClient client, Nullable`1 expectedStatusCode)
 at NuGet.PackageServer.PushPackageToServer(String apiKey, Func`1 packageStreamFactory, Int32 timeout)
 at NuGet.PackageServer.PushPackage(String apiKey, IPackage package, Int32 timeout)
 at NuGet.Commands.PushCommand.PushPackageCore(String source, String apiKey, PackageServer packageServer, String packageToPush, TimeSpan timeout)
 at NuGet.Commands.PushCommand.PushPackage(String packagePath, String source,String apiKey, TimeSpan timeout)
 at NuGet.Commands.PushCommand.ExecuteCommand()
 at NuGet.Commands.Command.Execute()
 at NuGet.Program.Main(String[] args)

[emayevski]

The same problem here - trying to upload 26Mb to default NuGet repository (from browser or from command line) causes the same error "System.Net.WebException: The request was aborted: The request was canceled", just without a call stack.

[jenergm]

Dear all,

I have the same problem using following command line:

C:>"C:Program FilesMSBuildNuGetnuget.exe" push "*.nupkg" username:password -Verbosity Detailed -Source http://ipaddress:8081/artifactory/api/nuget/nuget-repository/nuget-path"

Execution detailed:

Pushing Web.Content 5.5.1602.2201 to 'http://ipaddress:8081/artifactory/api/nuget/nuget-repository/nuget-path'...
PUT http://ipaddress:8081/artifactory/api/nuget/nuget-repository/nuget-path/
System.Net.WebException: The request was aborted: The request was canceled. --->
System.IO.IOException: Cannot close stream until all bytes are written.
at System.Net.ConnectStream.CloseInternal(Boolean internalCall, Boolean aborting)
--- End of inner exception stack trace ---
at System.Net.ConnectStream.CloseInternal(Boolean internalCall, Boolean aborting)
at System.Net.ConnectStream.System.Net.ICloseEx.CloseEx(CloseExState closeState)
at System.Net.ConnectStream.Dispose(Boolean disposing)
at System.IO.Stream.Close()
at System.IO.Stream.Dispose()
at NuGet.MultipartWebRequest.CreateMultipartRequest(WebRequest request)
at NuGet.PackageServer.<>c__DisplayClass4.b__2(Object sender, WebRequestEventArgs e)
at System.EventHandler1.Invoke(Object sender, TEventArgs e) at NuGet.HttpClient.RaiseSendingRequest(WebRequest webRequest) at NuGet.RequestHelper.GetResponse() at NuGet.HttpClient.GetResponse() at NuGet.PackageServer.EnsureSuccessfulResponse(HttpClient client, Nullable1 expectedStatusCode)
at NuGet.PackageServer.PushPackageToServer(String apiKey, Func`1 packageStreamFactory, Int64 packageSize, Int32 timeout, Boolean disableBuffering)
at NuGet.PackageServer.PushPackage(String apiKey, IPackage package, Int64 packageSize, Int32 timeout, Boolean disableBuffering)
at NuGet.CommandLine.PushCommand.PushPackageCore(String source, String apiKey, PackageServer packageServer, String packageToPush, TimeSpan timeout)
at NuGet.CommandLine.PushCommand.PushPackage(String packagePath, String source, String apiKey, TimeSpan timeout)
at NuGet.CommandLine.PushCommand.d__16.MoveNext()

Best regards,

[jenergm]

I think the problem is timeout issue, because closer the server I can put the 18MB file there.

I'm in Brazil sending the package to Spain. Without firewall in the intranet but using a international link. A small file it's possible. Once I could send this file, but most time I can't.

I wish this information can help us correct that.

Thanks a lot.

[maartenba]

Add the -Timeout switch to the command, e.g. -Timeout 4321

[falahati]

@maartenba This would have solve the problem if it was from our side. Unfortunately tho, problem lies at the NuGet servers and probably as a way to prevent R.U.D.Y attacks. Which mean they are not going to fix it any time soon.

[maartenba]

Have you tried it? It should work, the timeout happens in the client.

[jenergm]

Thanks maartenba! That's worked fine now! My problem was the value. I'd put in milisseconds like Timeout of .Net Framework, but the calculus should be in seconds. Now I put "-Timeout 2147483" and worked fine.

[jenergm]

My command line that worked fine!

C:>"C:Program FilesMSBuildNuGetnuget.exe" push "*.nupkg" username:password -Verbosity Detailed -Timeout 2147483 -Source http://ipaddress:8081/artifactory/api/nuget/nuget-repository/nuget-path"

[prestonj]

I'm also having this problem but the suggestion for adding the Timeout switch from @maartenba did not fix the issue for me. Seems there is another timeout on the HTTP PUT request that hits after 100 seconds. My command line is basically:

C:test>nuget push -ApiKey xxx -Timeout 120000 -Source http://nuget.org -Verbosity Detailed mything.nupkg

The output is:
Pushing mything.nupkg to 'http://nuget.org'...
PUT http://nuget.org/api/v2/package/
PUT http://nuget.org/api/v2/package/
PUT http://nuget.org/api/v2/package/
System.AggregateException: One or more errors occurred. ---> System.TimeoutException: The HTTP request to 'PUT http://nuget.org/api/v2/package/' has timed out after 100000ms.
at NuGet.CommandLine.PushCommand.d__16.MoveNext()
--- End of inner exception stack trace ---
at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
at NuGet.CommandLine.Command.Execute()
at NuGet.CommandLine.Program.MainCore(String workingDirectory, String[] args)
---> (Inner Exception #0) System.TimeoutException: The HTTP request to 'PUT http://nuget.org/api/v2/package/' has timed out after 100000ms.
at NuGet.CommandLine.PushCommand.d__16.MoveNext()<---

I'm using nuget.exe version 3.4.3.855

[jenergm]

Hi prestonj,

Use "-Timeout 2147483" as a maximum value, ok? Greater than this won't work.

Best regards,
Jener Garcia Menezes

[prestonj]

Hi Jener,

In the above case I used 120,000 which is less than 2,147,483. I've also tried using 2147483 as well as other more sane value like 900 and 600. Nothing seems to affect the 100 second timeout.

Cheers, Preston

[skofman1]

Seems like a client regression: https://github.com/NuGet/Home/issues/2785

[jenergm]

Hi Preston,

In your environment doesn't have any proxy? It was my another problem with this and we had fixed too.

Best regards,
Menezes, Jener

[prestonj]

Thanks for the suggestion Jener but I've tried the upload from two different locations - same result. I'm pretty sure it's not a proxy issue.

Can someone point me to a version of the nuget client that has a working "Timeout" option?

[jenergm]

Hi Preston,
I'm using "-Timeout 2147483" command with "-" as you can see...

"C:Program FilesMSBuildNuGetnuget.exe" push "E:Folder*.nupkg" UserName:SecurityPassword -Verbosity Detailed -Timeout 2147483 -Source http://host/artifactory/api/nuget/summer.net-nuget-local/summer.net

The NuGet.exe version is 3.3.0.212.

Regards,
Jener

[jenergm]

Hi Preston,

Do not use -ApiKey, use UserName and Encrypted Password as the command I sent you.

Regards,
Jener

[prestonj]

I was able to get this to work (using my previously mentioned command line) by downgrading to nuget 2.8.6 (to get the timeout flag to work) and splitting my packages.

[joelverhagen]

Hey @prestonj, I've applied a fix to address this problem. Could you try this prerelease build of nuget.exe to see if it supports the -Timeout option as expected?
nuget-dev-1296.zip

[kshelley]

@joelverhagen, I was experiencing the same issue and your prerelease version seems to have fixed the problem for me

[joelverhagen]

Closed with https://github.com/NuGet/NuGet.Client/pull/599.

[DarwinJS]

Did this make it into 3.5.0 RC1?

Sure seems like it did not - I still get timeout with 3.5.0.1737 (3.5.0 RC1) downloaded from the downloads page.

However, when I used the binary build attached to this issue, (3.5.0.1296) it worked.

[joelverhagen]

This was a client issue so more up-to-date details are available here:
https://github.com/NuGet/Home/issues/2785

The long and short of it is that there was a regression in this area for 3.5.0-rc1, but it has been fixed in 3.5 (rtm).

[DarwinJS]

Sorry, I didn't realize the regression was actually for 3.5.0 RC1 - had assumed it was for 3.4.3.
Thanks for your follow up!