Home: NuGet Restore/Update does not work when unrequired package sources are unavailable

[deadlydog]

Details about Problem

NuGet product used (NuGet.exe | VS UI | Package Manager Console | dotnet.exe): VS UI

NuGet version (x.x.x.xxx): NuGet Package Manager 4.5.0

VS version (if appropriate): VS 2017 (v15.5.2)

OS version (i.e. win10 v1607 (14393.321)): Win 10 (v10.0.16299 Build 16299)

Worked before? If so, with which NuGet version: No

Detailed repro steps so we can see the same problem

1. Add an additional package source that can only be accessed from the local network (or when connected via VPN).

2. Open up a solution in Visual Studio that only uses packages from nuget.org (not from the LAN package source).

3. Attempt to restore packages, or open the NuGet Package Manager UI with "nuget.org" as the "Package source" and attempt to update the packages, when the LAN package source is not available.

4. You will receive an error like:

Exception 'System.AggregateException' thrown when trying to add source 'https://packages.mydomain.com/nuget/nuget'. Please verify all your online package sources are available.

So even though the packages that you are trying to update/restore are not on the alternate package source, NuGet still wants that package source to be available before continuing. This means you either need to get on the LAN that has the package source (or connect to the VPN), or you need to go disable/remove that package source from your list of package sources.

This happens both when using the NuGet Package Manager UI and the Package Manager Console.

Other suggested things

I think a better approach would be to try and restore/update all of the packages, and if they can't be found report an error and have that error mention that it could not connect to the alternative package source, and that may be the source of the problem.

This is a problem for many people I work with, as we have an internal corporate nuget package source. Anytime we work remotely or from home, even on projects that don't use packages from our corporate source, we need to connect to the VPN in order to restore/update packages.

Verbose Logs

PM> Update-Package -Verbose
  GET https://api.nuget.org/v3/registration3-gz-semver2/microsoft.azure.mobile.analytics/index.json
  OK https://api.nuget.org/v3/registration3-gz-semver2/microsoft.azure.mobile.analytics/index.json 436ms
Update-Package : Unable to load the service index for source https://packages.mydomain.com/nuget/nuget.
  An error occurred while sending the request.
  The remote name could not be resolved: 'packages.mydomain.com'
At line:1 char:1
+ Update-Package -Verbose
+ ~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Update-Package], Exception
    + FullyQualifiedErrorId : NuGetCmdletUnhandledException,NuGet.PackageManagement.PowerShellCmdlets.UpdatePackageCommand

Time Elapsed: 00:00:00.6385733

[StingyJack]

I've logged this a few times and it gets closed with reason "working as designed".

https://developercommunity.visualstudio.com/content/problem/69961/manage-nuget-packages-for-solution-is-not-paying-a.html

https://github.com/NuGet/docs.microsoft.com-nuget/issues/407

https://developercommunity.visualstudio.com/content/problem/118566/vs-is-failing-to-install-a-nuget-package-from-nuge.html

[zhili1208]

this is by design, nuget restore hard fails when any enabled source is not available, we don't want to restore inconsistently based on which source is available. Please disable unused source before restore.

[StingyJack]

@zhili1208 - please explain how a restore can in any way be inconsistent in the scenario described above.

[StingyJack]

@zhili1208 - if I try to push a package to a nuget.server and that package has dependencies, but the server does not have those dependencies yet, will the push succeed?

[rlundy]

@zhili1208 This continues to baffle me. It baffles me, in fact, every day when VS fails to install a package or fails to update a package or fails to create a flippin' controller in an ASP.NET Core MVC project because a NuGet package source that _I'm not using and don't care about_ is unavailable.

Instead of continually closing this as "by design," what about a little introspection? What about examining a few alternatives? This keeps getting opened by people because it's a constant irritation. Essentially NuGet is deciding that it's smarter than we developers are, and that it must protect us from ourselves. Please _fix_ this so that NuGet will _ask_ whether I actually _care_ that a package source is unavailable before it fails.

[deadlydog]

I have to agree with @rlundy. I understand that you are moving towards a new PackageReference model and looking to improve the design in that "vNext" model, but the reality is that many people will continue to use the "old" packages.config model for many years yet. Adding a small UI enhancement to prompt users if they want to continue when a package source is unavailable, or simply attempting to get/upgrade packages and only display an error if there is a failure would go a long way toward a better user experience for developers that will be continuing to use VS 0217 and .Net Full (as opposed to .Net Core) for the next 5 years.

~@StingyJack~ @zhili1208 please reconsider re-opening this issue. Thanks.

[StingyJack]

@deadlydog - i didn't close it, @zhili1208 did. I've complained about the same thing a few times and get stonewalled because "muh transitivity" or "muh package source diagnostics".

The post I linked above is an actual example of how adding a package with dependencies that are not already present on the nuget server is blocked. This neuters the argument of one sources not being available could cause dependency issues, _as they all must be on the same server anyway_

Basic stuff doesn't work right with nuget and instead of fixing it, msft is interested in what is generally the most expensive option in software development - to trash something and start over, instead of addressing the gaps.

[StingyJack]

@zhili1208 or @rrelyea - if this is closed "by design" then you probably have a bug in your design.

[rrelyea]

On phone currently, so not easy to find backlog item...

1) believe in the need to require all feeds
2) can help users disable sources when offline — has a design, but have not got to that.
3) we should definitely make sure we have a clear error experience, which seems to be questioned by nick craver in this Tweet:
https://twitter.com/nick_craver/status/1021714890640289792?s=21

@zhili1208 - for #2, please find backlog item and add link to this comment.
For #3, is our error experience clear enough?
Please chat offline with me too

[NickCraver]

FWIW, we hit this constantly as we have a feed behind the VPN. We've also learned various ways to produce a minimal repro quickly. I'd be more than happy to chat with anyone on the team about fixing it.

[StingyJack]

On phone currently, so not easy to find backlog item...

believe in the need to require all feeds

@rrelyea - you cannot mean that Package A from Feed A cannot be installed when Feed B is unavailable, and when Package A has no immediate or ancestral dependency in Feed B.

That dependency scenario _cant happen anyway_ currently, as you cannot push Package A to a Feed A unless all its dependencies are already present in Feed A. EDIT: Feed A and B are private feeds using Nuget.Server

[jainaashish]

As I mentioned on https://github.com/NuGet/Home/issues/7015 this issue simply doesn't mean that we start to ignore failing sources while restoring (please read the full comment on #7015).

About the dependency scenario, it's quite easy possible that packageA is coming from private feedA but depends on packageB which is coming from public feedB. We've already seen several such cases for popular packages which often customers don't push to their private feeds rather directly bring them from public feeds.

So until we have some way to lock down package versions and each restore will resolve to the same version independent of feeds then this feature should be enabled to ignore failed sources in UI.

[StingyJack]

you cannot mean that Package A from Feed A cannot be installed when Feed B is unavailable, and when Package A has no immediate or ancestral dependency in Feed B. (me)

About the dependency scenario, it's quite easy possible that packageA is coming from private feedA but depends on packageB which is coming from public feedB. (you)

We are stating different scenarios.

While the dependency graph is built and nuget is querying sources to fill that graph, if it finds what it needs in one source, why should it demand the other sources be available? It seems inefficient.

If I cant update a public package from nuget.org, because the VPN isnt connected at the moment and cant reach a private feed (a feed that is _unavailable_, not _failing_) it is going to be viewed as an artificial and needless impediment.

[rlundy]

I understand that there's a real reason for this restriction to exist.

But there are also real reasons for this restriction not to _always_ exist.

NuGet can already pop up dialogs; it does so for license acceptance.

I think all we're looking for is for NuGet to pop up a dialog saying "Hey, I can't reach feed XYZ; should I proceed anyway?"

Just for review, the use case is this:

In addition to the NuGet public feed, I use a private feed from my company that I connect to by VPN. If I'm not connected to the VPN, that private feed is unavailable. In that case, NuGet simply refuses to work until I either connect to the VPN, or disable that package source (and then I have to remember to re-enable it later...and every time I do anything with NuGet on this project I have to disable it again).

If I'm working on a project that has nothing to do with my company's private NuGet feed, NuGet shouldn't refuse to work without me connecting to that unavailable feed. Or at least I should be able to tell NuGet "I don't care that I can't reach that feed" without having to continuously disable/enable feeds.

[goldsam]

How about adding an argument or switch to override the default behavior? This seriously kills productivity. I think there is a pretty clear community sentiment that the current behavior is unhelpful at best. To be honest, its downright scary that a single point of failure was intentionally introduced by design into mission critical tooling.

[rlundy]

Proposal for how to fix this: A configuration file that I can put in my project root with a list of NuGet feeds I want to ignore for this project.

For a new project, I'd create this file and put a list of NuGet feed URLs in there that I _never_ want to use with this project. (E.g. this is a hobby project, so I want to ignore my company's NuGet feed which is annoyingly behind a VPN)

It would mean I'd have to create this file once for each new project, but there's a fair bit of setup anyway for a new project, so I don't think this would be an unreasonable burden.

Maybe it could be a .nugetignore file. 😁

[StingyJack]

@rlundy - does this work to meet that need? https://docs.microsoft.com/en-us/nuget/reference/nuget-config-file#disabledpackagesources

[rlundy]

Holy moley!

I had no idea such a thing existed.

How did nobody mention it till now in a year and a half of this issue existing!?

Thanks, @StingyJack! It looks like precisely what I'm looking for. I'll have to give it a spin later.

[deadlydog]

Adding the disabledPackageSources is a potential work around, but still a very painful one. I shouldn't have to edit all of my personal (and open source) projects to tell them to ignore my work's internal NuGet package source just so I can restore NuGet packages when not connected to my work's VPN.

[dquist]

👋 +1 from me as this issue has been constantly impacting our team since we set up a private nuget feed to host internal packages. There's literally no reason nuget should require a private feed to exist if I'm explicitly telling it to install a public package from nuget.org. What a nuisance!

[agaace]

My biggest issue with this behavior is that it causes obscure failures if you use Visual Studio to build your projects. I opened this issue for Visual Studio:
https://developercommunity.visualstudio.com/content/problem/878141/nugget-restore-randomly-failing-with-obscure-error.html
and they closed it as a dupe of this issue.. So we're in a downward spiral of confusing/obscure behavior here.

[sharprs83]

This needs fixed. To have to go to Tools > Options to disable a source temporarily while not connected to my work VPN just to use a public package that can't depend on a private package is adversarial UX design.

[rlundy]

For those who missed it above...

WORKAROUND

From a command prompt:

> nuget sources

It shows you something like:

Registered Sources:

  1.  nuget.org [Enabled]
      https://api.nuget.org/v3/index.json
  2.  My Company's NuGet Server [Enabled]
      https://example.com/whatever/nuget
  3.  Microsoft Visual Studio Offline Packages [Enabled]
      C:\Program Files (x86)\Microsoft SDKs\NuGetPackages\

Now create a file called nuget.config. Put this in it:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <disabledPackageSources>
    <add key="My Company's NuGet Server" value="true" />
  </disabledPackageSources>
</configuration>

Put that file in the root of your non-company code folder.

Done!

To confirm, run nuget sources in that same folder and you should now see [Disabled] next to whichever NuGet sources you put in the file.

There are other ways to do this, such as globally disabling your company's NuGet source on your PC and then only enabling it for the folder where you keep your company's code. These methods are left as an exercise for the reader. 😉

Note: You do not need a nuget.config for every project! You only need it in the root folder and it cascades. If your situation is more complicated, you can put nuget.config files at any level and change the enable/disable stuff as necessary.