Sdk: dotnet restore fails behind proxy using windows authentication

[axelheer]

Steps to reproduce

• set nuget http_proxy as described here
• validate that "classic" nuget actually works
• create new basic app using dotnet new
• restore packages using dotnet restore
  

  Actual behavior

log  : Restoring packages for C:\TEAH\SRC\HelloWorld\project.json...
info :   GET https://api.nuget.org/v3-flatcontainer/microsoft.netcore.dotnethostresolver/index.json
info :   ProxyAuthenticationRequired https://api.nuget.org/v3-flatcontainer/microsoft.netcore.dotnethostresolver/index.json 14ms
log  : Retrying 'FindPackagesByIdAsync' for source 'https://api.nuget.org/v3-flatcontainer/microsoft.netcore.dotnethostresolver/index.json'.
log  : Response status code does not indicate success: 407 (Proxy Authentication Required).
info :   GET https://api.nuget.org/v3-flatcontainer/microsoft.netcore.dotnethostresolver/index.json
info :   ProxyAuthenticationRequired https://api.nuget.org/v3-flatcontainer/microsoft.netcore.dotnethostresolver/index.json 10ms
log  : Retrying 'FindPackagesByIdAsync' for source 'https://api.nuget.org/v3-flatcontainer/microsoft.netcore.dotnethostresolver/index.json'.
log  : Response status code does not indicate success: 407 (Proxy Authentication Required).
info :   GET https://api.nuget.org/v3-flatcontainer/microsoft.netcore.dotnethostresolver/index.json
info :   ProxyAuthenticationRequired https://api.nuget.org/v3-flatcontainer/microsoft.netcore.dotnethostresolver/index.json 11ms
error: Failed to retrieve information from remote source 'https://api.nuget.org/v3-flatcontainer/microsoft.netcore.dotnethostresolver/index.json'.
error: Response status code does not indicate success: 407 (Proxy Authentication Required).
error: Failed to retrieve information from remote source 'https://api.nuget.org/v3-flatcontainer/microsoft.netcore.dotnethostresolver/index.json'.
error:   Response status code does not indicate success: 407 (Proxy Authentication Required).

Environment data

.NET Command Line Tools (1.0.0-preview1-002702)

Product Information:
 Version:     1.0.0-preview1-002702
 Commit Sha:  6cde21225e

Runtime Environment:
 OS Name:     Windows
 OS Version:  6.1.7601
 OS Platform: Windows
 RID:         win7-x64

[AmirSasson]

Got into the same problem, did you managed to solve?

[axelheer]

I'm currently using Fiddler as a local proxy using the "Automatically Authenticate" rule for our corporate proxy while the nuget config points to http://127.0.0.1:8888. Not very handy, but works as a temporary workaround.

[gafemoyano]

What worked for me was leaving the NuGet.Config in AppData intact after installation and checking the "Automatically Authenticate" option on Fiddler.

[cbeall]

I'm experiencing the same issue. Is it fair to assume (and plan for) this being something that would not be resolved until RTM? Does anybody know if this is an issue exclusively with windows, or impacts all platforms?

[dmarkle]

This is pretty huge. I'm experiencing this as well. It's a complete blocker for us.

[veb82]

I've tried setting proxy in nuget.config in AppData (proxy, user and pass) and it somehow ignores that, but not completely. If I set the proxy to nonsense URL it fails saying that the URL is wrong. If I provide a correct one it complains about auth (407). So it knows about the settings but somehow partially only. It's a showstopper in the corporate world...

[gafemoyano]

@veb82 I had a similar issue, if I set the proxxy on nuget.config in AppData instead of (407) I'd get a timeout. As I said before what worked for me was reverting the nuget.config to it's original state, downloading Fiddler and enabling the "Automatically Authenticate" option while I ran "dotnet restore". You can close Fiddler after that.

[Michael-D-Pine]

Is this for real ? So basically I really should be considering this as a tool to use because I can't tell over DEV's that you need to install Fiddler and do this and that to get things working. Just going to have to stick with traditional ASP.Net MVC blank application and then will convert over to .Net core when they get their act together.

I can't believe they just toss aside corporate developers like this, the completely stupid thing is I can actually use NPM, Bower et al because I have CNtlm configured but dotnet restore just doesn't work at all or have I missed something somewhere ?

[axelheer]

@Michael-D-Pine CNtlm instead of Fiddler should work too. It's basically the same problem NPM, Bower et al have...

[Michael-D-Pine]

@axelheer I don't understand this, I have CNtlm installed and as far as I am aware working because I can use npm fine from a command line but this always fails, I don't think I have ever been able to get it work here are there environment variables or something else that needs to be set in a configuration file that will allow the dotnet restore command or other commands to work correctly ?

cheers
Michael

[axelheer]

@Michael-D-Pine try setting the http_proxy value of the NuGet Config File to point to your local CNtlm installation (e.g. http://127.0.0.1:3128)

[Michael-D-Pine]

@axelheer Thanks for the info and updates we went and had a chat with our Operations guys and they have something setup that they put in our PC name and it basically enables us for anonymous authentication with the Proxy and this has worked for us here.

But my CNtlm is still fine for npm but I will move away from that soon as I now don't have to worry about re-hasing my password on password changes.

[DHirani]

I am getting the same issue with NuGet under VS 2015, I've tried everything and no joy, any ideas?

PATH=.node_modules.bin;C:\Program Files (x86)\Microsoft Visual Studio 14.0\Web\External;%PATH%;C:\Program Files (x86)\Microsoft Visual Studio 14.0\Web\External\git
C:\Program Files\dotnet\dotnet.exe restore "d:\tempTest.vs\restore.dg"
log : Restoring packages for d:\tempTestsrcTest\project.json...
info : GET https://api.nuget.org/v3-flatcontainer/microsoft.netcore.dotnethostresolver/index.json
info : ProxyAuthenticationRequired https://api.nuget.org/v3-flatcontainer/microsoft.netcore.dotnethostresolver/index.json 8ms
log : Retrying 'FindPackagesByIdAsync' for source 'https://api.nuget.org/v3-flatcontainer/microsoft.netcore.dotnethostresolver/index.json'.
log : Response status code does not indicate success: 407 (Proxy Authentication Required).
info : GET https://api.nuget.org/v3-flatcontainer/microsoft.netcore.dotnethostresolver/index.json
info : ProxyAuthenticationRequired https://api.nuget.org/v3-flatcontainer/microsoft.netcore.dotnethostresolver/index.json 4ms
log : Retrying 'FindPackagesByIdAsync' for source 'https://api.nuget.org/v3-flatcontainer/microsoft.netcore.dotnethostresolver/index.json'.
log : Response status code does not indicate success: 407 (Proxy Authentication Required).
info : GET https://api.nuget.org/v3-flatcontainer/microsoft.netcore.dotnethostresolver/index.json
info : ProxyAuthenticationRequired https://api.nuget.org/v3-flatcontainer/microsoft.netcore.dotnethostresolver/index.json 3ms
error: Failed to retrieve information from remote source 'https://api.nuget.org/v3-flatcontainer/microsoft.netcore.dotnethostresolver/index.json'.
error: Response status code does not indicate success: 407 (Proxy Authentication Required).
error: Failed to retrieve information from remote source 'https://api.nuget.org/v3-flatcontainer/microsoft.netcore.dotnethostresolver/index.json'.
error: Response status code does not indicate success: 407 (Proxy Authentication Required).
PATH=.node_modules.bin;C:\Program Files (x86)\Microsoft Visual Studio 14.0\Web\External;%PATH%;C:\Program Files (x86)\Microsoft Visual Studio 14.0\Web\External\git
C:\Program Files\dotnet\dotnet.exe restore "d:\temp\DayTrader.vs\restore.dg"
log : Restoring packages for d:\temp\DayTradersrc\DayTrader\project.json...
info : GET https://api.nuget.org/v3-flatcontainer/microsoft.netcore.dotnethostresolver/index.json
info : ProxyAuthenticationRequired https://api.nuget.org/v3-flatcontainer/microsoft.netcore.dotnethostresolver/index.json 14ms
log : Retrying 'FindPackagesByIdAsync' for source 'https://api.nuget.org/v3-flatcontainer/microsoft.netcore.dotnethostresolver/index.json'.
log : Response status code does not indicate success: 407 (Proxy Authentication Required).
info : GET https://api.nuget.org/v3-flatcontainer/microsoft.netcore.dotnethostresolver/index.json
info : ProxyAuthenticationRequired https://api.nuget.org/v3-flatcontainer/microsoft.netcore.dotnethostresolver/index.json 4ms
log : Retrying 'FindPackagesByIdAsync' for source 'https://api.nuget.org/v3-flatcontainer/microsoft.netcore.dotnethostresolver/index.json'.
log : Response status code does not indicate success: 407 (Proxy Authentication Required).
info : GET https://api.nuget.org/v3-flatcontainer/microsoft.netcore.dotnethostresolver/index.json
info : ProxyAuthenticationRequired https://api.nuget.org/v3-flatcontainer/microsoft.netcore.dotnethostresolver/index.json 3ms
error: Failed to retrieve information from remote source 'https://api.nuget.org/v3-flatcontainer/microsoft.netcore.dotnethostresolver/index.json'.
error: Response status code does not indicate success: 407 (Proxy Authentication Required).
error: Failed to retrieve information from remote source 'https://api.nuget.org/v3-flatcontainer/microsoft.netcore.dotnethostresolver/index.json'.
error: Response status code does not indicate success: 407 (Proxy Authentication Required).

[mareek]

Can someone of the donet cli team comment on this issue, please ?

[brthor]

dotnet restore simply forwards to NuGet.

cc @emgarten

[Michael-D-Pine]

I am pretty sure given my experience that CNTLM will not fix this issue as I stated I could not get that to solve my problem I had to get myself added on our internal group which basically allows anonymous proxy authentication which solves the problem. However maybe I was just missing some settings/configurations to Cntlm to solve the problem, i.e how to configure Visual Studio/dotnet restore to us Cntlm for proxy authentication.

This is just oh so painful for something that should just work out of the box at least they gave us an error message this time.

[tebeco]

@brthor @degrim, I'm wondering ... I got the issue too, and here is all I tried :)

I've installed .Net Core RC 2 Preview One on top of Vs2015.2
I've also installed the nuget Vsix 3.5.0 beta

I'm created a new Web>Asp.Net Core Web Application (.NET Framework) named "ReactDemo"

When I try to Right click on the node "References> Restore packages" here is the full log :

C:\Program Files\dotnet\dotnet.exe restore "D:\Sources\ReactDemo.vs\restore.dg"
log : Restoring packages for D:\Sources\ReactDemosrc\ReactDemo\project.json...
log : Restoring packages for tool 'Microsoft.AspNetCore.Server.IISIntegration.Tools' in D:\Sources\ReactDemosrc\ReactDemo\project.json...
info : GET https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.NETCore.DotNetHostResolver'
info : ProxyAuthenticationRequired https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.NETCore.DotNetHostResolver' 46ms
log : Retrying 'FindPackagesByIdAsyncCore' for source 'https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.NETCore.DotNetHostResolver''.
log : Response status code does not indicate success: 407 (Proxy Authentication Required).
info : GET https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.NETCore.DotNetHostResolver'
info : ProxyAuthenticationRequired https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.NETCore.DotNetHostResolver' 22ms
log : Retrying 'FindPackagesByIdAsyncCore' for source 'https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.NETCore.DotNetHostResolver''.
log : Response status code does not indicate success: 407 (Proxy Authentication Required).
info : GET https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.NETCore.DotNetHostResolver'
info : ProxyAuthenticationRequired https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.NETCore.DotNetHostResolver' 20ms
error: Failed to retrieve information from remote source 'https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.NETCore.DotNetHostResolver''.
error: Response status code does not indicate success: 407 (Proxy Authentication Required).
error: Failed to retrieve information from remote source 'https://www.nuget.org/api/v2/FindPackagesById()?id='Microsoft.NETCore.DotNetHostResolver''.
error: Response status code does not indicate success: 407 (Proxy Authentication Required).

When I try in the command prompt from one of these folders :
"D:\Sources\ReactDemo\"
"D:\Sources\ReactDemosrc\"
"D:\Sources\ReactDemosrc\ReactDemo"

dotnet restore
I got the EXACT same log

Then I went to nuget.org and downloaded "nuget.exe" (v3.4.x)
and within the folder "D:\Sources\ReactDemosrc\ReactDemo"

nuget restore -NoCache

This is actually working ...

That beeing said .. I tried it after removing :
%USERPROFILE%.nuget\packages
%ProgramData%\NuGet\Config (temporaly)

I've disabled the Offline nuget feed from nuget.config :
Can be found here : %AppData%\NuGet\Nuget.Config (or uncheck from VS IDE)

I've also tried with AND WITHOUT nuget settings : HTTP_PROXY / HTTP_PROXY.USER / HTTP_PROXY.PASSWORD
It still works without :

nuget restore -NoCache

Restoring packages for D:\Sources\ReactDemo\src\ReactDemo\project.json...
[huge list of GET / OK / Installing packages]
Restoring packages for tool '.....
Committing restore...
D:\Sources\ReactDemo\src\ReactDemo\project.json
Restore completed in 6717ms.

NuGet Config files used:
C:\Usersahertogs031416AppData\Roaming\NuGet\NuGet.Config
C:\ProgramData\nuget\Config\Microsoft.VisualStudio.Offline.config

Feeds used:
https://www.nuget.org/api/v2/

Installed:
48 package(s) to D:\Sources\ReactDemosrc\ReactDemo\project.json

That beeing said how can _dotnet restore_ fail on proxy when _nuget restore_ works if quote :
dotnet restore simply forwards to NuGet.

[tebeco]

@brthor @degrim
Sry but i'll have to bump this post against since there's really an issue with dotnet.exe since "nuget.exe" is working ... please at least give us a hint you just read the previous all block posted that tend to indicate that something is not going well

_Lots of company use NTLM proxy as you have seen from the HUGE number of people requesting (or going after auth & proxy)_
We're trying to use .Net Core for funny Kata, BBL, R&D etc ...
It's really making a bad image if basic stuff like the base EXE is not stable against NTLM proxy

[brthor]

First off, thanks everyone for filing this bug! I want to clear up a couple things.

The nuget.xplat client dotnet restore forwards to is not the same code as the nuget.exe which runs against the desktop .NET framework.

This is likely the reason for the discrepancy you are seeing between the two.

dotnet restore does nothing but invoke this dll, so there is no logic to debug in dotnet restore itself, and nothing we can do on the side of the cli.

Please move this issue to https://github.com/nuget/home as that is the only way this can be triaged by the appropriate team and then fixed.

[emgarten]

This issue is tracked for nuget here: https://github.com/NuGet/Home/issues/2393

As @brthor mentions nuget.exe uses the desktop framework, dotnet.exe on the other hand targets .net core which does not contain the windows auth APIs that nuget.exe uses.

[juanreyesv]

I had a similar issue, inside our corporate network, NuGet was not able to restore packages and the error read: "Proxy Access Denied". I followed two different Posts on StackOverflow to fix this issue.

First, you need to find the address of your proxy server: http://superuser.com/a/346376

And then you can download NuGet and configure it from the command line (http://stackoverflow.com/a/15463892/394348).

This configuration worked for me:
nuget.exe config -set http_proxy=http://YOUR PROXY ADDRESS HERE
nuget.exe config -set http_proxy.user=YOUR DOMAIN\YOUR WINDOWS ID

In my case I didn't need to set my password which is good. Hope this helps

[dcworldwide]

Thankyou @juanreyesv

To reiterate the steps:

• download the command line version of nuget
• add it to the path env var
• nuget.exe config -set http_proxy=http://YOUR PROXY ADDRESS HERE
• nuget.exe config -set http_proxy.user=YOUR DOMAIN\YOUR WINDOWS ID
• Dotnet restore etc

In my case, I'm using cntlm to interface with my proxy.

[Barryrowe]

I know this was closed and moved over to the NuGet project, but I'm not 100% convinced this isn't an issue with the Visual Studio's execution context of the dotnet restore command.

What I'm seeing is that if I do a command line dotnet restore with the http_proxy environment variable set, it works no problem. Doing a restore from within Visual Studio (with the http_proxy environment variable set at the system level), it fails and I end up with a 407 proxy errors.

What I find really bizarre, is that if I open a command prompt, run set http_proxy=[myproxyserver] and then do a command line dotnet restore, after this it appears that I can install packages normally through the visual studio nuget extension.

As soon as I close the command prompt that I ran the dotnet restore from, any nuget installs from visual studio begin failing again. This makes no sense to me, as the environment contexts shouldn't be related, but it appears my understanding of ENV in windows isn't complete.

You can see the same notes provided to the NuGet ticket here:
NuGet/Home#2393 (comment)

[tebeco]

Does it only shock me that i have to set a variable with the proxy and since it's an NTLM one the password and user are completely visible by any app ?

[MeirionHughes]

@Barryrowe if relevent; I'm having an issue with the csharp extension for vscode; it seems to be trying to download its dependencies and failing.

Creating C:\Users\Meirion.vscode\extensions\ms-vscode.csharp-1.4.1\coreclr-debug\project.json
Telemetry is: Enabled
log : Restoring packages for C:\Users\Meirion.vscode\extensions\ms-vscode.csharp-1.4.1\coreclr-debug\project.json...
error: Unable to load the service index for source https://api.nuget.org/v3/index.json.
error: Response status code does not indicate success: 407 (Proxy Authentication Required).
dotnet exited with error code 1

both nuget and dotnet restore both work fine when used directly. :/

[tebeco]

@juanreyesv => So we have to tell nuget who we are to let dotnet restore nuget ?
how does nuget restore work when i'm not settings proxy & user then ?

[juanreyesv]

@tebeco Yes I think that's right. Dotnet restore uses nuget to restore the dependencies so if nuget is not configured to work behind a proxy (and your computer is behind a proxy) dotnet restore fails.

nuget restore download the dependencies from the internet, if your computer is not behind a proxy then it will just work, is like browsing any website. That is of course if the nuget package is available and everything is ok with your internet.

Cheers

[jbeacham]

@gafemoyano @axelheer @Michael-D-Pine
Opening Fiddler was the only way I could restore packages successfully as well. There is a setting in C:\Windows\Microsoft.NET\Framework\v4.0.30319\Config\web.config file that was setting the defaultProxy and causing my get package requests to certain nuget domains to timeout.

<system.net> <defaultProxy> <proxy usesystemdefault="true" /> </defaultProxy> </system.net>

Setting usesystemdefault="false" fixed my issue.

[WolfgangKluge]

I still have issues with this and don't know whom to blame ;)
IMO, even if dotnet restore "does nothing but invoke this dll", it's too easy to hand over the responsibility to another project - maybe you have to call it with another parameter/setting/environment variable/you name it..

What I've investigated so far:
nuget restore works like a charm with and without the settings http_proxy, http_proxy.user and http_proxy.password in nuget.exe.config (nuget v3.5.0.1996)... Without the settings it uses the system settings.

However, dotnet restore doesn't work at all here (dotnet 1.0.0-preview2-1-003177)
And I even can see the reason (with WireShark):
nuget restore : CONNECT api.nuget.org:443 HTTP/1.1 , NTLMSSP_AUTH, User: DOMAIN\USER
dotnet restore: CONNECT api.nuget.org:443 HTTP/1.1 , NTLMSSP_AUTH, User: PROXY_DOMAIN\USER

With PROXY_DOMAIN I really mean the domain part of the http_proxy-setting (or system setting), and this is just wrong here (maybe someone else could confirm that behaviour).

---

I doublechecked if the config file is used by dotnet, too - and it is used (at least partly as also mentioned by @veb82).

If I add http_proxy, the proxy is used, otherwise not (so yes, the config file is used).
If I add http_proxy.user ("DOMAIN\User", "User" only, wrong user), nothing happens.
If I add a username to the proxy url, nothing happens, too...

So it seems that dotnet restore only uses the domain part of the proxy setting.

---

I see that dotnet restore internally calls msbuild.exe, so @jbeacham's solution might work, but (currently) not for me...

---

edit
My latest investigation: In my company we have a pac-script that splits the incoming proxy requests onto multiple proxy servers (simple loadbalancing based on the ip-adress)
If I use one of those proxys directly as http_proxy, it works... (not a solution I like - but it's the first time I get dotnet restore to work here). Maybe that's the reason why not everyone behind a proxy has such problems...

[emgarten]

@WolfgangKluge can you try the latest dotnet preview4 or preview5? I would still expect nuget.exe and dotnet to work the same even in this unique situation. If you are still seeing issues please open a new issue to track this.

[WolfgangKluge]

@emgarten thanks for replying. The latest I could find is preview 2.1 (https://www.microsoft.com/net/download/core#/current and https://github.com/dotnet/core/releases).

.NET Command Line Tools (1.0.0-preview2-1-003177)

Product Information:
 Version:            1.0.0-preview2-1-003177
 Commit SHA-1 hash:  a2df9c2576

Runtime Environment:
 OS Name:     Windows
 OS Version:  6.1.7601
 OS Platform: Windows
 RID:         win7-x64

Ah, now I've found something https://github.com/dotnet/core/blob/master/release-notes/preview4-download.md
I still don't see a preview5...

btw: I don't get the version system.. (https://github.com/dotnet/cli/releases)

• .NET Core 1.0.0 SDK – Preview 2-003156
• .NET Core SDK 1.0.0-preview4-004233
• .NET Core 1.0.0 SDK – Preview 2-1-003177
• .NET Core 1.0.0 SDK – Preview 3-004056
• .NET Core 1.0.0 SDK – Preview 2.1-003155

So the sequence is 2, 3, 2.1, 3, 2.1??

Edit: I still have to use Visual Studio 2015 in my company until the purchasing department decide to buy and distribute a new version - seems like this is not supported anymore. It's a mess that VS 2015 is no more than two years old and could not (and will not be able to) handle the new project file format

---

Edit: But yes, this works now, I think (at least outside VS)