Azure-pipelines-agent: Configuration & running agent behind corporate proxy

[jhmoy]

Wondering if there is a way we can specify proxy settings when running the configuration?

At the moment within corporate boundaries I'm getting this after running ./config.sh and entering my server URL and PAT ...

Connecting to server ...
An error occurred while sending the request.
Failed to connect. Try again or ctrl-c to quit.

However, when I connect my Mac outside of corporate boundaries the configuration and running of the agent works OK.

I noticed this thread for the previous agent https://github.com/Microsoft/vso-agent/issues/67 . I tried this ...

export HTTP_PROXY=[proxy url]

... but it didn't seem to work. Does this agent support proxies?

[bryanmacfarlane]

We will keep this open to track the work. Ideally, net core will provide the mechanism. In the previous agent, that setting was only for debugging (Fiddler, Charles, etc...) and didn't have full proxy support.

[stiliev]

@jhmoy I'm looking into adding proxy support. Do you know what proxy software you are using? Also is it authenticated proxy?

[jhmoy]

@stiliev I'm working at a site that uses BlueCoat as the proxy software. And yes it is an authenticated proxy.

[ghost]

@jhmoy have you been able to make it work behind the proxy? I face the same issue though in my case the proxy does not require authentication. For me neither the vsts-agent nor the vso agent work behind the proxy. vsts-agent fails with the same error despite I even set the proxy settings for curl incase.

[bryanmacfarlane]

Unauthenticated proxy support will be in the next preview this week.

[ghost]

@bryanmacfarlane Thank you! Looking forward to it.

[bryanmacfarlane]

2.102.0 supports setting proxy url. See troubleshooting section. Doesn't support setting proxy creds via envvars yet.

[schnitty]

I am tyring to configure an agent behind a corporate proxy and can't get it to work.
I am using 2.104.0

When I tried to configure without setting the VSTS_HTTP_PROXY environment variable it couldn't connect with an error straight away.

When I try to configure with the VSTS_HTTP_PROXY environment variable set it gets further through the configuration process, it even adds the agent (Can see it in Team Services web portal although marked red) but still crashes with an error.

Config.cmd Output

C:\Agent>config.cmd

>> Connect:

Enter server URL > https://pulselogistics.visualstudio.com
Enter authentication type (press enter for PAT) >
Enter personal access token > **************************************************
**
Connecting to server ...

>> Register Agent:

Enter agent pool (press enter for default) > Redacted
Enter agent name (press enter for PLSDB-1) > Redacted
Scanning for tool capabilities.
Connecting to the server.
Successfully added the agent
Testing agent connection.
Error while copying content to a stream.

C:\Agent>

I have attached the _diag entries for reference.
_diag.txt

[bryanmacfarlane]

@schnitty - it didn't crash. It did the test connection and failed to stream the result, detect the error, reported it and exited. The test connection did it's job.

Does the proxy require authentication? (we don't support that right now). Anything else you can tell us about your proxy server?

[TingluoHuang]

@schnitty are you able to get fiddler trace when the error happen?

[schnitty]

First let me say i'm thrilled with the quick response.

@bryanmacfarlane yes sorry it doesn't crash, I should have said that it failed to complete the entire configuration process.

The proxy may require authentication, I'll check with our client's IT department. I thought I may have gotten lucky (in the sense that I got this far without it) but if it is expected that it doesn't work at that point in the code (needs credentials for proxy) then that is good to know.

I pulled down the source (As a long time MS dev, it feels good to be able to do this now) and did some debugging on my dev machine and noted that the initial connection to register the agent and query the capabilities works ok but the second connection doesn't, even though the calling code is similar/same. I figured that if the first call worked (through the proxy) then the second one would also. I'm probably showing my ignorance here but seemed logical at the time :)

At this juncture I will speak with their IT to determine if the proxy needs authentication details or not. If it does then I guess I have my answer. I'll have to get them to allow requests out to our team services account to bypass the proxy. If it doesn't require authentication then I will see when/if I can do a fiddler trace and get back to you.

Thanks

[schnitty]

We managed to get the agent to configure.

Their IT modified their proxy to bypass authentication for the following sites:

https://<myteamservicesaccount>.visualstudio.com 
https://<myteamservicesaccount>.vssps.visualstudio.com

Doing that got past the error that I was experiencing.
I won't know if that is enough until I can test releasing using the agent which I can't do until I can schedule some downtime on their server to install the latest version of .NET framework.

Will update how I go next week.

[bryanmacfarlane]

@schnitty - thanks for the update. This issue is tracking the ability to supply credentials which we will get to soon I hope.

[jhmoy]

I should mention that in relation to my customer, they ended up agreeing to place the Mac in a separate zone that didn't require proxy access to the internet.

[bryanmacfarlane]

@jhmoy Thanks. Hopefully we can get to full support very soon.

[claytob86]

Hi, thanks for all the comments above.

I've run into this issue and our networking team have enabled bypass authentication of the sites @schnitty mentioned above but I'm still unable to set up the agent. Bypassing https://myteamservicesaccount.visualstudio.com allowed the application to reach the ">>Register Agent:" prompt but it's a couple of steps later that it's failing at now:

Connecting to server
Register Agent:
Enter agent pool (press enter for default) >
Enter agent name (press enter for MyAgent) >
Scanning for tool capabilities.
Connecting to the server.
An error occurred while sending the request.

I've had a look at the traffic in Wireshark and it appears there's a subsequent connection attempt to mycompute2.iaas.ext.azure.com:433. We've tried bypassing the proxy for this too but to no avail...

[jdshkolnik]

I also tried setting VSTS_HTTP_PROXY and event viewer started showing An error occurred: Invalid URI: The URI scheme is not valid. right away. I've had to disable the agent until there's a fix.

[steveplatz]

I wanted to chime in to mention that I'm also having trouble getting the agent to work with my company's proxy, which does normally require authentication. However, I was given a proxy address that should allow the agent to bypass the authentication, but when I set it using set VSTS_HTTP_PROXY=https://<proxyaddress>:<port> I still get stuck during configuration. It almost seems like it's ignoring the proxy settings.

Fiddler doesn't show anything different when using the proxy above or using 127.0.0.1:8888.

>> Connect:

Enter server URL > https://<account>.visualstudio.com
Enter authentication type (press enter for PAT) >
Enter personal access token > **************************************************
**
Connecting to server ...

>> Register Agent:

Enter agent pool (press enter for default) >
Enter agent name (press enter for SERVERNAME) >
Scanning for tool capabilities.
Connecting to the server.
Enter replace? (Y/N) (press enter for N) > y
Successfully replaced the agent
Testing agent connection.
A task was canceled.

[schnitty]

Just another little update in case this helps anyone or not we got the agent configured but then it wouldn't run. We got it running but also had to add the following sites to the list of sites to not require authenticate when using the proxy.

https://github-cloud.s3.amazonaws.com
https://github.com

[bryanmacfarlane]

@schnitty - great. Note that the github endpoints are used when the agent is updated. It might be github, it might be download center eventually etc...

[schnitty]

Thanks for the info. I haven't put it under any significant tests yet so from your post and other people's experience it looks like we have two options. 1. Wait until authenticated proxies are supported. 2. Get the IT team to keep a trace on the sites that it hits and keep adding them to the list.

[bryanmacfarlane]

Authenticated proxies will be supported with the 2.105.0 release tonight/tomorrow

[bryanmacfarlane]

2.105.0 is uploaded

https://github.com/Microsoft/vsts-agent/blob/master/docs/start/proxyconfig.md

[jdshkolnik]

@bryanmacfarlane Awesome, thanks. How long does it usually take before it appears in VSTS? I just downloaded an agent and it was 2.104.2.

[bryanmacfarlane]

We're doing the config change today. You can always download directly from GitHub. the config change just changes the url and version in our DB to point.

[jdshkolnik]

@bryanmacfarlane I downloaded, removed current agent, set variables (username and password), rebooted, confirmed variables set, added new agent, added .proxy file, rebooted, and it shows just as before...registered but offline.

[2016-08-17 22:38:27Z INFO Terminal] WRITE LINE: Connecting to the server.
[2016-08-17 22:38:27Z INFO MessageListener] Attempt to create session.
[2016-08-17 22:38:27Z INFO MessageListener] Connecting to the Agent Server...
[2016-08-17 22:43:02Z ERR  MessageListener] Catch exception during create session.
[2016-08-17 22:43:02Z ERR  MessageListener] System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.Http.WinHttpException: The operation timed out
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Net.Http.WinHttpHandler.<StartRequest>d__101.MoveNext()
   --- End of inner exception stack trace ---

[bryanmacfarlane]

We had some customers get through it. They had to trim the leading or trailing white space in the .proxy file. We're updating doc and next patch will trim

[jdshkolnik]

That was it! Instant green.

[bryanmacfarlane]

Awesome. We will fix in .1

[bryanmacfarlane]

Closing as we've had a couple of positive confirmations. We did the trim fix already.

[naveenkumargc]

Thank you trim fix. works great !

[ARMAAN7139]

I am getting the similar error the logs of agent is as follows:

>
System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.Http.CurlException: Failed initialization
at System.Net.Http.CurlHandler.ThrowIfCURLEError(CURLcode error)
at System.Net.Http.CurlHandler.EasyRequest.SetNetworkingOptions()
at System.Net.Http.CurlHandler.EasyRequest.InitializeCurl()
at System.Net.Http.CurlHandler.MultiAgent.ActivateNewRequest(EasyRequest easy)
--- End of inner exception stack trace ---
at Microsoft.VisualStudio.Services.Common.VssHttpRetryMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts) at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.SendAsync(HttpRequestMessage message, HttpCompletionOption completionOption, Object userState, CancellationToken cancellationToken) at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.SendAsync[T](HttpRequestMessage message, Object userState, CancellationToken cancellationToken) at Microsoft.VisualStudio.Services.Location.Client.LocationHttpClient.GetConnectionDataAsync(ConnectOptions connectOptions, Int64 lastChangeId, CancellationToken cancellationToken, Object userState) at Microsoft.VisualStudio.Services.WebApi.Location.VssServerDataProvider.ConnectAsync(ConnectOptions connectOptions, CancellationToken cancellationToken) at Microsoft.VisualStudio.Services.Agent.LocationServer.ConnectAsync(VssConnection jobConnection) [2019-04-29 20:36:16Z INFO VisualStudioServices] Starting operation Location.GetConnectionData [2019-04-29 20:36:16Z ERR VisualStudioServices] GET request to https://bpt-informationsecurity-cat-com.visualstudio.com/_apis/connectionData?connectOptions=1&lastChangeId=-1&lastChangeId64=-1 failed. System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.Http.CurlException: Failed initialization at System.Net.Http.CurlHandler.ThrowIfCURLEError(CURLcode error) at System.Net.Http.CurlHandler.EasyRequest.SetNetworkingOptions() at System.Net.Http.CurlHandler.EasyRequest.InitializeCurl() at System.Net.Http.CurlHandler.MultiAgent.ActivateNewRequest(EasyRequest easy) --- End of inner exception stack trace --- at Microsoft.VisualStudio.Services.Common.VssHttpMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at Microsoft.VisualStudio.Services.Common.VssHttpRetryMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) [2019-04-29 20:36:16Z INFO VisualStudioServices] Finished operation Location.GetConnectionData [2019-04-29 20:36:16Z INFO CommandSettings] Flag 'unattended': 'False' [2019-04-29 20:36:16Z ERR Terminal] WRITE ERROR (exception): [2019-04-29 20:36:16Z ERR Terminal] System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.Http.CurlException: Failed initialization at System.Net.Http.CurlHandler.ThrowIfCURLEError(CURLcode error) at System.Net.Http.CurlHandler.EasyRequest.SetNetworkingOptions() at System.Net.Http.CurlHandler.EasyRequest.InitializeCurl() at System.Net.Http.CurlHandler.MultiAgent.ActivateNewRequest(EasyRequest easy) --- End of inner exception stack trace --- at Microsoft.VisualStudio.Services.Common.VssHttpRetryMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.SendAsync(HttpRequestMessage message, HttpCompletionOption completionOption, Object userState, CancellationToken cancellationToken)
at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.SendAsyncT
at Microsoft.VisualStudio.Services.Location.Client.LocationHttpClient.GetConnectionDataAsync(ConnectOptions connectOptions, Int64 lastChangeId, CancellationToken cancellationToken, Object userState)
at Microsoft.VisualStudio.Services.WebApi.Location.VssServerDataProvider.ConnectAsync(ConnectOptions connectOptions, CancellationToken cancellationToken)
at Microsoft.VisualStudio.Services.Agent.LocationServer.ConnectAsync(VssConnection jobConnection)
at Microsoft.VisualStudio.Services.Agent.Listener.Configuration.ConfigurationManager.IsHostedServer(String serverUrl, VssCredentials credentials)
at Microsoft.VisualStudio.Services.Agent.Listener.Configuration.ConfigurationManager.ConfigureAsync(CommandSettings command)
[2019-04-29 20:36:16Z ERR Terminal] WRITE ERROR: Failed to connect. Try again or ctrl-c to quit
[2019-04-29 20:36:16Z INFO CommandSettings] Arg 'url': ''
[2019-04-29 20:36:16Z INFO CommandSettings] Flag 'unattended': 'False'
[2019-04-29 20:36:16Z INFO PromptManager] ReadValue
[2019-04-29 20:36:16Z INFO Terminal] WRITE: Enter server URL >
[2019-04-29 20:36:16Z INFO Terminal] READ LINE
[2019-04-29 20:36:58Z INFO Terminal] WRITE LINE: Exiting.

Any help would be great.

[TingluoHuang]

@ARMAAN7139 try use CURL to hit the service through your proxy server, the error you get might means the libcurl on your box is not properly installed.

[ARMAAN7139]

@TingluoHuang Thank you so much, after installing the new curl version, I was able to connect to the server successfully. Cheers.

[TingluoHuang]

@ARMAAN7139 cool, grad you unblocked. 🎉

[4F2E4A2E]

I am experiencing the same error all over different agent version:
at System.Net.Http.CurlHandler.ThrowIfCURLEError(CURLcode error)

Pls anyone, what is this? Can you provide some dotnet new console example code to reproduce this in a manner that brings us closer to a solution?

all agent dependencies: installed and functional
all dotnet dependencies: installed and functional
curl: works
csharp-example-code: works
libcurl: works
openssh: works
ssl connections in any way: works
dotnet dependencies and tools: works

azure agent for linux: does not work at all

Experiencing the same behavior on rhel6 with curl 7.67, no matter what I do.
openssh, curl and dotnet-console tests to the very same host can be established are nominal

but with the agent it's not working at all