Version of your agent? 2.115.0
OS of the machine running the agent? Windows
VisualStudio.com or On-Prem TFS? VisualStudio.com
If On-Prem TFS, which release? 2015 RTM/QU1/QU2/...
If VisualStudio.com, what is your account name? http://etiengineeringteam.visualstudio.com
GIT fetch is not working from Bitbucket. I'm using basic authentication method and the issue seems to be random for me. Sometimes a new build just works fine. This happens in many projects I've setup.
I think I manage to "fix" it by re-configuing the repository details: I mean editing the build, clearing the connection setting, and selecting it again. Same with the repository and branch (same settings).
Nevertheless is not a 100% working fix, since some projects keeps on failing. Repository URLs and branches looks fine.
offtopic-related: Is there a way to bulk update all the connections settings for each one of my build projects? I have between 50 and 60 projects. I used to have a token based authentication which stopped working. I had to create this new connection (basic authentication) and I'm still updating the projects on demand. In the meantime the projects I fixed keeps on failing randomly on fetch.
No matter is a bug or not I'm very happy with VSTS
Thank you!
2018-08-22T19:16:46.9010669Z git version 2.14.3.windows.1
2018-08-22T19:16:46.9016002Z ##[debug]Detect git version: 2.14.3.
2018-08-22T19:16:46.9068530Z ##[debug]Set git useragent to: git/2.14.3 (vsts-agent-git/2.131.0).
2018-08-22T19:16:46.9105130Z ##[debug]Checking if the repo on c:agentsagent1_work\43\s matches the expected repository origin URL. expected Url: https://bitbucket.org//eti*/
2018-08-22T19:16:46.9142121Z ##[debug]Inspect remote.origin.url for repository under c:agentsagent1_work\43\s
2018-08-22T19:16:46.9142560Z ##[command]git config --get remote.origin.url
2018-08-22T19:16:46.9302767Z ##[debug]Get remote origin fetch url from git config: https://bitbucket.org//et**/
2018-08-22T19:16:46.9303440Z ##[debug]Repository remote origin fetch url is https://bitbucket.org//et**/
2018-08-22T19:16:46.9304009Z ##[debug]URLs match.
2018-08-22T19:16:46.9333431Z ##[debug]Disable git auto garbage collection.
2018-08-22T19:16:46.9361121Z ##[command]git config gc.auto 0
2018-08-22T19:16:46.9554376Z ##[debug]Checking git config http.https://bitbucket.org//et**/.extraheader exist or not
2018-08-22T19:16:46.9554881Z ##[command]git config --get-all http.https://bitbucket.org//et*/.extraheader
2018-08-22T19:16:46.9711345Z ##[debug]Checking git config http.proxy exist or not
2018-08-22T19:16:46.9711750Z ##[command]git config --get-all http.proxy
2018-08-22T19:16:46.9913423Z ##[debug]Fetch git repository at: c:agentsagent1_work\43\s remote: origin.
2018-08-22T19:16:46.9942708Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --tags --prune --progress --no-recurse-submodules origin
2018-08-22T19:16:47.5633335Z fatal: The remote end hung up unexpectedly
2018-08-22T19:16:47.5634034Z error: RPC failed; HTTP 405 curl 22 The requested URL returned error: 405 Method Not Allowed
2018-08-22T19:16:47.5832279Z ##[error]Git fetch failed with exit code: 128
2018-08-22T19:16:47.5855088Z ##[debug]System.InvalidOperationException: Git fetch failed with exit code: 128
at Microsoft.VisualStudio.Services.Agent.Worker.Build.GitSourceProvider.
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.VisualStudio.Services.Agent.Worker.Build.BuildJobExtension.
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.VisualStudio.Services.Agent.Worker.JobExtensionRunner.
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.VisualStudio.Services.Agent.Worker.StepsRunner.
2018-08-22T19:16:47.5864203Z ##[section]Finishing: Get Sources
Please ensure you are logging issues to the correct repository in order to get the best support.
If you are hitting a generic issue about VSTS/TFS, please report it to the Developer Community
@gonzaloluna there is a server side issue for this, a fix is rolling to all VSTS rings. you don't have to change anything, everything should be back to normal later today.
@gonzaloluna deployment has finished on your account, can you check whether your existing bitbucket build definition start working?
@TingluoHuang it seems to be working again. Should I expect to happen again? this seemed to be a recurring issue for the last days (if not weeks).
Do you know what's the fastest way to bulk updated the connection setting for all of my build definitions?
Thanks
@gonzaloluna it's bug we didn't catch during test, we added a canary definition to make sure this don't happen again.
you can use REST API update your definition in script.
you can leave your definition as it is today, it should continue work.
i am going to close this issue for now since the fix has deployed to everywhere. thanks for reporting this!
FYI still happened to me, it happens once in a while so I guess not a bad config on our end. Might be that the fix was not deployed everywhere yet?
2018-08-23T23:54:40.6024614Z ##[section]Starting: Get sources
2018-08-23T23:54:40.6269233Z ==============================================================================
2018-08-23T23:54:40.6269483Z Task : Get sources
2018-08-23T23:54:40.6269711Z Description : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.
2018-08-23T23:54:40.6269949Z Version : 1.0.0
2018-08-23T23:54:40.6270104Z Author : Microsoft
2018-08-23T23:54:40.6270313Z Help : [More Information](https://go.microsoft.com/fwlink/?LinkId=798199)
2018-08-23T23:54:40.6270536Z ==============================================================================
2018-08-23T23:54:41.7981175Z Syncing repository: ____ (Bitbucket)
2018-08-23T23:54:41.8016538Z Prepending Path environment variable with directory containing 'git.exe'.
2018-08-23T23:54:41.8023725Z ##[command]git version
2018-08-23T23:54:42.7936221Z git version 2.17.1.windows.2
2018-08-23T23:54:42.7940403Z ##[command]git lfs version
2018-08-23T23:54:47.4364431Z git-lfs/2.4.2 (GitHub; windows amd64; go 1.8.3; git 6f4b2e98)
2018-08-23T23:54:47.4663192Z ##[command]git init "D:\a\1\s"
2018-08-23T23:54:47.9132703Z Initialized empty Git repository in D:/a/1/s/.git/
2018-08-23T23:54:47.9193529Z ##[command]git remote add origin https://bitbucket.org//____
2018-08-23T23:54:48.0604975Z ##[command]git config gc.auto 0
2018-08-23T23:54:48.2013004Z ##[command]git config --get-all http.https://bitbucket.org//____.extraheader
2018-08-23T23:54:48.2318530Z ##[command]git config --get-all http.proxy
2018-08-23T23:54:48.2689404Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --tags --prune --progress --no-recurse-submodules origin
2018-08-23T23:54:52.1856231Z fatal: The remote end hung up unexpectedly
2018-08-23T23:54:52.1869463Z error: RPC failed; HTTP 405 curl 22 The requested URL returned error: 405 Method Not Allowed
2018-08-23T23:54:52.1983589Z ##[warning]Git fetch failed with exit code 128, back off 3.836 seconds before retry.
2018-08-23T23:54:56.0304295Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --tags --prune --progress --no-recurse-submodules origin
2018-08-23T23:54:56.5619631Z fatal: The remote end hung up unexpectedly
2018-08-23T23:54:56.5635372Z error: RPC failed; HTTP 405 curl 22 The requested URL returned error: 405 Method Not Allowed
2018-08-23T23:54:56.5695100Z ##[warning]Git fetch failed with exit code 128, back off 2.893 seconds before retry.
2018-08-23T23:54:59.4679848Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --tags --prune --progress --no-recurse-submodules origin
2018-08-23T23:54:59.8304320Z remote: Repository /____ not found
2018-08-23T23:54:59.8304960Z fatal: repository 'https://bitbucket.org//____/' not found
2018-08-23T23:54:59.8658296Z ##[error]System.InvalidOperationException: Git fetch failed with exit code: 128
at Agent.Plugins.Repository.GitSourceProvider.<GetSourceAsync>d__11.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Agent.Plugins.Repository.CheckoutTask.<RunAsync>d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Agent.PluginHost.Program.Main(String[] args)
2018-08-23T23:54:59.8747605Z ##[section]Finishing: Get sources
Our account name: mediaclip.visualstudio.com, build ID 1664
We are seeing the same thing in our account, frznmtn.visualstudio.com. Please re-open this issue!
@TingluoHuang we did a bit more debugging, and it looks like this is specific to the Bitbucket Cloud service type in VSTS. Specifically, the URL it generates is wrong. An extra slash is added after bitbucket.org (e.g. https://bitbucket.org//___/ instead of https://bitbucket.org/__/).
We can kinda work around the issue by using a new service connection set up using the External Git service type in VSTS, but then we lose our triggers.
This is likely going to affect anyone using VSTS build pipelines with Bitbucket...
the extra / shouldn't cause any problem, I have it in my build and it works fine.
I am reopen the issue.
2018-08-24T12:12:33.9454803Z ##[command]git remote add origin https://bitbucket.org//***/test-main-repository
2018-08-24T12:12:34.0637217Z ##[command]git config gc.auto 0
2018-08-24T12:12:34.2366151Z ##[command]git config --get-all http.https://bitbucket.org//***/test-main-repository.extraheader
2018-08-24T12:12:34.2794754Z ##[command]git config --get-all http.proxy
2018-08-24T12:12:34.3269920Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --tags --prune --progress --no-recurse-submodules origin
2018-08-24T12:12:41.3028258Z remote: Counting objects: 47, done.
2018-08-24T12:12:41.4868478Z remote: Compressing objects: 2% (1/44)
2018-08-24T12:12:41.4869886Z remote: Compressing objects: 4% (2/44)
2018-08-24T12:12:41.4870453Z remote: Compressing objects: 6% (3/44)
2018-08-24T12:12:41.4897939Z remote: Compressing objects: 9% (4/44)
2018-08-24T12:12:41.4898439Z remote: Compressing objects: 11% (5/44)
2018-08-24T12:12:41.4898850Z remote: Compressing objects: 13% (6/44)
2018-08-24T12:12:41.4899137Z remote: Compressing objects: 15% (7/44)
2018-08-24T12:12:41.4928226Z remote: Compressing objects: 18% (8/44)
2018-08-24T12:12:41.4928686Z remote: Compressing objects: 20% (9/44)
2018-08-24T12:12:41.4929042Z remote: Compressing objects: 22% (10/44)
@avenema @christianrondeau I can repro it, but the error is really intermittent, I queue 10 builds, 3 of them failed with the error, are you guys experiences the same problem?
It is intermittent. I can see successful builds with warnings because of this error, but it automatically retried and it went through.
i queued another 10 builds, and all of them are succeed, there is nothing change on VSTS/Agent side in the last 24 hours, i am wonder there was an incident in Bitbucket that cause the intermittent failure, i am going to close this issue again as the original problem has fixed :D
I know a fix has been released; but I am still receiving this error
2018-08-27T18:45:26.7538340Z error: RPC failed; HTTP 405 curl 22 The requested URL returned error: 405 Method Not Allowed
2018-08-27T18:45:27.2065279Z ##[error]System.InvalidOperationException: Git fetch failed with exit code: 128
at Agent.Plugins.Repository.GitSourceProvider.<GetSourceAsync>d__11.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Agent.Plugins.Repository.CheckoutTask.<RunAsync>d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Agent.PluginHost.Program.Main(String[] args)
@TingluoHuang
Issue still persist to me as well please see below logs
Enumerating objects: 73, done.
Counting objects: 100% (73/73), done.
Delta compression using up to 8 threads.
Compressing objects: 100% (41/41), done.
Writing objects: 100% (41/41), 705.54 KiB | 8.02 MiB/s, done.
Total 41 (delta 24), reused 0 (delta 0)
error: RPC failed; HTTP 405 curl 22 The requested URL returned error: 405 Method Not Allowed
fatal: The remote end hung up unexpectedly
fatal: The remote end hung up unexpectedly
@legend712 @j2ali the error makes me feel something wrong on bitbucket side, since the agent just use git.exe to interactive with bitbucket and error is through from git.exe
@TingluoHuang so how to fix it then ?
thanks
@TingluoHuang seems the remote wasn't added correctly due to which it failed I was able to pushed once I fix that remote :)
Thanks
@legend712 what did you change to fix it? the error is intermittent on fetch.
@TingluoHuang I change the remote and then I able to push may be it wasn't right
Thanks
I am getting the same issue. Any workarounds?
2018-08-27T23:15:16.6419430Z error: RPC failed; HTTP 405 curl 22 The requested URL returned error: 405 Method Not Allowed
2018-08-27T23:15:16.6649528Z ##[warning]Git fetch failed with exit code 128, back off 5.543 seconds before retry.
2018-08-27T23:15:22.2194750Z ##[command]git -c http.extraheader="AUTHORIZATION: basic " fetch --tags --prune --progress --no-recurse-submodules origin
2018-08-27T23:15:22.8564216Z remote: Repository / not found
2018-08-27T23:15:22.8565929Z fatal: repository 'https://bitbucket.org//*/test/' not found
2018-08-27T23:15:22.9127674Z ##[error]System.InvalidOperationException: Git fetch failed with exit code: 128
at Agent.Plugins.Repository.GitSourceProvider.
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Agent.Plugins.Repository.CheckoutTask.
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Agent.PluginHost.Program.Main(String[] args)
2018-08-27T23:15:22.9200636Z ##[section]Finishing: Get sources
+1: Our team is also experiencing this issue at the moment. Presently all builds are failing with this error:
2018-08-27T18:27:45.8202357Z Syncing repository: ***************/*********** (Bitbucket)
2018-08-27T18:27:45.8242995Z Prepending Path environment variable with directory containing 'git.exe'.
2018-08-27T18:27:45.9024962Z ##[command]git version
2018-08-27T18:27:47.0547497Z git version 2.17.1.windows.2
2018-08-27T18:27:47.0817744Z ##[command]git lfs version
2018-08-27T18:27:53.5553798Z git-lfs/2.4.2 (GitHub; windows amd64; go 1.8.3; git 6f4b2e98)
2018-08-27T18:27:53.5928132Z ##[command]git init "D:\a\1\s"
2018-08-27T18:27:54.2265862Z Initialized empty Git repository in D:/a/1/s/.git/
2018-08-27T18:27:54.2328432Z ##[command]git remote add origin https://bitbucket.org//***************/***********
2018-08-27T18:27:54.3198320Z ##[command]git config gc.auto 0
2018-08-27T18:27:54.3982035Z ##[command]git config --get-all http.https://bitbucket.org//***************/***********.extraheader
2018-08-27T18:27:54.4359016Z ##[command]git config --get-all http.proxy
2018-08-27T18:27:54.4800925Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --tags --prune --progress --no-recurse-submodules origin
2018-08-27T18:27:59.4626772Z fatal: The remote end hung up unexpectedly
2018-08-27T18:27:59.4637172Z error: RPC failed; HTTP 405 curl 22 The requested URL returned error: 405 Method Not Allowed
2018-08-27T18:28:01.2836072Z ##[error]System.InvalidOperationException: Git fetch failed with exit code: 128
at Agent.Plugins.Repository.GitSourceProvider.<GetSourceAsync>d__11.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Agent.Plugins.Repository.CheckoutTask.<RunAsync>d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Agent.PluginHost.Program.Main(String[] args)
2018-08-27T18:28:01.3605791Z ##[section]Finishing: Get sources
I have updated to the latest (v2.139.1), the error has changed, however; I am unable to build.
2018-08-27T23:42:53.8176019Z ##[section]Starting: Get sources
2018-08-27T23:42:53.8347516Z ==============================================================================
2018-08-27T23:42:53.8347682Z Task : Get sources
2018-08-27T23:42:53.8347969Z Description : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.
2018-08-27T23:42:53.8348194Z Version : 1.0.0
2018-08-27T23:42:53.8348288Z Author : Microsoft
2018-08-27T23:42:53.8348549Z Help : More Information
2018-08-27T23:42:53.8348683Z ==============================================================================
2018-08-27T23:42:54.2072700Z Syncing repository: /Test (Bitbucket)
2018-08-27T23:42:54.2106222Z Prepending Path environment variable with directory containing 'git.exe'.
2018-08-27T23:42:54.2167431Z ##[command]git version
2018-08-27T23:42:54.2996053Z git version 2.17.1.windows.2
2018-08-27T23:42:54.3228568Z ##[command]git config --get remote.origin.url
2018-08-27T23:42:54.3581583Z ##[command]git config gc.auto 0
2018-08-27T23:42:54.3955568Z ##[command]git config --get-all http.https://bitbucket.org///Test.extraheader
2018-08-27T23:42:54.4333161Z ##[command]git config --get-all http.proxy
2018-08-27T23:42:54.4732126Z ##[command]git -c http.extraheader="AUTHORIZATION: basic " fetch --tags --prune --progress --no-recurse-submodules origin
2018-08-27T23:42:55.0112039Z remote: Repository / not found
2018-08-27T23:42:55.0113769Z fatal: repository 'https://bitbucket.org///Test/' not found
2018-08-27T23:42:55.0445018Z ##[warning]Git fetch failed with exit code 128, back off 2.404 seconds before retry.
2018-08-27T23:42:57.4366006Z ##[command]git -c http.extraheader="AUTHORIZATION: basic *" fetch --tags --prune --progress --no-recurse-submodules origin
2018-08-27T23:42:58.0712320Z remote: Repository / not found
2018-08-27T23:42:58.0713933Z fatal: repository 'https://bitbucket.org///Test/' not found
2018-08-27T23:42:58.0884937Z ##[warning]Git fetch failed with exit code 128, back off 8.388 seconds before retry.
2018-08-27T23:43:06.4903824Z ##[command]git -c http.extraheader="AUTHORIZATION: basic " fetch --tags --prune --progress --no-recurse-submodules origin
2018-08-27T23:43:07.1388301Z remote: Repository / not found
2018-08-27T23:43:07.1389937Z fatal: repository 'https://bitbucket.org//**/Test/' not found
2018-08-27T23:43:07.2036102Z ##[error]System.InvalidOperationException: Git fetch failed with exit code: 128
at Agent.Plugins.Repository.GitSourceProvider.
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Agent.Plugins.Repository.CheckoutTask.
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Agent.PluginHost.Program.Main(String[] args)
2018-08-27T23:43:07.2189419Z ##[section]Finishing: Get sources
@jonreis @LeastOne can you guys fetch your bitbucket repro from command line successfully using https?
@jonreis what's the agent version you have before?
Hello @TingluoHuang it was vsts-agent-win-x64-2.138.6
Sorry @TingluoHuang I forgot to answer your other question. Yes, I can fetch my repo from bitbucket with the command line using https.
@TingluoHuang: I was actually just doing exactly that ... yes, I grabbed the identical commands from the build log and ran them locally with no issue ...
git init
git remote add origin https://bitbucket.org//***************/***********
git config gc.auto 0
git config --get-all http.https://bitbucket.org//***************/***********.extraheader
git config --get-all http.proxy
git -c http.extraheader="AUTHORIZATION: basic **************************" fetch --tags --prune --progress --no-recurse-submodules origin
remote: Counting objects: 111260, done.
remote: Compressing objects: 100% (26180/26180), done.
remote: Total 111260 (delta 89548), reused 103680 (delta 83866)
Receiving objects: 100% (111260/111260), 653.30 MiB | 10.80 MiB/s, done.
Resolving deltas: 100% (89548/89548), done.
Next, I'm going to take you up on your suggestion to configure the repo as an external git service to hopefully get builds working despite triggers.
@TingluoHuang: Attempting to configure the repo as an external git service also yielded an error, only it was a 504 gateway timeout instead of a 405 method not allowed. Thoughts?
2018-08-27T23:51:32.5958348Z ##[section]Starting: Get sources
2018-08-27T23:51:32.6228888Z ==============================================================================
2018-08-27T23:51:32.6229320Z Task : Get sources
2018-08-27T23:51:32.6229694Z Description :
2018-08-27T23:51:32.6230018Z Version : 1.0.0
2018-08-27T23:51:32.6230340Z Author : Microsoft
2018-08-27T23:51:32.6230666Z Help :
2018-08-27T23:51:32.6231061Z ==============================================================================
2018-08-27T23:51:33.8234579Z Syncing repository: repository (ExternalGit)
2018-08-27T23:51:33.8368840Z Prepending Path environment variable with directory containing 'git.exe'.
2018-08-27T23:51:33.9294821Z ##[command]git version
2018-08-27T23:51:35.0881611Z git version 2.17.1.windows.2
2018-08-27T23:51:35.1556000Z ##[command]git lfs version
2018-08-27T23:51:41.9815469Z git-lfs/2.4.2 (GitHub; windows amd64; go 1.8.3; git 6f4b2e98)
2018-08-27T23:51:42.0222316Z ##[command]git init "D:\a\1\s"
2018-08-27T23:51:43.0792016Z Initialized empty Git repository in D:/a/1/s/.git/
2018-08-27T23:51:43.0860951Z ##[command]git remote add origin https://***@bitbucket.org/***************/***********.git
2018-08-27T23:51:43.1830171Z ##[command]git config gc.auto 0
2018-08-27T23:51:43.2757874Z ##[command]git config --get-all http.https://***@bitbucket.org/***************/***********.git.extraheader
2018-08-27T23:51:43.3103154Z ##[command]git config --get-all http.proxy
2018-08-27T23:51:43.3467229Z ##[command]git remote set-url origin https://***:***@bitbucket.org/***************/***********.git
2018-08-27T23:51:43.3820521Z ##[command]git remote set-url --push origin https://***:***@bitbucket.org/***************/***********.git
2018-08-27T23:51:43.4202596Z ##[command]git fetch --tags --prune --progress --no-recurse-submodules origin
2018-08-28T00:06:39.0151187Z fatal: The remote end hung up unexpectedly
2018-08-28T00:06:39.0172548Z error: RPC failed; HTTP 504 curl 22 The requested URL returned error: 504 Gateway Time-out
2018-08-28T00:06:39.3497463Z ##[error]System.InvalidOperationException: Git fetch failed with exit code: 128
at Agent.Plugins.Repository.GitSourceProvider.<GetSourceAsync>d__11.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Agent.Plugins.Repository.CheckoutTask.<RunAsync>d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Agent.PluginHost.Program.Main(String[] args)
2018-08-28T00:06:39.3595207Z ##[section]Finishing: Get sources
@TingluoHuang, I stopped my agent, deleted the _work folder, and that seemed to fix things.
Thanks @jonreis. Stopping my agent and removing the _work did it for me too ;)
@TingluoHuang Still an issue for me - but i'm using shared hosted agent. (VS2017 agent pool)

Hi @TingluoHuang, it worked for a while then the error started up again
yeah same here @jonreis and @TingluoHuang
I had some luck using the External Git option rather than Bitbucket Cloud when selecting a source. You'll need to create service connection. Looks something like this
Server URL : https://[email protected]/team/repositoryname
Username : username
Password : password (or generate a personal access token like I did)
@frankfuu Using the External Git connection worked for me (at least once, and so far).
I just need to get a build out, but one thing i'm wondering if anyone has tried is to revoke oauth access (assuming that's how you're connected through the bitbucket connection) - and then reauth VSTS?
@trowpa yep, I tried creating a new Oath connection but without luck either.
i am really think something on bitbucket side happened. in my test, i can see the 3 times retry helped.
the exactly command runs 3 times and succeed on latest time it.
2018-08-28T14:14:08.5185650Z ##[section]Starting: Checkout
2018-08-28T14:14:08.5349064Z ==============================================================================
2018-08-28T14:14:08.5350522Z Task : Get sources
2018-08-28T14:14:08.5351808Z Description : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.
2018-08-28T14:14:08.5352352Z Version : 1.0.0
2018-08-28T14:14:08.5353524Z Author : Microsoft
2018-08-28T14:14:08.5354716Z Help : [More Information](https://go.microsoft.com/fwlink/?LinkId=798199)
2018-08-28T14:14:08.5355322Z ==============================================================================
2018-08-28T14:14:10.1572717Z Syncing repository: ***/test-main-repository (Bitbucket)
2018-08-28T14:14:10.1609340Z Prepending Path environment variable with directory containing 'git.exe'.
2018-08-28T14:14:10.1614990Z ##[command]git version
2018-08-28T14:14:10.1619944Z git version 2.17.1.windows.2
2018-08-28T14:14:10.1621792Z ##[command]git lfs version
2018-08-28T14:14:11.9141012Z git-lfs/2.4.0 (GitHub; windows amd64; go 1.8.3; git 1a98fa5c)
2018-08-28T14:14:11.9478584Z ##[command]git init "c:\vsts-agent\_layout\_work\1\s"
2018-08-28T14:14:12.0870563Z Initialized empty Git repository in c:/vsts-agent/_layout/_work/1/s/.git/
2018-08-28T14:14:12.1017831Z ##[command]git remote add origin https://bitbucket.org//***/test-main-repository
2018-08-28T14:14:12.2106210Z ##[command]git config gc.auto 0
2018-08-28T14:14:12.2964675Z ##[command]git config --get-all http.https://bitbucket.org//***/test-main-repository.extraheader
2018-08-28T14:14:12.3755393Z ##[command]git config --get-all http.proxy
2018-08-28T14:14:12.4560622Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" -c http.proxy="http://127.0.0.1:8888" fetch --tags --prune --progress --no-recurse-submodules origin
2018-08-28T14:14:16.3089284Z fatal: The remote end hung up unexpectedly
2018-08-28T14:14:16.3133483Z error: RPC failed; HTTP 405 curl 22 The requested URL returned error: 405 Method Not Allowed
2018-08-28T14:14:16.3365391Z ##[warning]Git fetch failed with exit code 128, back off 8.345 seconds before retry.
2018-08-28T14:14:24.6790699Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" -c http.proxy="http://127.0.0.1:8888" fetch --tags --prune --progress --no-recurse-submodules origin
2018-08-28T14:14:25.2170605Z remote: Repository /*** not found
2018-08-28T14:14:25.2172003Z fatal: repository 'https://bitbucket.org//***/test-main-repository/' not found
2018-08-28T14:14:25.2393861Z ##[warning]Git fetch failed with exit code 128, back off 4.691 seconds before retry.
2018-08-28T14:14:29.9305390Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" -c http.proxy="http://127.0.0.1:8888" fetch --tags --prune --progress --no-recurse-submodules origin
2018-08-28T14:14:31.4537892Z remote: Counting objects: 47, done.
2018-08-28T14:14:31.4539060Z remote: Compressing objects: 2% (1/44)
2018-08-28T14:14:31.4546215Z remote: Compressing objects: 4% (2/44)
2018-08-28T14:14:31.4549968Z remote: Compressing objects: 6% (3/44)
......
2018-08-28T14:14:31.4574238Z remote: Compressing objects: 93% (41/44)
2018-08-28T14:14:31.4574652Z remote: Compressing objects: 95% (42/44)
2018-08-28T14:14:31.4575084Z remote: Compressing objects: 97% (43/44)
2018-08-28T14:14:31.4575499Z remote: Compressing objects: 100% (44/44)
2018-08-28T14:14:31.4575938Z remote: Compressing objects: 100% (44/44), done.
2018-08-28T14:14:31.4576387Z remote: Total 47 (delta 15), reused 0 (delta 0)
2018-08-28T14:14:32.1305154Z From https://bitbucket.org//***/test-main-repository
2018-08-28T14:14:32.1306198Z * [new branch] master -> origin/master
2018-08-28T14:14:32.1306712Z * [new branch] users/tihuang/test1 -> origin/users/tihuang/test1
2018-08-28T14:14:32.1683302Z ##[command]git checkout --progress --force f87931d0e784cf815a84affac82a53184af1871a
2018-08-28T14:14:32.3334234Z Note: checking out 'f87931d0e784cf815a84affac82a53184af1871a'.
2018-08-28T14:14:32.3335237Z
2018-08-28T14:14:32.3335766Z You are in 'detached HEAD' state. You can look around, make experimental
2018-08-28T14:14:32.3336242Z changes and commit them, and you can discard any commits you make in this
2018-08-28T14:14:32.3336722Z state without impacting any branches by performing another checkout.
2018-08-28T14:14:32.3337049Z
2018-08-28T14:14:32.3337480Z If you want to create a new branch to retain commits you create, you may
2018-08-28T14:14:32.3338806Z do so (now or later) by using -b with the checkout command again. Example:
2018-08-28T14:14:32.3339200Z
2018-08-28T14:14:32.3339608Z git checkout -b <new-branch-name>
2018-08-28T14:14:32.3339912Z
2018-08-28T14:14:32.3340403Z HEAD is now at f87931d Revert "Revert "Revert "test"""
2018-08-28T14:14:32.3445766Z ##[command]git submodule sync
2018-08-28T14:14:33.7130417Z ##[command]git -c http.https://bitbucket.org.extraheader="AUTHORIZATION: basic ***" -c http.proxy="http://127.0.0.1:8888" submodule update --init --force
2018-08-28T14:14:34.7297542Z Submodule 'subrepository' (https://bitbucket.org//***/test-subrepository.git) registered for path 'subrepository'
2018-08-28T14:14:37.1631380Z Cloning into 'C:/vsts-agent/_layout/_work/1/s/subrepository'...
2018-08-28T14:14:38.1688137Z Submodule path 'subrepository': checked out '86370c6c014576e0b96be93640ccdd0137bf6d87'
2018-08-28T14:14:38.2173872Z ##[section]Finishing: Checkout
Got the same symptoms here... intermittent connectivity errors with Bitbucket Cloud
Same issue with VSTS Hosted VS2017 agent and BitBucket Clould. Intermittent Git fetch failures on builds.
2018-08-27T20:54:44.0215119Z ##[section]Starting: Get sources
2018-08-27T20:54:44.0497993Z ==============================================================================
2018-08-27T20:54:44.0498260Z Task : Get sources
2018-08-27T20:54:44.0498494Z Description : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.
2018-08-27T20:54:44.0498741Z Version : 1.0.0
2018-08-27T20:54:44.0498905Z Author : Microsoft
2018-08-27T20:54:44.0499103Z Help : [More Information](https://go.microsoft.com/fwlink/?LinkId=798199)
2018-08-27T20:54:44.0499343Z ==============================================================================
2018-08-27T20:54:45.7458347Z Syncing repository: *****/***** (Bitbucket)
2018-08-27T20:54:45.7499117Z Prepending Path environment variable with directory containing 'git.exe'.
2018-08-27T20:54:45.7510409Z ##[command]git version
2018-08-27T20:54:46.8931560Z git version 2.17.1.windows.2
2018-08-27T20:54:47.0251610Z ##[command]git lfs version
2018-08-27T20:54:53.3192743Z git-lfs/2.4.2 (GitHub; windows amd64; go 1.8.3; git 6f4b2e98)
2018-08-27T20:54:53.3483848Z ##[command]git init "D:\a\1\s"
2018-08-27T20:54:54.1422315Z Initialized empty Git repository in D:/a/1/s/.git/
2018-08-27T20:54:54.1508939Z ##[command]git remote add origin https://bitbucket.org//*****/*****
2018-08-27T20:54:54.2751756Z ##[command]git config gc.auto 0
2018-08-27T20:54:54.3579641Z ##[command]git config --get-all http.https://bitbucket.org//*****/*****.extraheader
2018-08-27T20:54:54.3963838Z ##[command]git config --get-all http.proxy
2018-08-27T20:54:54.4401582Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --tags --prune --progress --no-recurse-submodules origin
2018-08-27T20:55:00.9599977Z fatal: The remote end hung up unexpectedly
2018-08-27T20:55:00.9612560Z error: RPC failed; HTTP 405 curl 22 The requested URL returned error: 405 Method Not Allowed
2018-08-27T20:55:00.9763359Z ##[warning]Git fetch failed with exit code 128, back off 2.852 seconds before retry.
2018-08-27T20:55:03.8311241Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --tags --prune --progress --no-recurse-submodules origin
2018-08-27T20:55:04.2220663Z remote: Repository /***** not found
2018-08-27T20:55:04.2221774Z fatal: repository 'https://bitbucket.org//*****/*****/' not found
2018-08-27T20:55:04.2285734Z ##[warning]Git fetch failed with exit code 128, back off 7.452 seconds before retry.
2018-08-27T20:55:11.6814357Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --tags --prune --progress --no-recurse-submodules origin
2018-08-27T20:55:12.2713071Z fatal: The remote end hung up unexpectedly
2018-08-27T20:55:12.2723505Z error: RPC failed; HTTP 405 curl 22 The requested URL returned error: 405 Method Not Allowed
2018-08-27T20:55:14.5677603Z ##[error]System.InvalidOperationException: Git fetch failed with exit code: 128
at Agent.Plugins.Repository.GitSourceProvider.<GetSourceAsync>d__11.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Agent.Plugins.Repository.CheckoutTask.<RunAsync>d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Agent.PluginHost.Program.Main(String[] args)
2018-08-27T20:55:14.6227004Z ##[section]Finishing: Get sources
We have been experiencing the same issue over the last several weeks. Over the last few weeks the issue has been intermittent, were we can try a few times in a row and eventually it will connect properly. However, today it doesn't seem to connect at all.
We are getting the same RPC failed; HTTP 405 curl 22 The requested URL returned error: 405 Method Not Allowed error that others are getting as well as fatal: repository 'https://bitbucket.org/ourrepor/source/src/' not found.
This is highly problematic for collaborative teams. It is definitely something on bitbucket's side.
@MattCarpenter03 are you experience this in VSTS or from git.exe on your dev box?
I am not sure if this is anecdotal, or not, but when I'm having this issue I restart my agent and the issue goes away. Maybe a timing thing with Bitbucket becoming available right then?
My issue is from git.exe on the dev device.
@MattCarpenter03 is you repo remote url contains double /, like https://bitbucket.org//account/name/?
@TingluoHuang it is not.
I just tried to add the double // and the same issue persists.
Something else of note. According to this blog post they are updating their IP address.
(https://blog.bitbucket.org/2018/07/02/new-ip-addresses-bitbucket-cloud/?_ga=2.52060909.536191314.1535479044-715674301.1534970972)
So I ran a packet capture (see attached) It is connecting to 18.205.93.0 Thats strange that they would set their IP to use .0 as that is typically reserved as a broadcast
The domain bitbucket.org is on a block of IP so the IP doesn't seem to be the issue either because it fails on other ip's in the range as well.
I have opened a ticket regarding the issue. I will keep you posted here.
Reopening until it's fully resolved from bitbucket. @MattCarpenter03 - let us know what you find.
We're experiencing the same issue for infocaster.vistualstudio.com since yesterday.
All our builds are failing at the moment, which s*cks ofcourse...
I am also seeing 500 instead of 405 when the url is right (no extra slash), something definitely goes wrong on bitbucket side
2018-08-29T12:12:21.4152592Z ##[command]git remote add origin https://bitbucket.org/***/test-main-repository.git
2018-08-29T12:12:21.4893735Z ##[command]git config gc.auto 0
2018-08-29T12:12:21.5053549Z ##[command]git config --get-all http.https://bitbucket.org/***/test-main-repository.git.extraheader
2018-08-29T12:12:21.5185602Z ##[command]git config --get-all http.proxy
2018-08-29T12:12:21.5317528Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --tags --prune --progress --no-recurse-submodules origin
2018-08-29T12:12:23.5629307Z error: RPC failed; HTTP 500 curl 22 The requested URL returned error: 500 Internal Server Error
2018-08-29T12:12:23.5645773Z fatal: The remote end hung up unexpectedly
2018-08-29T12:12:23.5778019Z ##[warning]Git fetch failed with exit code 128, back off 2.921 seconds before retry.
2018-08-29T12:12:26.4950418Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --tags --prune --progress --no-recurse-submodules origin
2018-08-29T12:12:28.2480843Z remote: Counting objects: 47, done.
2018-08-29T12:12:28.4537894Z remote: Compressing objects: 2% (1/44)
Even the External Git (still fetching from bitbucket repository) option is not working now for self hosted agents. Using hosted agent Hosted VS2017 though still works for External Git as opposed to Bitbucket Cloud so I'm not sure if fault lies entirely on Bitbucket's side.
Here are my logs
2018-08-29T12:39:13.8541524Z ##[section]Starting: Get sources
2018-08-29T12:39:13.8820559Z ==============================================================================
2018-08-29T12:39:13.8820776Z Task : Get sources
2018-08-29T12:39:13.8820932Z Description : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.
2018-08-29T12:39:13.8821064Z Version : 1.0.0
2018-08-29T12:39:13.8821174Z Author : Microsoft
2018-08-29T12:39:13.8821327Z Help : [More Information](https://go.microsoft.com/fwlink/?LinkId=798199)
2018-08-29T12:39:13.8821480Z ==============================================================================
2018-08-29T12:39:16.0704421Z Syncing repository: repository (ExternalGit)
2018-08-29T12:39:16.0739560Z Prepending Path environment variable with directory containing 'git.exe'.
2018-08-29T12:39:16.0746549Z ##[command]git version
2018-08-29T12:39:17.1072296Z git version 2.17.1.windows.2
2018-08-29T12:39:17.1358607Z ##[command]git lfs version
2018-08-29T12:39:23.2271990Z git-lfs/2.4.2 (GitHub; windows amd64; go 1.8.3; git 6f4b2e98)
2018-08-29T12:39:23.2594213Z ##[command]git init "D:\a\1\s"
2018-08-29T12:39:24.0504177Z Initialized empty Git repository in D:/a/1/s/.git/
2018-08-29T12:39:24.0564710Z ##[command]git remote add origin https://***@bitbucket.org/testrepo/testrepo.git
2018-08-29T12:39:24.1877250Z ##[command]git config gc.auto 0
2018-08-29T12:39:24.3561804Z ##[command]git config --get-all http.https://***@bitbucket.org/testrepo/testrepo.git.extraheader
2018-08-29T12:39:24.3898081Z ##[command]git config --get-all http.proxy
2018-08-29T12:39:24.4233018Z ##[command]git remote set-url origin https://***:***@bitbucket.org/testrepo/testrepo.git
2018-08-29T12:39:24.4572398Z ##[command]git remote set-url --push origin https://***:***@bitbucket.org/testrepo/testrepo.git
2018-08-29T12:39:24.4956948Z ##[command]git fetch --tags --prune --progress --no-recurse-submodules origin
2018-08-29T12:39:39.1382534Z remote: Counting objects: 1
2018-08-29T12:39:39.1383054Z remote: Counting objects: 262115, done.
2018-08-29T12:39:44.1380695Z remote: Compressing objects: 0% (1/55941)
2018-08-29T12:39:44.1381220Z remote: Compressing objects: 1% (560/55941)
2018-08-29T12:39:44.1381760Z remote: Compressing objects: 2% (1119/55941)
2018-08-29T12:39:44.1381932Z remote: Compressing objects: 3% (1679/55941)
2018-08-29T12:39:44.1382080Z remote: Compressing objects: 4% (2238/55941)
2018-08-29T12:39:44.1382264Z remote: Compressing objects: 5% (2798/55941)
Hi @mark-adams. We met at GopherCon yesterday. Do you know someone from Bitbucket Cloud who can help us resolve this issue with HTTP 405 and 500 while VSTS fetches code from Bitbucket Cloud?
There is a bug in the bug tracker of bitbucket. https://bitbucket.org/site/master/issues/17066/bitbucket-connection-to-vsts there is mention of an support ticket that is probably no public
I opened a ticket with Bitbucket Clould yesterday and received this response this morning:
Pravin Mahendran commented:
Hi Jay,
Thank you for contacting Bitbucket Cloud Support
I believe the cause of this issue is due to VSTS appending a double slash to the remote URL. Therefore when calling to Bitbucket, we are not able to locate the repository with the double slash.
We are reviewing this issue internally to further investigate the cause of this issue.
Appreciate your patience while we review this issue further.
Please do let us know if you have any questions.
Thanks & Regards,
Pravin
Same here, intermittent issues started 2 days ago.
At this point, VSTS has already stopped sending extra / to bitbucket, can anyone confirm whether they are still seeing intermittent failure?
Wrong, the extra / is still sent in our account.
@kipusoep what's your account name?
https://github.com/Microsoft/vsts-agent/issues/1778#issuecomment-416865588 ;-)
Thanks @DavidStaheli!
I'll reach out to the BB team internally and see what I can find out
the mitigation to remove the extra / got revert last night, i have re-apply the mitigation just now, can anyone confirm whether they are still seeing the extra /?
@TingluoHuang I just ran a couple of successful builds and it seems the extra / is not there.
@mark-adams - one good thing to understand from BB side is why extra slash was working, then intermittent and now failing all the time. My guess is there was a changing rolling out on the BB side. Nevertheless, the fix to never have the extra slash is rolling out on our side. The intermittent 500 seems like an orthogonal issue. If there's anything we can help with, let us know.
This is rolled out on our side. If anyone has a 405 failure, let us know.
Works! :-)
Cool. Closing for now. If anyone sees a 405. let us know.
Thanks Bryan,
I pinged the team to see if I can get some more details for you.
On Wed, Aug 29, 2018, 11:40 Bryan MacFarlane notifications@github.com
wrote:
@mark-adams https://github.com/mark-adams - one good thing to
understand from BB side is why extra slash was working, then intermittent
and now failing all the time. My guess is there was a changing rolling
out on the BB side. Nevertheless, the fix to never have the extra slash is
rolling out on our side. The intermittent 500 seems like an orthogonal
issue. If there's anything we can help with, let us know.—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/Microsoft/vsts-agent/issues/1778#issuecomment-417041064,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAc1Hs38kiXHcDiqHHDQ3V11vfTwARRRks5uVtIEgaJpZM4WIcta
.
Bitbucket dev here - @bryanmacfarlane is spot on. We have been going through the process of rolling our our new Git over HTTPS service for the past month or so and the new service is strict(er) with regards to URL's than the old service. When this new service was handling <50% of requests this problem would present itself as an intermittent failure since a retry was likely to hit a legacy node and work fine. For an ~18hr window we spun the new service up to 100% of traffic, which meant it affected every request.
Coincidentally one of our ALB's started to get throttled at peak service hours causing 500's and increased latency on top of the URL handling issues. We have spun the new service back down and are working on relaxing the URL pattern matching to fix this issue on our end going forward. Thanks for patching it on your end so quickly!
I am getting a 405 when trying to clone a Bitbucket repo via VSTS:
Git failed with a fatal error.
unable to access 'https://bitbucket.org/*/*/': The requested URL returned error: 405
Any suggestions? I've made sure the URL matches exactly.
@Giterdone11 can you clone the repo manually from command line locally?
@TingluoHuang I realized from command line this solution is mercurial and not git. Thanks for the tip.
Most helpful comment
Bitbucket dev here - @bryanmacfarlane is spot on. We have been going through the process of rolling our our new Git over HTTPS service for the past month or so and the new service is strict(er) with regards to URL's than the old service. When this new service was handling <50% of requests this problem would present itself as an intermittent failure since a retry was likely to hit a legacy node and work fine. For an ~18hr window we spun the new service up to 100% of traffic, which meant it affected every request.
Coincidentally one of our ALB's started to get throttled at peak service hours causing 500's and increased latency on top of the URL handling issues. We have spun the new service back down and are working on relaxing the URL pattern matching to fix this issue on our end going forward. Thanks for patching it on your end so quickly!