Azure-pipelines-tasks: Android Build failing, unable to download from Google Repository

[mgj]

Troubleshooting

Checkout how to troubleshoot failures and collect debug logs: https://docs.microsoft.com/en-us/vsts/build-release/actions/troubleshooting

Environment

• Server - VSTS or TFS on-premises?
  VSTS

• Agent - Hosted or Private:
  Hosted

• If using Hosted agent, provide agent queue name:
  "Hosted" and "Hosted VS2017"

Issue Description

Unable to build the same commit that was building yesterday. Seems like a duplicate of #568
I'm currently seeing this using com.android.tools.build:gradle:3.1.0 and com.google.gms:google-services:3.1.1 when building on Hosted VS2017 or Hosted

Error logs

Download https://maven.google.com/com/android/tools/annotations/26.1.0/annotations-26.1.0.pom
FAILURE: Build failed with an exception.
* What went wrong:
A problem occurred configuring root project 'Carlog'.
> Could not resolve all files for configuration ':classpath'.
   > Could not resolve com.google.gms:google-services:3.1.1.
     Required by:
         project :
      > Could not resolve com.google.gms:google-services:3.1.1.
         > Could not get resource 'https://jcenter.bintray.com/com/google/gms/google-services/3.1.1/google-services-3.1.1.pom'.
            > Could not GET 'https://jcenter.bintray.com/com/google/gms/google-services/3.1.1/google-services-3.1.1.pom'. Received status code 502 from server: Bad Gateway
   > Could not resolve org.jetbrains.kotlin:kotlin-stdlib-jre8:1.2.0.
     Required by:
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:gradle-core:3.1.0
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:gradle-core:3.1.0 > com.android.tools.build:builder:3.1.0
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:gradle-core:3.1.0 > com.android.tools.lint:lint-gradle-api:26.1.0
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:gradle-core:3.1.0 > com.android.tools.build:gradle-api:3.1.0
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:gradle-core:3.1.0 > com.android.databinding:compilerCommon:3.1.0
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:gradle-core:3.1.0 > com.android.tools.build:builder:3.1.0 > com.android.tools:sdk-common:26.1.0
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:gradle-core:3.1.0 > com.android.tools.build:builder:3.1.0 > com.android.tools.build:manifest-merger:26.1.0
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:gradle-core:3.1.0 > com.android.tools.build:builder:3.1.0 > com.android.tools:sdklib:26.1.0 > com.android.tools:repository:26.1.0
      > Skipped due to earlier error
   > Could not resolve org.ow2.asm:asm:5.1.
     Required by:
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:gradle-core:3.1.0
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:gradle-core:3.1.0 > com.android.tools.build:builder:3.1.0
      > Skipped due to earlier error
   > Could not resolve org.ow2.asm:asm-analysis:5.1.
     Required by:
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:gradle-core:3.1.0
      > Skipped due to earlier error
   > Could not resolve org.ow2.asm:asm-commons:5.1.
     Required by:
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:gradle-core:3.1.0
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:gradle-core:3.1.0 > com.android.tools.build:builder:3.1.0
      > Skipped due to earlier error
   > Could not resolve org.ow2.asm:asm-util:5.1.
     Required by:
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:gradle-core:3.1.0
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:gradle-core:3.1.0 > com.android.tools.build:builder:3.1.0
      > Skipped due to earlier error
   > Could not resolve net.sf.jopt-simple:jopt-simple:4.9.
     Required by:
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:gradle-core:3.1.0
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:gradle-core:3.1.0 > com.android.tools.build:builder:3.1.0
      > Skipped due to earlier error
   > Could not resolve net.sf.proguard:proguard-gradle:5.3.3.
     Required by:
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:gradle-core:3.1.0
      > Skipped due to earlier error
   > Could not resolve com.google.auto.value:auto-value:1.5.2.
     Required by:
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:bundletool:0.1.0-alpha01
      > Skipped due to earlier error
   > Could not resolve com.google.guava:guava:22.0.
     Required by:
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:bundletool:0.1.0-alpha01
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:gradle-core:3.1.0 > com.android.tools.lint:lint-gradle-api:26.1.0
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:gradle-core:3.1.0 > com.android.tools.build:gradle-api:3.1.0
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:gradle-core:3.1.0 > com.android.databinding:compilerCommon:3.1.0
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:gradle-core:3.1.0 > com.android.tools.build:builder:3.1.0 > com.android.tools:common:26.1.0
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:gradle-core:3.1.0 > com.android.tools.build:builder:3.1.0 > com.android.tools.analytics-library:shared:26.1.0
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:gradle-core:3.1.0 > com.android.tools.build:builder:3.1.0 > com.android.tools.analytics-library:tracker:26.1.0
      > Skipped due to earlier error
   > Could not resolve com.google.protobuf:protobuf-java:3.4.0.
     Required by:
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:bundletool:0.1.0-alpha01
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:bundletool:0.1.0-alpha01 > com.android.tools.build:aapt2-proto:0.1.0
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:gradle-core:3.1.0 > com.android.tools.build:builder:3.1.0 > com.android.tools:sdk-common:26.1.0
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:gradle-core:3.1.0 > com.android.tools.build:builder:3.1.0 > com.android.tools.analytics-library:protos:26.1.0
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:gradle-core:3.1.0 > com.android.tools.build:builder:3.1.0 > com.android.tools.analytics-library:tracker:26.1.0
      > Skipped due to earlier error
   > Could not resolve com.google.protobuf:protobuf-java-util:3.4.0.
     Required by:
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:bundletool:0.1.0-alpha01
      > Skipped due to earlier error
   > Could not resolve com.google.errorprone:error_prone_annotations:2.1.2.
     Required by:
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:bundletool:0.1.0-alpha01
      > Skipped due to earlier error
   > Could not resolve com.squareup:javawriter:2.5.0.
     Required by:
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:gradle-core:3.1.0 > com.android.tools.build:builder:3.1.0
      > Skipped due to earlier error
   > Could not resolve org.bouncycastle:bcpkix-jdk15on:1.56.
     Required by:
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:gradle-core:3.1.0 > com.android.tools.build:builder:3.1.0
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:gradle-core:3.1.0 > com.android.tools.build:builder:3.1.0 > com.android.tools:sdk-common:26.1.0
      > Skipped due to earlier error
   > Could not resolve org.bouncycastle:bcprov-jdk15on:1.56.
     Required by:
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:gradle-core:3.1.0 > com.android.tools.build:builder:3.1.0
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:gradle-core:3.1.0 > com.android.tools.build:builder:3.1.0 > com.android.tools:sdk-common:26.1.0
      > Skipped due to earlier error
   > Could not resolve org.ow2.asm:asm-tree:5.1.
     Required by:
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:gradle-core:3.1.0 > com.android.tools.build:builder:3.1.0
      > Skipped due to earlier error
   > Could not resolve it.unimi.dsi:fastutil:7.2.0.
     Required by:
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:gradle-core:3.1.0 > com.android.tools.build:builder:3.1.0
      > Skipped due to earlier error
   > Could not resolve com.googlecode.json-simple:json-simple:1.1.
     Required by:
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:gradle-core:3.1.0 > com.android.tools.build:builder:3.1.0
      > Skipped due to earlier error
   > Could not resolve org.jetbrains.kotlin:kotlin-reflect:1.2.0.
     Required by:
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:gradle-core:3.1.0 > com.android.tools.lint:lint-gradle-api:26.1.0
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:gradle-core:3.1.0 > com.android.tools.build:builder:3.1.0 > com.android.tools:sdk-common:26.1.0
      > Skipped due to earlier error
   > Could not resolve org.antlr:antlr4:4.5.3.
     Required by:
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:gradle-core:3.1.0 > com.android.databinding:compilerCommon:3.1.0
      > Skipped due to earlier error
   > Could not resolve commons-io:commons-io:2.4.
     Required by:
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:gradle-core:3.1.0 > com.android.databinding:compilerCommon:3.1.0
      > Skipped due to earlier error
   > Could not resolve com.googlecode.juniversalchardet:juniversalchardet:1.0.3.
     Required by:
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:gradle-core:3.1.0 > com.android.databinding:compilerCommon:3.1.0
      > Skipped due to earlier error
   > Could not resolve com.squareup:javapoet:1.8.0.
     Required by:
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:gradle-core:3.1.0 > com.android.databinding:compilerCommon:3.1.0
      > Skipped due to earlier error
   > Could not resolve com.google.code.gson:gson:2.7.
     Required by:
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:gradle-core:3.1.0 > com.android.databinding:compilerCommon:3.1.0
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:gradle-core:3.1.0 > com.android.tools.build:builder:3.1.0 > com.android.tools:sdklib:26.1.0
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:gradle-core:3.1.0 > com.android.tools.build:builder:3.1.0 > com.android.tools.build:manifest-merger:26.1.0
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:gradle-core:3.1.0 > com.android.tools.build:builder:3.1.0 > com.android.tools.analytics-library:shared:26.1.0
      > Skipped due to earlier error
   > Could not resolve org.apache.commons:commons-compress:1.12.
     Required by:
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:gradle-core:3.1.0 > com.android.tools.build:builder:3.1.0 > com.android.tools:sdklib:26.1.0
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:gradle-core:3.1.0 > com.android.tools.build:builder:3.1.0 > com.android.tools:sdklib:26.1.0 > com.android.tools:repository:26.1.0
      > Skipped due to earlier error
   > Could not resolve org.apache.httpcomponents:httpclient:4.2.6.
     Required by:
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:gradle-core:3.1.0 > com.android.tools.build:builder:3.1.0 > com.android.tools:sdklib:26.1.0
      > Skipped due to earlier error
   > Could not resolve org.apache.httpcomponents:httpmime:4.1.
     Required by:
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:gradle-core:3.1.0 > com.android.tools.build:builder:3.1.0 > com.android.tools:sdklib:26.1.0
      > Skipped due to earlier error
   > Could not resolve net.sf.kxml:kxml2:2.3.0.
     Required by:
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:gradle-core:3.1.0 > com.android.tools.build:builder:3.1.0 > com.android.tools.build:manifest-merger:26.1.0
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:gradle-core:3.1.0 > com.android.tools.build:builder:3.1.0 > com.android.tools.ddms:ddmlib:26.1.0
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:gradle-core:3.1.0 > com.android.tools.build:builder:3.1.0 > com.android.tools:sdklib:26.1.0 > com.android.tools.layoutlib:layoutlib-api:26.1.0
      > Skipped due to earlier error
   > Could not resolve org.jetbrains:annotations:13.0.
     Required by:
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:gradle-core:3.1.0 > com.android.tools.build:builder:3.1.0 > com.android.tools:sdklib:26.1.0 > com.android.tools.layoutlib:layoutlib-api:26.1.0
      > Skipped due to earlier error
   > Could not resolve com.sun.activation:javax.activation:1.2.0.
     Required by:
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:gradle-core:3.1.0 > com.android.tools.build:builder:3.1.0 > com.android.tools:sdklib:26.1.0 > com.android.tools:repository:26.1.0
      > Skipped due to earlier error
   > Could not resolve org.glassfish.jaxb:jaxb-runtime:2.2.11.
     Required by:
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:gradle-core:3.1.0 > com.android.tools.build:builder:3.1.0 > com.android.tools:sdklib:26.1.0 > com.android.tools:repository:26.1.0
      > Skipped due to earlier error
   > Could not resolve com.google.jimfs:jimfs:1.1.
     Required by:
         project : > com.android.tools.build:gradle:3.1.0 > com.android.tools.build:gradle-core:3.1.0 > com.android.tools.build:builder:3.1.0 > com.android.tools:sdklib:26.1.0 > com.android.tools:repository:26.1.0
      > Skipped due to earlier error
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
BUILD FAILED in 1m 54s

[brcrista]

The problem is in the log. https://jcenter.bintray.com/com/google/gms/google-services/3.1.1/google-services-3.1.1.pom is returning 502.

[madhurig]

Our investigation determined that this was not specific to VSTS Hosted Build VMs and happens on other machines too. But the issue is amplified on the Hosted build VMs since we start with a clean VM each time. On a private build agent or developer machine, maven will cache the dependencies, so they don’t need to be downloaded from the server every time.

Thanks,
Madhuri