Incubator-mxnet: Clojure test failure: Could not transfer artifact com.fasterxml.jackson.core

[haojin2]

Encountered at http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Funix-cpu/detail/PR-16597/4/pipeline/ :
Could not transfer artifact com.fasterxml.jackson.core:jackson-core:jar:2.9.0 from/to central (https://repo1.maven.org/maven2/): GET request of: com/fasterxml/jackson/core/jackson-core/2.9.0/jackson-core-2.9.0.jar from central failed
Could not find artifact com.fasterxml.jackson.core:jackson-core:jar:2.9.0 in clojars (https://repo.clojars.org/)
This could be due to a typo in :dependencies, file system permissions, or network issues.
If you are behind a proxy, try setting the 'http_proxy' environment variable.
@gigasquid Is this due to the flakiness of package serving infrastructure?

[gigasquid]

This is a flaky package serving infrastructure problem getting a well known library from maven central https://mvnrepository.com/artifact/com.fasterxml.jackson.core/jackson-core

[gigasquid]

This type of error has also happened with other Scala java dependencies that are pulled. I'm not sure what can be changed to alleviate it other than maybe separating out any dependency fetching call and putting retries around it, but the solution should be considered holistically through all the MXNet fetches

[gigasquid]

@lanking520 Do you have any ideas to make the jar dependency fetching more robust for Scala/ Clojure from maven? Maybe we should make a different issue to track these more globally and close this one?

[lanking520]

@gigasquid I guess that will be more of a CI problem? Most of the cases is like the network reset @marcoabreu have you faced the network problem before? Is there any good solution to solve it?

[marcoabreu]

Are there multiple maven repositories with different SLAs? I'm thinking that we might be hitting an endpoint which is not highly available. Or is there a possibility to define mirrors as fallback?

[gigasquid]

Maven central is now part of the apache infrastructure I think. I don't believe it has an SLA - there are times that it has trouble https://status.maven.org/
There do appear to be mirrors https://stackoverflow.com/questions/5233610/what-are-the-official-mirrors-of-the-maven-central-repository

[gigasquid]

Mirrors can be set in leiningen (the clojure side) and also in the maven pom.xmls for Scala https://maven.apache.org/guides/mini/guide-mirror-settings.html.

It would be a good chunk of work to research and implement the mirroring. I vote that we see if it happens more often with Clojure/Java/Scala jars before taking action on it.

[marcoabreu]

As far as I understand this wouldn't be in the application pom.xml but rather in the .m2/settings.xml of the Jenkins slave - thus something that we can do on the infrastructure side by adding it to the docker container. It should be a matter of adding the settings.xml in the install scala.sh file.

[gigasquid]

@marcoabreu If that's true, then it will get "work" for Clojure too, since it's built on the Scala image and it uses .m2/settings.xml

[samskalicky]

@lanking520 assign @larroy
Pedro please add this to the backlog for CI enhancements to download dependencies in advance and cache

[ChaiBapchya]

Similar one here for slingshot (clojure jar) for unrelated PR - https://github.com/apache/incubator-mxnet/pull/16894

Could not transfer artifact slingshot:slingshot:pom:0.12.2 from/to clojars (https://repo.clojars.org/): Failed to transfer file: https://repo.clojars.org/slingshot/slingshot/0.12.2/slingshot-0.12.2.pom. Return code is: 503 , ReasonPhrase:Backend unavailable, connection timeout.

This could be due to a typo in :dependencies, file system permissions, or network issues.

If you are behind a proxy, try setting the 'http_proxy' environment variable.

2019-11-25 01:56:51,849 - root - INFO - Waiting for status of container 824684cbe67d for 600 s.

2019-11-25 01:56:52,064 - root - INFO - Container exit status: {'Error': None, 'StatusCode': 1}

2019-11-25 01:56:52,065 - root - ERROR - Container exited with an error 😞

[larroy]

ok

[ChaiBapchya]

Connection timeout. Unrelated PR : #18206
http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Funix-cpu/detail/PR-18206/6/pipeline

Could not transfer artifact org.clojure:tools.analyzer:jar:0.6.9 from/to central (https://repo1.maven.org/maven2/): Connect to repo1.maven.org:443 [repo1.maven.org/151.101.52.209] failed: Connection timed out (Connection timed out)

[2020-05-05T19:08:36.663Z] Could not find artifact org.clojure:tools.analyzer:jar:0.6.9 in clojars (https://repo.clojars.org/)

[2020-05-05T19:08:36.663Z] This could be due to a typo in :dependencies, file system permissions, or network issues.