Glide: No build available for linux AMD (Travis CI)

[jraede]

This started happening a few days ago. When running curl https://glide.sh/get | sh in a TravisCI build I get the following output. Seems like there is an issue with the release tagging or the glide.sh/get script.

$ go version
go version go1.7 linux/amd64
go.env
$ go env
GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/travis/gopath"
GORACE=""
GOROOT="/home/travis/.gimme/versions/go1.7.linux.amd64"
GOTOOLDIR="/home/travis/.gimme/versions/go1.7.linux.amd64/pkg/tool/linux_amd64"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build194054222=/tmp/go-build"
CXX="g++"
CGO_ENABLED="1"
install.1
0.00s$ mkdir $GOPATH/bin
0.56s$ curl https://glide.sh/get | sh
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  3241  100  3241    0     0  11185      0 --:--:-- --:--:-- --:--:-- 17518
Sorry, we dont have a dist for your system: linux amd64

[franciscocpg]

I just run it now and here is my output

curl https://glide.sh/get | sh
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  3241  100  3241    0     0   4721      0 --:--:-- --:--:-- --:--:--  4717
Downloading https://github.com/Masterminds/glide/releases/download/v0.12.3/glide-v0.12.3-linux-amd64.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   608    0   608    0     0    636      0 --:--:-- --:--:-- --:--:--   636
100 3344k  100 3344k    0     0   406k      0  0:00:08  0:00:08 --:--:--  582k
glide version v0.12.3 installed succesfully

Maybe you can download the get script and then add some debug information in the method downloadFile.

[kseo]

I also encountered the same problem.

$ mkdir -p $GOPATH/bin
0.49s$ curl https://glide.sh/get | sh
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  3241  100  3241    0     0  16245      0 --:--:-- --:--:-- --:--:-- 17906
Sorry, we dont have a dist for your system: linux amd64
You can ask one here: https://github.com/Masterminds/glide/issues
Fail to install glide

[jraede]

Seems to be sporadic. I copied the file as-is, added some debug lines, changed the travis.yml to run it locally, and committed/pushed, and then it worked. Immediately restarted the successful build and got the same error as above. Might thus be a travis issue with some sort of caching rather than an issue with glide. Will keep this issue updated as I learn more.

[jraede]

Scratch the above, I am not getting that same issue. Since committing the contents of glide.sh/get into my repo, I have not had a problem. Is there an issue with the glide.sh server? In both cases we still download the release from Github so that seems to rule that out as a potential culprit.

[jraede]

And now I had the same issue even with the file committed. I might be going insane. Anyway, that means that there could be an intermittent issue with the github API returning the list of releases.

[sdboyer]

My first would be that you're running into GitHub's anonymous request rate limiting. Since you've already committed the script, if you modify it so that HTTP header information is printed to the console, you'd be able to verify that that's what's actually causing the failures when the next one occurs.

[franciscocpg]

I think @sdboyer suggestion could be valid.
Printing this LATEST_RELEASE_JSON=$(curl -s "$LATEST_RELEASE_URL") and adding a verbose flag to curl (curl -sv "$LATEST_RELEASE_URL") should help to understand what's going wrong.
Besides that, I think the script can be improved to check the http status code and then print the correct error information to the user.

[franciscocpg]

This PR https://github.com/Masterminds/glide.sh/pull/10 has some improvements about catching erros.
After merged this should show a better output when errors with http calls occurs.
About this issue, like @sdboyer mentioned, github api has rate limits. This sequence of commands shows it:

curl -s https://api.github.com/rate_limit
curl -s https://api.github.com/repos/Masterminds/glide/releases/tags/v0.12.3
curl -s https://api.github.com/rate_limit

Take a look at key rate.limit, it is decreasing.
My suggestion when using a CI tool is to install with apt-get install glide or build with a docker image like rest4hub/golang-glide.

@mattfarina, On the glide side, we are making the call to the github API just to get the latest download URL. Maybe you should publish this URL to the https://github.com/Masterminds/glide.sh repository (like you do with version for example) so glide get script doesn't have to deal with this github API rate limits.

[franciscocpg]

Example adding ppa to travis: https://docs.travis-ci.com/user/installing-dependencies/#Installing-Packages-from-a-custom-APT-repository
So it would be something like this

before_install:
  - sudo add-apt-repository ppa:masterminds/glide -y
  - sudo apt-get update -q
  - sudo apt-get install glide -y

[mattfarina]

I think @sdboyer is right about the API limit. they've removed the limit for some requests but it's likely better to just put the data on glide.sh as @franciscocpg suggested.

[jraede]

RE: adding PPA to Travis, that requires the "sudo" directive which doesn't allow you to use their new container-based (faster) infrastructure. I think bypassing github to get the releases directly in glide.sh will cover the most edge cases here.

[franciscocpg]

@jraede, also building with a docker image requires sudo too, :(.
Someone already requested adding glide to container apt source whitelist here https://github.com/travis-ci/apt-source-whitelist/issues/318.
But it looks like this can take months to happen.

[franciscocpg]

@jraede, I don't know if you already have a workaround, but according to https://docs.travis-ci.com/user/migrating-from-legacy/#How-Do-I-Install-Custom-Software%3F this should works by now.

before_script:
  - wget https://github.com/Masterminds/glide/releases/download/v0.12.3/glide-v0.12.3-darwin-amd64.tar.gz -O /tmp/glide-v0.12.3-darwin-amd64.tar.gz
  - tar -xvf /tmp/glide-v0.12.3-darwin-amd64.tar.gz
  - export PATH=$PATH:$PWD/darwin-amd64/

Again, this is just a temporary solution while not bypassing github API.

[seh]

Supposedly a forthcoming version of Travis CI does away with the APT source white list altogether, which explains the sluggishness in updating it. However, there isn't anything we as users/customers can do to usher in that better tomorrow any sooner.

[jmazzitelli]

Add me to the list of hitting this problem as well. I see a few suggestions in here, but is there a recommended workaround we can use today?

[franciscocpg]

@jmazzitelli, if you are using travis https://github.com/Masterminds/glide/issues/639#issuecomment-255521986

[jmazzitelli]

Ok. Thanks @franciscocpg - I tweaked that a bit so I didn't hardcode to a fixed version of Glide or to the OS/Arch. This is what I have in my .travis.yml and it seems to work

before_install:
  - GLIDE_OS_ARCH=`go env GOHOSTOS`-`go env GOHOSTARCH`
  - GLIDE_TAG=$(curl -s https://glide.sh/version)
  - GLIDE_LATEST_RELEASE_URL="https://github.com/Masterminds/glide/releases/download/${GLIDE_TAG}/glide-${GLIDE_TAG}-${GLIDE_OS_ARCH}.tar.gz"
  - wget ${GLIDE_LATEST_RELEASE_URL} -O /tmp/glide.tar.gz
  - mkdir /tmp/glide
  - tar --directory=/tmp/glide -xvf /tmp/glide.tar.gz
  - export PATH=$PATH:/tmp/glide/${GLIDE_OS_ARCH}

[alexjomin]

@jmazzitelli works fine, thank you !

[sagikazarmark]

I wonder why the API call is necessary in the first place. You could just check if the generated download URL returns anything and fail if not. I can't think of any extra overhead, since this would replace the API call.

[sagikazarmark]

I made a gist out of it https://gist.github.com/sagikazarmark/b61ffdc0a6520d5d405b15fbef37e61d

/cc @jmazzitelli

[skipor]

before_install:
  - sudo add-apt-repository ppa:masterminds/glide -y
  - sudo apt-get update -q
  - sudo apt-get install glide -y

Works now.

[andytom]

You can also use the apt addon which works on the container infrastructure.

addons:
  apt:
    sources:
      - sourceline: 'ppa:masterminds/glide'
    packages:
      - glide