Hub: Problem with the linux binaries on Alpine

[philwinder]

Hi there. I wasn't able to download and install the linux binaries properly on Alpine linux.

I've created a Dockerfile to recreate the problem.

FROM alpine:3.8

RUN apk add --no-cache bash

RUN wget --quiet https://github.com/github/hub/releases/download/v2.5.0/hub-linux-amd64-2.5.0.tgz -O hub.tgz ; \
    mkdir hub ; \
    tar -xvzf hub.tgz --directory hub --strip-components=1 ; \
    ./hub*/install

RUN hub

If you build this then you will get:

docker build -t temp .
Sending build context to Docker daemon  2.048kB
Step 1/4 : FROM alpine:3.8
 ---> 11cd0b38bc3c
Step 2/4 : RUN apk add --no-cache bash
 ---> Using cache
 ---> b3d1d700abe2
Step 3/4 : RUN wget --quiet https://github.com/github/hub/releases/download/v2.5.0/hub-linux-amd64-2.5.0.tgz -O hub.tgz ;     mkdir hub ;     tar -xvzf hub.tgz --directory hub --strip-components=1 ;     ./hub*/install
 ---> Running in a62f8f93e4e4
hub-linux-amd64-2.5.0/

...

hub-linux-amd64-2.5.0/install
Removing intermediate container a62f8f93e4e4
 ---> d842b5922714
Step 4/4 : RUN hub
 ---> Running in 83fcf019ef52
/bin/sh: hub: not found
The command '/bin/sh -c hub' returned a non-zero code: 127

As a workaround, I found that if I used the hub package in the alpine testing branch, that works. I.e.

RUN echo "http://dl-cdn.alpinelinux.org/alpine/edge/testing" >> /etc/apk/repositories
RUN apk add --no-cache \
    git=2.18.0-r0 \
    bash=4.4.19-r1 \
    hub=2.4.0-r0

Thanks!

[mislav]

Hi I can't help you with linux commands and making docker containers. You'll have to figure it out on your own.

For what it's worth, this script installs hub into /usr/local/bin/hub:

mkdir hub && \
wget https://github.com/github/hub/releases/download/v2.5.0/hub-linux-amd64-2.5.0.tgz -O- | \
tar xz --strip-components 1 --directory hub && \
./hub/install && rm -r hub

You can change the installation location with PREFIX=/usr when running the install script. The $PREFIX/bin directory should be in your PATH before you can run hub.

[philwinder]

Thanks for the response. You can see that your script is the same as my script that I provided in the "how to recreate the problem" code. If you run that, the install script doesn't work.

I can appreciate that you want to prune issues. But I would suggest that not being able to install on Linux is a problem, considering that many of your users are on Linux.

Your install script, or the install instructions, are broken.

[mislav]

I'm sorry for my terse reply. I'm aware that my installation instructions are almost identical to yours. But I must assure you, the installation script works:

FROM alpine:3.8

RUN apk add --no-cache bash

RUN mkdir hub && \
    wget https://github.com/github/hub/releases/download/v2.5.0/hub-linux-amd64-2.5.0.tgz -O- | \
    tar xz --strip-components 1 --directory hub && \
    ./hub/install && rm -r hub

CMD ls -l /usr/local/bin/hub

hub was installed to PREFIX, which is /usr/local by default:

$ docker build -t hub-test .
$ docker run -t hub-test
-rwxr-xr-x    1 root     root       8579250 Aug  1 10:50 /usr/local/bin/hub

However, attempting to run the hub executable fails with a misleading error that you've experienced, even if you provide the full path to the executable:

$ docker run -t hub-test /usr/local/bin/hub
standard_init_linux.go:190: exec user process caused "no such file or directory"

This is because Alpine can't execute hub precompiled binaries, which I suspect is due to its architecture being very different from other linux. Preliminary searches for cross-compiling Go binaries for Alpine yield results that suggest specific workarounds, most of which come down to actually compiling the project on Alpine itself. As I said, I can't help with that. The binaries that we provide and the install script work on a generic flavor of linux, but they're not guaranteed to work on all linux. Making hub work for you on Alpine is something that, for now, you'll have to figure out how to build, and you're welcome to share instructions once successful. 🙇

[philwinder]

Ah interesting. Ok, thanks for that. I guess it depends what you're using to compile. I've compiled go projects in the past and they've worked fine.

The workaround I'm using at the moment is listed at the top. It's using an alpine package provided thanks to a user called Roberto Oliveira.

[mislav]

I've compiled hub in an golang:alpine container and copied the build product into a fresh alpine container. The result is a 28MB image:

FROM golang:alpine as builder

RUN apk add --no-cache bash

COPY . $GOPATH/src/github.com/github/hub/

WORKDIR $GOPATH/src/github.com/github/hub/

RUN ./script/build -o bin/hub

RUN bash < ./script/install.sh

FROM alpine:3.8

RUN apk add --no-cache git

COPY --from=builder /usr/local/bin/hub /usr/local/bin/hub
COPY --from=builder /usr/local/share/man/man1/hub*.1 /usr/local/share/man/man1/

$ docker run -t hub-test hub version
git version 2.18.0
hub version 2.5.0

[andyneff]

@mislav it's a glibc vs musl issue

Most go binary projects I've seen (gosu for example) compile binary in such a way that they are statically linked. For some reason, the hub executable is not statically linked, and thus needs glibc. ldd shows this.

docker run -it --rm alpine:3.8

apk add --no-cache wget
wget https://github.com/tianon/gosu/releases/download/1.10/gosu-amd64
chmod 755 gosu-amd64
ldd gosu-amd64
./gosu-amd64

Shows

/ # ldd gosu-amd64
ldd: gosu-amd64: Not a valid dynamic program

But that one gosu exectuable works on both glibc and musl linux alike.

For some reason, the hub downloaded from the release page shows this

/hub/bin # ldd hub
        /lib64/ld-linux-x86-64.so.2 (0x7fe2c974c000)
        libpthread.so.0 => /lib64/ld-linux-x86-64.so.2 (0x7fe2c974c000)
        libc.so.6 => /lib64/ld-linux-x86-64.so.2 (0x7fe2c974c000)

Unfortunately, I don't know about golang to explain how you fix it. But if you start statically linking the hub executable, then the release downloads will work for both.

[philwinder]

Something like this would statically compile:

CGO_ENABLED=0 GOOS=linux go build -a -ldflags '-extldflags "-static"' .

This is passing -static to the external linker and disabling cgo (i.e. disable the ability to call c code. Not sure if it's required or not but all the examples I've seen disable it).

[mislav]

Thank you @philwinder @andyneff. I've also been reading https://medium.com/@diogok/on-golang-static-binaries-cross-compiling-and-plugins-1aed33499671 which suggests:

CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -a \
  -tags netgo -ldflags '-w -extldflags "-static"' -o mybin *.go

Now the question is, which precompiled builds should I apply these flags to? Or, should I make a separate precompiled build with these settings? When to instruct people in the install docs to try downloading that build?

[stealthybox]

@mislav Can we reopen this and clarify in the title that it's a musl dynamic-linking issue, or would you like me to open a new one?

docker run -it --rm alpine
/ # apk add git curl bash
fetch http://dl-cdn.alpinelinux.org/alpine/v3.10/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.10/community/x86_64/APKINDEX.tar.gz
(1/12) Installing ncurses-terminfo-base (6.1_p20190518-r0)
(2/12) Installing ncurses-terminfo (6.1_p20190518-r0)
(3/12) Installing ncurses-libs (6.1_p20190518-r0)
(4/12) Installing readline (8.0.0-r0)
(5/12) Installing bash (5.0.0-r0)
Executing bash-5.0.0-r0.post-install
(6/12) Installing ca-certificates (20190108-r0)
(7/12) Installing nghttp2-libs (1.39.2-r0)
(8/12) Installing libcurl (7.66.0-r0)
(9/12) Installing curl (7.66.0-r0)
(10/12) Installing expat (2.2.8-r0)
(11/12) Installing pcre2 (10.33-r0)
(12/12) Installing git (2.22.0-r0)
Executing busybox-1.30.1-r2.trigger
Executing ca-certificates-20190108-r0.trigger
OK: 30 MiB in 26 packages
/ # curl -fLs https://github.com/github/hub/releases/download/v2.12.8/hub-linux-amd64-2.12.8.tgz | tar xzf -
/ # ./hub-linux-amd64-2.12.8/install 
/ # ldd $(which hub)
    /lib64/ld-linux-x86-64.so.2 (0x7f46356e9000)
    libpthread.so.0 => /lib64/ld-linux-x86-64.so.2 (0x7f46356e9000)
    libc.so.6 => /lib64/ld-linux-x86-64.so.2 (0x7f46356e9000)
/ # hub
/bin/sh: hub: not found
/ # 

---

I'd be fine personally as a user if the binary release was statically linked.

Some projects go as far as to release a dynamic-glibc, dynamic-musl, and static-go variant.
You can also statically compile using C libs instead of netgo /w -linkmode external using either musl (MIT) or glibc (GPL), subject to the respective license.

[mislav]

Some projects go as far as to release a dynamic-glibc, dynamic-musl, and static-go variant.
You can also statically compile using C libs instead of netgo /w -linkmode external using either musl (MIT) or glibc (GPL), subject to the respective license.

Thanks for the tips.

Since existing linux build seem to work (at least, there were no complains until now) for _most_ people except those who use Alpine, I don't want to risk kicking the hornet's nest by changing the build configuration for those. Instead, I agree that we could offer alternate builds. But since I'm short on time and this issue personally doesn't affect me, I would like yours or others' help on figuring out what the exact build flags should be, and how to call this build. If we're doing this primarily to benefit Alpine users, should we name the build "alpine"? Or, if this is just for alpine, maybe we can go the dynamic-musl route?

Reopening

[XVilka]

Note, there is a package for Alpine, but in 'testing' repository, which you should enable explicitly: https://pkgs.alpinelinux.org/packages?name=hub

[stainboy]

Adding environment variable CGO_ENABLED=0 before go build should solve this issue.

[infa-ddeore]

Note, there is a package for Alpine, but in 'testing' repository, which you should enable explicitly: https://pkgs.alpinelinux.org/packages?name=hub

install until then with

apk add --no-cache -X http://dl-cdn.alpinelinux.org/alpine/edge/testing hub