Docker-alpine: `RUN apk ...` in a Dockerfile can succeed while failing to install packages

[squaremo]

If I put the Dockerfile example from http://gliderlabs.viewdocs.io/docker-alpine/usage in a file and do Docker build ., it will often fail to install packages, but not fail (that step of) the build.

Here's a transcript of this happening:

$ cat Dockerfile 
FROM gliderlabs/alpine:3.1
RUN apk add --update \
    python \
    python-dev \
    py-pip \
    build-base \
  && pip install virtualenv \
  && rm -rf /var/cache/apk/*
$ docker build .
Sending build context to Docker daemon 2.048 kB
Sending build context to Docker daemon 
Step 0 : FROM gliderlabs/alpine:3.1
 ---> c9fa955c112e
Step 1 : RUN apk add --update     python     python-dev     py-pip     build-base   && pip install virtualenv   && rm -rf /var/cache/apk/*
 ---> Running in 8b06601a9ccd
fetch http://dl-4.alpinelinux.org/alpine/v3.1/main/x86_64/APKINDEX.tar.gz
(1/34) Installing libbz2 (1.0.6-r3)
(2/34) Installing expat (2.1.0-r1)
(3/34) Installing libffi (3.0.13-r0)
(4/34) Installing gdbm (1.11-r0)
(5/34) Installing ncurses-terminfo-base (5.9-r3)
(6/34) Installing ncurses-libs (5.9-r3)
(7/34) Installing readline (6.3-r3)
(8/34) Installing sqlite-libs (3.8.7.4-r0)
(9/34) Installing python (2.7.9-r0)
ERROR: python-2.7.9-r0: Resource temporarily unavailable
(10/34) Installing python-doc (2.7.9-r0)
(11/34) Installing python-tests (2.7.9-r0)
ERROR: python-tests-2.7.9-r0: Resource temporarily unavailable
(12/34) Installing py-gdbm (2.7.9-r0)
(13/34) Installing pkgconf (0.9.7-r0)
(14/34) Installing pkgconfig (0.25-r1)
(15/34) Installing python-dev (2.7.9-r0)
(16/34) Installing py-setuptools (1.1.7-r0)
(17/34) Installing py-pip (1.5.6-r2)
ERROR: py-pip-1.5.6-r2: Resource temporarily unavailable
(18/34) Installing binutils-libs (2.24-r3)
(19/34) Installing binutils (2.24-r3)
(20/34) Installing libgomp (4.8.3-r0)
(21/34) Installing gmp5 (5.1.3-r0)
ERROR: gmp5-5.1.3-r0: Resource temporarily unavailable
(22/34) Installing mpfr3 (3.1.2-r0)
(23/34) Installing mpc1 (1.0.1-r0)
ERROR: mpc1-1.0.1-r0: Resource temporarily unavailable
(24/34) Installing gcc (4.8.3-r0)
ERROR: gcc-4.8.3-r0: Resource temporarily unavailable
(25/34) Installing make (4.1-r0)
(26/34) Installing patch (2.7.1-r1)
(27/34) Installing musl-dbg (1.1.5-r2)
(28/34) Installing musl-dev (1.1.5-r2)
(29/34) Installing linux-headers (3.12.6-r1)
(30/34) Installing libc-dev (0.6-r0)
(31/34) Installing libgcc (4.8.3-r0)
ERROR: libgcc-4.8.3-r0: Resource temporarily unavailable
(32/34) Installing libstdc++ (4.8.3-r0)
ERROR: libstdc++-4.8.3-r0: Resource temporarily unavailable
(33/34) Installing g++ (4.8.3-r0)
(34/34) Installing build-base (0.3-r0)
Executing busybox-1.22.1-r14.trigger
8 errors; 87 MiB in 41 packages
/bin/sh: pip: not found

Edit: I'm running docker 1.5 in a VirtualBox VM with Ubuntu 14.04.

[kalak451]

I'm currently seeing exactly this same problem running in VirtualBox on a Ubuntu 14.04 image (using vagrants ubutntu/trusty64 box)

[kalak451]

I tracked down the cause of the "Resource temporarily unavailable" errors to intermittent DNS errors. I pointed the vagrant box straight to google's DNS servers and now everything is fine.

[andyshinn]

I think this might be an upstream issue with apk. I'm trying to find a good way to reproduce it so I can try and locate the bug.

[philm]

It looks like an issue with one of the mirrors. When using this image, the error pops up when http://dl-4.alpinelinux.org is called (set in https://github.com/gliderlabs/docker-alpine/blob/1845dc52cc5d484dcf3cac4ce6a40e45bf69e3c8/versions/gliderlabs-3.1/options#L2).

I discovered this when I switched over to https://github.com/mini-containers/base as the base image, which pulls from http://nl.alpinelinux.org.

[ifduyue]

I'm seeing this, too. It's dns errors like @kalak451 said.
But, if there are errors, apk add should fail.

I've submitted an issue to the upstream issue tracker. http://bugs.alpinelinux.org/issues/4019

[andyshinn]

It would likely be helpful to have steps for them to reproduce the issue outside of Docker. This would separate any dependency on Docker. I'm guessing that apk isn't failing correctly on bad DNS responses (which sometimes happen with the VirtualBox DNS proxy). But I haven't thought of a good way to reproduce this outside of Docker to eliminate that part of the chain for them.

[andyshinn]

It looks like @ncopa might have fixed this in http://git.alpinelinux.org/cgit/apk-tools/commit/?id=944eae4b27d039608dd72b1d7b3ed015c20bdcc0 and http://git.alpinelinux.org/cgit/apk-tools/commit/?id=7e3f4c3d7996811a37393514d6f0eaacae7178e2. Can someone who was experiencing the issue try manually installing http://dl-4.alpinelinux.org/alpine/edge/main/x86_64/apk-tools-2.6.0_rc1-r1.apk first and see if the issue persists?

[chriswessels]

I'm having the same problem. It's good that this has been fixed upstream (apk add emitting a failure code), but it's still not great to be pulling from a flaky mirror. Is there a way to switch the the pull location to http://nl.alpinelinux.org?

@andyshinn the apk link you reference is 404ing.

[chriswessels]

As an update, I've found that if you set the contents of the /etc/apk/repositories file to a list of mirrors, apk add is much more fault tolerant.

http://nl.alpinelinux.org/alpine/v3.1/main
http://nl.alpinelinux.org/alpine/v3.1/main
http://dl-2.alpinelinux.org/alpine/v3.1/main
http://dl-3.alpinelinux.org/alpine/v3.1/main
http://dl-4.alpinelinux.org/alpine/v3.1/main
http://dl-5.alpinelinux.org/alpine/v3.1/main
http://mirror.yandex.ru/mirrors/alpine/v3.1/main
http://repos.lax-noc.com/alpine/v3.1/main
http://distrib-coffee.ipsl.jussieu.fr/pub/linux/alpine/alpine/v3.1/main

Then run apk update and apk add ...

[ifduyue]

I can confirm that upstream has fixed this problem. However, it doesn't work in an exit-on-error way, you've to wait for the apk add to finish. Better than nothing.

@chriswessels +1
@andyshinn please consider adding more mirrors to /etc/apk/repositories

[andyshinn]

Can someone open a separate issue for more mirrors? I wasn't aware it would
do automatic fail over if one is down. Definitely something I'll test.
On Apr 29, 2015 8:51 PM, "Yue Du" [email protected] wrote:

I can confirm that upstream has fixed this problem. However, it doesn't
work in an exit-on-error way, you've to wait for the apk add to finish.
Better than nothing.

@chriswessels https://github.com/chriswessels +1
@andyshinn https://github.com/andyshinn please consider adding more
mirrors to /etc/apk/repositories

—
Reply to this email directly or view it on GitHub
https://github.com/gliderlabs/docker-alpine/issues/15#issuecomment-97632090
.

[andyshinn]

I just pushed new gliderlabs/alpine:edge image which has the latest apk-tools with the exit on error fixes. I`m going to close this issue as the fixes should be available upstream in 3.2. Re-open this issue if it is still not erroring properly during build.

I'll work on the separate issue of multiple repositories for fail over separately.

[dpicky]

Python needs adding to Dockerfile in order for it to build

RUN apk --update add python

[Dannyad84]

Python needs adding to Dockerfile in order for it to build

RUN apk --update add python

THANK YOU!