Docker: Using docker build plugin and getting "Cannot connect to the Docker daemon."

[CameronGo]

I'm not sure the best strategy for this, but the path I took was to create a custom image from the public jenkins image, and I have installed docker into my custom image, adding the jenkins user to the docker group. This is what my Dockerfile looks like:
FROM jenkins
USER root
RUN curl -sSL https://get.docker.com/ | sh && \
usermod -aG docker jenkins
USER jenkins

Then I launch the container using this command:
docker run -d --name=jenkins -p 8080:8080 -p 50000:50000 -v /data/jenkins:/var/jenkins_home jenkinscustom

As soon as I attempt a build using the plugin; however, I get the following error immediately after the docker build command is issued:
Cannot connect to the Docker daemon. Is the docker daemon running on this host?

Obviously the docker daemon isn't running since it is installed as a service. I'm not sure this is even the right approach to take. Is there any guidance for the best way to use the Docker Build plugins when running jenkins inside a Docker container?

[dweomer]

You need to have a docker daemon to connect to. As per https://jpetazzo.github.io/2015/09/03/do-not-use-docker-in-docker-for-ci/, a convenient way of doing such is bind-mounting /var/run/docker.dock into your container. The next problem that you will run into, however, is docker group permission mismatch. The docker group is commonly created with gid=999 but that cannot be relied upon when you have imported in /var/run/docker.sock from the host system (the docker socket has gid=100 on my boot2docker instance on my mac). What I have done is

• extend the official image to run as root
• installed docker static executable
• add a boot time script that executes something like this:

#!/bin/bash -x

# this only works if the docker group does not already exist

DOCKER_SOCKET=/var/run/docker.sock
DOCKER_GROUP=docker

if [ -S ${DOCKER_SOCKET} ]; then
    DOCKER_GID=$(stat -c '%g' ${DOCKER_SOCKET})
    groupadd -for -g ${DOCKER_GID} ${DOCKER_GROUP}
    usermod -aG ${DOCKER_GROUP} ${JENKINS_USER}
fi

• installed gosu static executable
• use gosu to drop to the jenkins user and call /usr/local/bin/jenkins.sh

[carlossg]

right, you can't just run a docker daemon in a docker container

[CameronGo]

Thanks for the link and the additional info @dweomer. That came in handy. I managed to get something working that is a variation on what you suggested and I'd be curious to get your, or anyone else's opinions on best practices.

1. I tried the config as outlined in the ~jpetazzo article. The issue I encountered due to bind-mounting both the docker.sock and the /usr/bin/docker binary is the other dependencies to run "docker build" were not present, so I encountered an error:
   docker: error while loading shared libraries: libapparmor.so.1: cannot open shared object file: No such file or directory

So, from this I concluded I have to at least install docker into my image. Note: Since I'm now installing docker into the image I had to map the group ID to a different name in the run.sh script below.

1. I haven't used gosu before, so I stuck to sudo, which I know and either had to be installed in the image regardless. Using sudo I ran into a few issues with ENV variables between the users and HOME dir, which were resolved with parameters passed to the sudo command.

So now I have a Dockerfile that looks like this:
FROM jenkins USER root RUN curl -sSL https://get.docker.com/ | sh && \ usermod -aG docker jenkins && \ apt-get install sudo COPY run.sh ./run.sh ENTRYPOINT ["/bin/bash","-c","./run.sh"]

And I have a run.sh script that looks like this:
`#!/bin/bash -x

DOCKER_SOCKET=/var/run/docker.sock
DOCKER_GROUP=dockerhost
JENKINS_USER=jenkins

if [ -S ${DOCKER_SOCKET} ]; then
DOCKER_GID=$(stat -c '%g' ${DOCKER_SOCKET})
groupadd -for -g ${DOCKER_GID} ${DOCKER_GROUP}
usermod -aG ${DOCKER_GROUP} ${JENKINS_USER}
fi

exec sudo -E -H -u jenkins bash -c /usr/local/bin/jenkins.sh`

[dweomer]

Bind-mounting the Docker executable is only feasible when it is a static (aka standalone) executable. Older articles mention doing this because it was common practice that relied on the fact that the docker executable for the longest time _was_ statically compiled.

Installing docker via the get.docker.com URL is overkill for inside of container where you only really need a client setup (the script from that URL is a full daemon + client setup with a few extra packages that you simply do not need in a client container). Instead do something like this in your Dockerfile:

### Install Docker
RUN curl -fsSL --create-dirs --output /usr/local/bin/docker \
        "https://get.docker.com/builds/$(uname -s)/$(uname -m)/docker-${DOCKER_VERSION}" \
 && chmod +x /usr/local/bin/docker

This installs the statically linked Docker executable and does not attempt to install support packages like apparmor or aufs which will happen, depending on your base container, when using the get.docker.com script.

[CameronGo]

@dweomer: Good tip, thanks! I updated my image accordingly and its still all working, so I think this is the way for me to go.

[bkcummins]

Thx, super helpful!

Here's how I got this to work w/o the curl to get.docker.com.

_./docker-compose.yml_

jenkins:
    build: ./jenkins
    volumes:
      - /var/jenkins_home
      - /var/run/docker.sock:/var/run/docker.sock
      - /usr/bin/docker:/usr/bin/docker
    environment:
      DOCKER_SOCKET: /var/run/docker.sock
      DOCKER_GROUP: dockerhost
      JENKINS_USER: jenkins
      JENKINS_URL: "https://jenkins.example.com/"
      DOCKER_HOST: unix:///var/run/docker.sock

_./jenkins/Dockerfile_

FROM jenkins

USER root 

COPY ["entrypoint.sh", "/"]

RUN apt-get update && \
    apt-get install sudo && \
    chmod 755 /entrypoint.sh

ENTRYPOINT ["/bin/bash","-c","./entrypoint.sh"]

_./jenkins/entrypoint.sh_

#!/bin/bash

if [ -S ${DOCKER_SOCKET} ]; then
    DOCKER_GID=$(stat -c '%g' ${DOCKER_SOCKET})
    groupadd -for -g ${DOCKER_GID} ${DOCKER_GROUP}
    usermod -aG ${DOCKER_GROUP} ${JENKINS_USER}
fi

exec sudo -E -H -u jenkins bash -c /usr/local/bin/jenkins.sh

[brthor]

thanks @bkcummins and @CameronGo

I managed to stick the contents of entrypoint in cmd rather than the file. In my deployment environment this is easier to manage than having another external file.

FROM jenkins:2.7.1
USER 0
CMD DOCKER_GID=$(stat -c '%g' /var/run/docker.sock) && \
    groupadd -for -g ${DOCKER_GID} docker && \
    usermod -aG docker jenkins && \
    sudo -E -H -u jenkins bash -c /usr/local/bin/jenkins.sh

[isuftin]

Unfortunately this does not work in docker-machine since you can't (afaik) mount /var/run/docker.sock from your Docker Machine VM to the container running in the VM.

[asos-VasilisKortsimelidis]

Thanks @CameronGo and @bkcummins. I tried what you suggested but the if statement fails because the /var/run/docker.sock is missing.

if [ -S ${DOCKER_SOCKET} ]

The volumes property in my docker-compose.yml is

volumes:
      - /var/jenkins_home
      - /var/run/docker.sock:/var/run/docker.sock
      - /usr/local/bin/docker:/usr/bin/docker

any ideas what I might be doing wrong please?:)

[CameronGo]

Here's my docker-compose.yml:

version: '3'

services:
  jenkins-server:
    build: .
    environment: 
      - DOCKER_SOCKET=/var/run/docker.sock
      - DOCKER_GROUP=dockerhost
      - JENKINS_USER=jenkins
    ports:
      - 8080:8080
      - 50000:50000
    volumes:
      - /data/jenkins:/var/jenkins_home
      - /var/run/docker.sock:/var/run/docker.sock

and here is my Dockerfile:

FROM jenkins/jenkins:2.118
USER root
COPY run.sh ./run.sh
RUN     curl -fsSLO https://download.docker.com/linux/static/stable/x86_64/docker-17.03.2-ce.tgz && tar --strip-components=1 -xvzf docker-17.03.2-ce.tgz -C /usr/local/bin && \
        chmod +x ./run.sh && \
        apt-get update -y && apt-get install -y sudo python-pip && \
        pip install awscli && \
        apt-get clean -y

ENTRYPOINT ["/bin/bash","-c","./run.sh"]

and here is the run.sh script that is running at launch:

#!/bin/bash -x

if [ -S ${DOCKER_SOCKET} ]; then
    DOCKER_GID=$(stat -c '%g' ${DOCKER_SOCKET})
    groupadd -for -g ${DOCKER_GID} ${DOCKER_GROUP}
    usermod -aG ${DOCKER_GROUP} ${JENKINS_USER}
fi

exec sudo -E -H -u jenkins bash -c /usr/local/bin/jenkins.sh

If you put all 3 of those files into a folder and run docker-compose up then you should get a jenkins instance up and running that let's you run docker build inside of it provided the docker host also has docker installed and running.