Consul: The `consul` user does not have write access to /consul/data in the offical docker image.

[fxdgear]

The GH repo https://github.com/hashicorp/docker-consul does not support issues, so filing here.

consul version for both Client and Server

Client: consul:latest https://hub.docker.com/r/library/consul/
Server: consul:latest https://hub.docker.com/r/library/consul/

Operating system and Environment details

Offical Docker Image. https://github.com/hashicorp/docker-consul

Description of the Issue (and unexpected/desired result)

When running docker-compose with the offical docker image I get the following error

Error starting agent: Failed to start Consul server: Failed to start Raft: mkdir /consul/data/raft: permission denied

Reproduction steps

docker-compose.yml

consul:
  image: consul:latest
  command: agent -server -config-file /consul/config/consul.json -ui-dir /opt/consul/ui
  volumes:
    - "./services/vault/consul/assets/conf.d:/consul/config:ro"
    - "./services/vault/consul/data:/consul/data:rw"

Then run docker-compose up

But if I change the entrypoint

consul:
  image: consul:latest
  entrypoint: /bin/consul   # < -- override the entry point here
  command: agent -server -config-file /consul/config/consul.json -ui-dir /opt/consul/ui
  volumes:
    - "./services/vault/consul/assets/conf.d:/consul/config:ro"
    - "./services/vault/consul/data:/consul/data:rw"

it works fine.

[slackpad]

Hi @fxdgear could this be a problem with the permissions on "/services/vault/consul/data" that's being bound to that volume? The Dockerfile creates the data dir and assigns the consul user rights to it.

[fxdgear]

@slackpad thanks for the response. I'm curious what the perms should be.

[fxdgear]

@slackpad so for a bit more info I'm using

docker-machine version 0.7.0, build a650a40

docker-compose version 1.7.0, build 0d7bf73
docker-py version: 1.8.0
CPython version: 2.7.9
OpenSSL version: OpenSSL 1.0.1j 15 Oct 2014

Client:
 Version:      1.11.1
 API version:  1.23
 Go version:   go1.5.4
 Git commit:   5604cbe
 Built:        Wed Apr 27 00:34:20 2016
 OS/Arch:      darwin/amd64

Server:
 Version:      1.11.1
 API version:  1.23
 Go version:   go1.5.4
 Git commit:   5604cbe
 Built:        Wed Apr 27 00:34:20 2016
 OS/Arch:      linux/amd64

[fxdgear]

@slackpad ahh sorry I realized this is an issue with docker-machine and not with consul. :(

[slackpad]

@fxdgear appreciate the update. Please link any context here if you can in case others run into this. Thanks!

[fxdgear]

workaround would be (for the time being)

chmod -R 777 <volume on host>

There's an issue with the way docker-machine translates permissions from the host to the vm to the docker engine.

[deviantony]

Actually, how would you persist consul data on the docker host?

I'm having the same issue using docker engine 1.11.1, I try to start consul this way:

$ docker run -d --restart always --net host -v "/opt/consul:/consul/data" -p 8500:8500 --name consul_node1 consul agent -client <NODEIP> -bind <NODEIP> -server -bootstrap-expect 3

==> Error starting agent: Failed to start Consul server: Failed to start Raft: mkdir /consul/data/raft: permission denied

Obviously the chmod 777 workaround should work, but I do not like it so much.

[sheerun]

chmod 777 is not a solution.. please reopen?

[jhmartin]

@deviantony Could http://www.projectatomic.io/blog/2015/06/using-volumes-with-docker-can-cause-problems-with-selinux/ be related to your issue?

[deviantony]

@jhmartin I'm having this issue on a fresh Ubuntu 14.04 server box (no SELinux).

[slackpad]

I enabled issues on https://github.com/hashicorp/docker-consul so it's probably best to track this there for visibility and link to this issue. I'm not sure if the consul image itself can do much to fix this, but I'm open to any ideas folks have.

[barbarello]

@slackpad
do you mind letting us know which issue in https://github.com/hashicorp/docker-consul tracks this one here ?

@jhmartin
the z or Z options do not work for me

Ubuntu 16.04 , no SELinux,
$ docker version
Client:
Version: 1.11.2
API version: 1.23
Go version: go1.5.4
Git commit: b9f10c9
Built: Wed Jun 1 22:00:43 2016
OS/Arch: linux/amd64

Server:
Version: 1.11.2
API version: 1.23
Go version: go1.5.4
Git commit: b9f10c9
Built: Wed Jun 1 22:00:43 2016
OS/Arch: linux/amd64

docker logs:

$ docker logs consul
WARNING: ca_cert.pem does not contain exactly one certificate or CRL: skipping
WARNING: ca-certificates.crt does not contain exactly one certificate or CRL: skipping
WARNING: ca-cert-consulca.pem does not contain exactly one certificate or CRL: skipping
==> WARNING: Expect Mode enabled, expecting 3 servers
==> Starting Consul agent...
==> Error starting agent: Failed to configure keyring: mkdir /data/serf: permission denied

[slackpad]

Hi @barbarello there's currently no issue tracking this - rather than re-open here it probably makes sense to make one over there and link it so it'll get better visibility.