Compose: Not mounting host volumes on `docker-compose up`
We have an machine that's having an issue where host volumes aren't being mounted inside of the container on docker-compose up. We're running Arch x86_64 with Linux 4.2.3, Docker version 1.8.3, build f4bf5c7-dirty, and docker-compose version: 1.4.2. Here's our config:
docker-compose.yml
db:
image: postgres:9.4
wsgi:
build: app
links:
- db
environment:
LM_PROD:
restart: always
web:
build: web
links:
- wsgi
ports:
- "80:80"
- "443:443"
volumes_from:
- wsgi
restart: always
dev:
build: app
links:
- db
ports:
- "8000:8000"
volumes:
- ./app/:/mnt/app/
environment:
LM_DEBUG: "true"
LM_PROD:
restart: always
While wsgi service works just fine (which uses the same ./app directory), the dev service doesn't. Even stranger, it seems that we're able to get host volumes mounted inside the container just fine with a shell command that does the same thing.
docker build -t dev app
db_link="$(docker run -d postgres:9.4)"
docker run \
--restart=always \
-p 8000:8000 \
--link "$db_link:db" \
-e "LM_DEBUG=true" \
-e "LM_PROD=$LM_PROD" \
-v "$(pwd)/app/:/mnt/app/" \
dev
Weird, right? Here are our logs, showing that the file ./app/util/start isn't being mounted to /mnt/app/util/start, whereas it's definitely there when we build the directory (as opposed to mounting a host volume).
$ docker-compose up
Creating locationmanagement_db_1...
Creating locationmanagement_wsgi_1...
Creating locationmanagement_web_1...
Creating locationmanagement_dev_1...
Attaching to locationmanagement_db_1, locationmanagement_wsgi_1, locationmanagement_web_1, locationmanagement_dev_1
dev_1 | /bin/sh: 1: /mnt/app/util/start: not found
locationmanagement_dev_1 exited with code 0
Gracefully stopping... (press Ctrl+C again to force)
Stopping locationmanagement_dev_1... error
Stopping locationmanagement_web_1... done
Stopping locationmanagement_wsgi_1... done
Stopping locationmanagement_db_1... done
And the results of lxc-checkconfig:
christianbundy
All 68 comments
There is probably a container volume that is masking the host volume. what do you see for mounts/binds in docker inspect locationmanagement_dev_1 ?
dnephin
on 23 Oct 2015
I just pinged the developer working on this machine to send me the output of the docker inspect command, I'll add that as soon as I get it.
For your reference, here's the relevant Dockerfile:
FROM debian:jessie
RUN apt-get update && apt-get install -y \
libpcre3 \
libpcre3-dev \
libpq-dev \
postgresql-client-9.4 \
python3-dev \
python3-pip
RUN mkdir -p /mnt/media/static/prebooking/temp_uploads/
RUN mkdir -p /mnt/static
ADD ./requirements.txt /mnt/app/requirements.txt
RUN pip3 install -r /mnt/app/requirements.txt
VOLUME ["/mnt/media", "/mnt/static"]
EXPOSE 8000
EXPOSE 8001
ADD . /mnt/app
RUN mkdir /mnt/gpg
RUN gpg --homedir /mnt/gpg --import /mnt/app/prebooking/keys/public.asc
RUN gpg --homedir /mnt/gpg --allow-secret-key-import --import /mnt/app/prebooking/keys/private.asc
RUN chown -R www-data /mnt/gpg
RUN chmod 700 /mnt/gpg
RUN chmod 600 /mnt/gpg/*
CMD /mnt/app/util/start
Just got the inspection data back - it looks like /mnt/app is missing.
[
{
"Id":"b5ff2411d361dd0b0e3c16703cf4dcd6625f822b3dc69827f89c6414b7145957",
"Parent":"c1987193c2197454163b04b79b5e87a9dfb5260304d37db4251409e89b2a0a4c",
"Comment":"",
"Created":"2015-10-23T21:14:47.532402742Z",
"Container":"cfe49e79daa35b3f6c73db58ac528cd06c5d4c75ec02f8cc03fd03842e8e25ae",
"ContainerConfig":{
"Hostname":"e615e9fecf7f",
"Domainname":"",
"User":"",
"AttachStdin":false,
"AttachStdout":false,
"AttachStderr":false,
"ExposedPorts":{
"8000/tcp":{
},
"8001/tcp":{
}
},
"Tty":false,
"OpenStdin":false,
"StdinOnce":false,
"Env":[
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],
"Cmd":[
"/bin/sh",
"-c",
"#(nop) CMD [\"/bin/sh\" \"-c\" \"/mnt/app/util/start\"]"
],
"Image":"c1987193c2197454163b04b79b5e87a9dfb5260304d37db4251409e89b2a0a4c",
"Volumes":{
"/mnt/media":{
},
"/mnt/static":{
}
},
"WorkingDir":"",
"Entrypoint":null,
"OnBuild":[
],
"Labels":null
},
"DockerVersion":"1.8.3",
"Author":"",
"Config":{
"Hostname":"e615e9fecf7f",
"Domainname":"",
"User":"",
"AttachStdin":false,
"AttachStdout":false,
"AttachStderr":false,
"ExposedPorts":{
"8000/tcp":{
},
"8001/tcp":{
}
},
"Tty":false,
"OpenStdin":false,
"StdinOnce":false,
"Env":[
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],
"Cmd":[
"/bin/sh",
"-c",
"/mnt/app/util/start"
],
"Image":"c1987193c2197454163b04b79b5e87a9dfb5260304d37db4251409e89b2a0a4c",
"Volumes":{
"/mnt/media":{
},
"/mnt/static":{
}
},
"WorkingDir":"",
"Entrypoint":null,
"OnBuild":[
],
"Labels":null
},
"Architecture":"amd64",
"Os":"linux",
"Size":0,
"VirtualSize":530323586,
"GraphDriver":{
"Name":"aufs",
"Data":null
}
}
]
cc: @dnephin
christianbundy
on 23 Oct 2015
I think this is an inspect of the image (locationmanagement_dev) not the container (locationmanagement_dev_1). The image wouldn't have them because they're only added to the container.
dnephin
on 23 Oct 2015
Nice catch, I just got the output back from the machine.
[
{
"Id":"fc65e50ea81f44354e94d0b5bebd2da1a6b26071ba9d9e289d0d0ddf3c78d561",
"Created":"2015-10-27T22:40:36.361795003Z",
"Path":"/bin/sh",
"Args":[
"-c",
"/mnt/app/util/start"
],
"State":{
"Running":false,
"Paused":false,
"Restarting":false,
"OOMKilled":false,
"Dead":false,
"Pid":0,
"ExitCode":127,
"Error":"",
"StartedAt":"2015-10-27T22:40:37.400811807Z",
"FinishedAt":"2015-10-27T22:40:37.433149865Z"
},
"Image":"a3feea22da553d7f39e9dde6823fcdba01cfdf584f3b0a48d3b8af472ffa310d",
"NetworkSettings":{
"Bridge":"",
"EndpointID":"",
"Gateway":"",
"GlobalIPv6Address":"",
"GlobalIPv6PrefixLen":0,
"HairpinMode":false,
"IPAddress":"",
"IPPrefixLen":0,
"IPv6Gateway":"",
"LinkLocalIPv6Address":"",
"LinkLocalIPv6PrefixLen":0,
"MacAddress":"",
"NetworkID":"",
"PortMapping":null,
"Ports":null,
"SandboxKey":"",
"SecondaryIPAddresses":null,
"SecondaryIPv6Addresses":null
},
"ResolvConfPath":"/mnt/sda1/var/lib/docker/containers/fc65e50ea81f44354e94d0b5bebd2da1a6b26071ba9d9e289d0d0ddf3c78d561/resolv.conf",
"HostnamePath":"/mnt/sda1/var/lib/docker/containers/fc65e50ea81f44354e94d0b5bebd2da1a6b26071ba9d9e289d0d0ddf3c78d561/hostname",
"HostsPath":"/mnt/sda1/var/lib/docker/containers/fc65e50ea81f44354e94d0b5bebd2da1a6b26071ba9d9e289d0d0ddf3c78d561/hosts",
"LogPath":"/mnt/sda1/var/lib/docker/containers/fc65e50ea81f44354e94d0b5bebd2da1a6b26071ba9d9e289d0d0ddf3c78d561/fc65e50ea81f44354e94d0b5bebd2da1a6b26071ba9d9e289d0d0ddf3c78d561-json.log",
"Name":"/locationmanagement_dev_1",
"RestartCount":2,
"Driver":"aufs",
"ExecDriver":"native-0.2",
"MountLabel":"",
"ProcessLabel":"",
"AppArmorProfile":"",
"ExecIDs":null,
"HostConfig":{
"Binds":[
"/home/zissou/Documents/Devel/TerraSeer/LocationManagement/app:/mnt/app:rw"
],
"ContainerIDFile":"",
"LxcConf":null,
"Memory":0,
"MemorySwap":0,
"CpuShares":0,
"CpuPeriod":0,
"CpusetCpus":"",
"CpusetMems":"",
"CpuQuota":0,
"BlkioWeight":0,
"OomKillDisable":false,
"MemorySwappiness":null,
"Privileged":false,
"PortBindings":{
"8000/tcp":[
{
"HostIp":"",
"HostPort":"8000"
}
]
},
"Links":[
"/locationmanagement_db_1:/locationmanagement_dev_1/db",
"/locationmanagement_db_1:/locationmanagement_dev_1/db_1",
"/locationmanagement_db_1:/locationmanagement_dev_1/locationmanagement_db_1"
],
"PublishAllPorts":false,
"Dns":null,
"DnsSearch":null,
"ExtraHosts":[
],
"VolumesFrom":[
],
"Devices":null,
"NetworkMode":"default",
"IpcMode":"",
"PidMode":"",
"UTSMode":"",
"CapAdd":null,
"CapDrop":null,
"GroupAdd":null,
"RestartPolicy":{
"Name":"always",
"MaximumRetryCount":0
},
"SecurityOpt":null,
"ReadonlyRootfs":false,
"Ulimits":null,
"LogConfig":{
"Type":"json-file",
"Config":{
}
},
"CgroupParent":"",
"ConsoleSize":[
0,
0
]
},
"GraphDriver":{
"Name":"aufs",
"Data":null
},
"Mounts":[
{
"Source":"/home/zissou/Documents/Devel/TerraSeer/LocationManagement/app",
"Destination":"/mnt/app",
"Mode":"rw",
"RW":true
},
{
"Name":"ca519e865d7a9e08b39b0667d74cbcd9bc02218e6d04acebfabf24db8cde5e97",
"Source":"/mnt/sda1/var/lib/docker/volumes/ca519e865d7a9e08b39b0667d74cbcd9bc02218e6d04acebfabf24db8cde5e97/_data",
"Destination":"/mnt/media",
"Driver":"local",
"Mode":"",
"RW":true
},
{
"Name":"81fa34435f0f97e85588d7d44d11708d2985659792d588166ce67e7a0be49427",
"Source":"/mnt/sda1/var/lib/docker/volumes/81fa34435f0f97e85588d7d44d11708d2985659792d588166ce67e7a0be49427/_data",
"Destination":"/mnt/static",
"Driver":"local",
"Mode":"",
"RW":true
}
],
"Config":{
"Hostname":"fc65e50ea81f",
"Domainname":"",
"User":"",
"AttachStdin":false,
"AttachStdout":false,
"AttachStderr":false,
"ExposedPorts":{
"8000/tcp":{
},
"8001/tcp":{
}
},
"PublishService":"",
"Tty":false,
"OpenStdin":false,
"StdinOnce":false,
"Env":[
"LM_DEBUG=true",
"LM_PROD=",
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],
"Cmd":[
"/bin/sh",
"-c",
"/mnt/app/util/start"
],
"Image":"locationmanagement_dev",
"Volumes":{
"/mnt/app":{
},
"/mnt/media":{
},
"/mnt/static":{
}
},
"VolumeDriver":"",
"WorkingDir":"",
"Entrypoint":null,
"NetworkDisabled":false,
"MacAddress":"",
"OnBuild":null,
"Labels":{
"com.docker.compose.config-hash":"b684aea725820e30820410a784f3ba25fdbeaafc8795e6a84e4e6fb8df656360",
"com.docker.compose.container-number":"1",
"com.docker.compose.oneoff":"False",
"com.docker.compose.project":"locationmanagement",
"com.docker.compose.service":"dev",
"com.docker.compose.version":"1.4.2"
}
}
}
]
The mount looks ok to me:
{
"Source":"/home/zissou/Documents/Devel/TerraSeer/LocationManagement/app",
"Destination":"/mnt/app",
"Mode":"rw",
"RW":true
},
I don't see any container volumes that would be masking that path.Does that host path look right?
dnephin
on 28 Oct 2015
Yep, that's all correct. Since everything works with the normal docker inspect command, I ran this on the machine:
docker build -t dev app
db_link="$(docker run -d postgres:9.4)"
dev_link="$(docker run \
-d \
--restart=always \
-p 8000:8000 \
--link "$db_link:db" \
-e "LM_DEBUG=true" \
-e "LM_PROD=$LM_PROD" \
-v "$(pwd)/app/:/mnt/app/" \
dev)"
docker inspect "$dev_link"
and then ran a diff between that and the docker inspect location_management_dev_1. Here are the results:
diff --git a/docker-compose b/docker
index 5af9981..97b2691 100644
--- a/docker-compose
+++ b/docker
@@ -1,51 +1,59 @@
[
{
- "Id":"fc65e50ea81f44354e94d0b5bebd2da1a6b26071ba9d9e289d0d0ddf3c78d561",
- "Created":"2015-10-27T22:40:36.361795003Z",
+ "Id":"97e978fa53f16713f7665a581ee4b5335444e06dcada97dd23785e44ade0d68c",
+ "Created":"2015-10-28T01:47:25.825321376Z",
"Path":"/bin/sh",
"Args":[
"-c",
"/mnt/app/util/start"
],
"State":{
- "Running":false,
+ "Running":true,
"Paused":false,
- "Restarting":false,
+ "Restarting":true,
"OOMKilled":false,
"Dead":false,
"Pid":0,
"ExitCode":127,
"Error":"",
- "StartedAt":"2015-10-27T22:40:37.400811807Z",
- "FinishedAt":"2015-10-27T22:40:37.433149865Z"
+ "StartedAt":"2015-10-28T01:50:50.968126914Z",
+ "FinishedAt":"2015-10-28T01:50:50.990319687Z"
},
- "Image":"a3feea22da553d7f39e9dde6823fcdba01cfdf584f3b0a48d3b8af472ffa310d",
+ "Image":"6cfaebb77316901b5cdddb9d0ae69eb8242986abe8ae700e5487c3c5b6fb9561",
"NetworkSettings":{
"Bridge":"",
- "EndpointID":"",
- "Gateway":"",
+ "EndpointID":"b7a030af5605f3e257208061ad1daf6431fd8f2a73e1a0d1d8b1b4f906c22d0b",
+ "Gateway":"172.17.42.1",
"GlobalIPv6Address":"",
"GlobalIPv6PrefixLen":0,
"HairpinMode":false,
- "IPAddress":"",
- "IPPrefixLen":0,
+ "IPAddress":"172.17.0.35",
+ "IPPrefixLen":16,
"IPv6Gateway":"",
"LinkLocalIPv6Address":"",
"LinkLocalIPv6PrefixLen":0,
- "MacAddress":"",
- "NetworkID":"",
+ "MacAddress":"02:42:ac:11:00:23",
+ "NetworkID":"fbf7849c846c8c075a2f7905ddb237fdaffe472b5b81904ed15c073aa98d9f22",
"PortMapping":null,
- "Ports":null,
- "SandboxKey":"",
+ "Ports":{
+ "8000/tcp":[
+ {
+ "HostIp":"0.0.0.0",
+ "HostPort":"8000"
+ }
+ ],
+ "8001/tcp":null
+ },
+ "SandboxKey":"/var/run/docker/netns/97e978fa53f1",
"SecondaryIPAddresses":null,
"SecondaryIPv6Addresses":null
},
- "ResolvConfPath":"/mnt/sda1/var/lib/docker/containers/fc65e50ea81f44354e94d0b5bebd2da1a6b26071ba9d9e289d0d0ddf3c78d561/resolv.conf",
- "HostnamePath":"/mnt/sda1/var/lib/docker/containers/fc65e50ea81f44354e94d0b5bebd2da1a6b26071ba9d9e289d0d0ddf3c78d561/hostname",
- "HostsPath":"/mnt/sda1/var/lib/docker/containers/fc65e50ea81f44354e94d0b5bebd2da1a6b26071ba9d9e289d0d0ddf3c78d561/hosts",
- "LogPath":"/mnt/sda1/var/lib/docker/containers/fc65e50ea81f44354e94d0b5bebd2da1a6b26071ba9d9e289d0d0ddf3c78d561/fc65e50ea81f44354e94d0b5bebd2da1a6b26071ba9d9e289d0d0ddf3c78d561-json.log",
- "Name":"/locationmanagement_dev_1",
- "RestartCount":2,
+ "ResolvConfPath":"/mnt/sda1/var/lib/docker/containers/97e978fa53f16713f7665a581ee4b5335444e06dcada97dd23785e44ade0d68c/resolv.conf",
+ "HostnamePath":"/mnt/sda1/var/lib/docker/containers/97e978fa53f16713f7665a581ee4b5335444e06dcada97dd23785e44ade0d68c/hostname",
+ "HostsPath":"/mnt/sda1/var/lib/docker/containers/97e978fa53f16713f7665a581ee4b5335444e06dcada97dd23785e44ade0d68c/hosts",
+ "LogPath":"/mnt/sda1/var/lib/docker/containers/97e978fa53f16713f7665a581ee4b5335444e06dcada97dd23785e44ade0d68c/97e978fa53f16713f7665a581ee4b5335444e06dcada97dd23785e44ade0d68c-json.log",
+ "Name":"/angry_euclid",
+ "RestartCount":10,
"Driver":"aufs",
"ExecDriver":"native-0.2",
"MountLabel":"",
@@ -54,10 +62,12 @@
"ExecIDs":null,
"HostConfig":{
"Binds":[
- "/home/zissou/Documents/Devel/TerraSeer/LocationManagement/app:/mnt/app:rw"
+ "/home/zissou/Documents/Devel/TerraSeer/LocationManagement/app/:/mnt/app"
],
"ContainerIDFile":"",
- "LxcConf":null,
+ "LxcConf":[
+
+ ],
"Memory":0,
"MemorySwap":0,
"CpuShares":0,
@@ -67,7 +77,7 @@
"CpuQuota":0,
"BlkioWeight":0,
"OomKillDisable":false,
- "MemorySwappiness":null,
+ "MemorySwappiness":-1,
"Privileged":false,
"PortBindings":{
"8000/tcp":[
@@ -78,20 +88,16 @@
]
},
"Links":[
- "/locationmanagement_db_1:/locationmanagement_dev_1/db",
- "/locationmanagement_db_1:/locationmanagement_dev_1/db_1",
- "/locationmanagement_db_1:/locationmanagement_dev_1/locationmanagement_db_1"
+ "/jolly_ptolemy:/angry_euclid/db"
],
"PublishAllPorts":false,
"Dns":null,
"DnsSearch":null,
- "ExtraHosts":[
-
- ],
- "VolumesFrom":[
+ "ExtraHosts":null,
+ "VolumesFrom":null,
+ "Devices":[
],
- "Devices":null,
"NetworkMode":"default",
"IpcMode":"",
"PidMode":"",
@@ -124,30 +130,30 @@
},
"Mounts":[
{
- "Source":"/home/zissou/Documents/Devel/TerraSeer/LocationManagement/app",
- "Destination":"/mnt/app",
- "Mode":"rw",
+ "Name":"d9f69cac02b603bd26c1a7ea74390d4f31e70ce152b0315efd016129de334417",
+ "Source":"/mnt/sda1/var/lib/docker/volumes/d9f69cac02b603bd26c1a7ea74390d4f31e70ce152b0315efd016129de334417/_data",
+ "Destination":"/mnt/static",
+ "Driver":"local",
+ "Mode":"",
"RW":true
},
{
- "Name":"ca519e865d7a9e08b39b0667d74cbcd9bc02218e6d04acebfabf24db8cde5e97",
- "Source":"/mnt/sda1/var/lib/docker/volumes/ca519e865d7a9e08b39b0667d74cbcd9bc02218e6d04acebfabf24db8cde5e97/_data",
- "Destination":"/mnt/media",
- "Driver":"local",
+ "Source":"/home/zissou/Documents/Devel/TerraSeer/LocationManagement/app",
+ "Destination":"/mnt/app",
"Mode":"",
"RW":true
},
{
- "Name":"81fa34435f0f97e85588d7d44d11708d2985659792d588166ce67e7a0be49427",
- "Source":"/mnt/sda1/var/lib/docker/volumes/81fa34435f0f97e85588d7d44d11708d2985659792d588166ce67e7a0be49427/_data",
- "Destination":"/mnt/static",
+ "Name":"08a105e3cdd3bfa7ef0f99000cf0f8840a18ffdb015fe2045aa7dfce532c0dd9",
+ "Source":"/mnt/sda1/var/lib/docker/volumes/08a105e3cdd3bfa7ef0f99000cf0f8840a18ffdb015fe2045aa7dfce532c0dd9/_data",
+ "Destination":"/mnt/media",
"Driver":"local",
"Mode":"",
"RW":true
}
],
"Config":{
- "Hostname":"fc65e50ea81f",
+ "Hostname":"97e978fa53f1",
"Domainname":"",
"User":"",
"AttachStdin":false,
@@ -166,7 +172,7 @@
"OpenStdin":false,
"StdinOnce":false,
"Env":[
- "LM_DEBUG=true",
+ "LM_DEBUG=",
"LM_PROD=",
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],
@@ -175,11 +181,8 @@
"-c",
"/mnt/app/util/start"
],
- "Image":"locationmanagement_dev",
+ "Image":"dev",
"Volumes":{
- "/mnt/app":{
-
- },
"/mnt/media":{
},
@@ -194,12 +197,7 @@
"MacAddress":"",
"OnBuild":null,
"Labels":{
- "com.docker.compose.config-hash":"b684aea725820e30820410a784f3ba25fdbeaafc8795e6a84e4e6fb8df656360",
- "com.docker.compose.container-number":"1",
- "com.docker.compose.oneoff":"False",
- "com.docker.compose.project":"locationmanagement",
- "com.docker.compose.service":"dev",
- "com.docker.compose.version":"1.4.2"
+
}
}
}
Does any of that help at all?
christianbundy
on 28 Oct 2015
That all looks like an expected diff to me.
The LM_DEBUG looks different, any way that could be changing the code path to cause this?
dnephin
on 28 Oct 2015
Unfortunately the LM_DEBUG is something that /mnt/app/util/start is supposed to look for, so it shouldn't have any effect at all on whether the host volume is mounted. Are there any other ways that we might be able to troubleshoot this?
I'll admit I'm pretty stumped at this point.
christianbundy
on 28 Oct 2015
I'd try building the image with compose (docker-compose build), then running it with docker-compose run dev bash and taking a look at the files from there.
dnephin
on 28 Oct 2015
Yeah, they've got nothing. On my machine, I can run docker-compose run dev find /mnt/app -maxdepth 2 -type f -ls it returns a bunch of files (/mnt/app/util/start being one of them), but these machines are returning /mnt/app/ as an empty directory.
christianbundy
on 28 Oct 2015
I'm not sure how that is , considering the mounts look correct. The host has the expected files at /home/zissou/Documents/Devel/TerraSeer/LocationManagement/app ?
This isn't running in a VM or anything? Is $DOCKER_HOST set and it's actually hitting a different host that doesn't have any files in that path?
dnephin
on 28 Oct 2015
ping @christianbundy did you have a look at the questions @dnephin asked above?
thaJeztah
on 25 Dec 2015
I can't reproduce since everything works on my machine, but I'll ping the two who were running into this issue.
cc: @32footsteps @mdegraw
christianbundy
on 26 Dec 2015
I'm going to close this now, thanks for your time guys. Hopefully if someone bumps into the same problem down the road they'll be able to re-open the issue. Cheers!
christianbundy
on 25 Jan 2016
Unfortunately I'm stumbling upon the same issue. Running on Mac OS X 10.11.2 and Docker:
bash-3.2$ docker version
Client:
Version: 1.9.1
API version: 1.21
Go version: go1.4.3
Git commit: a34a1d5
Built: Fri Nov 20 17:56:04 UTC 2015
OS/Arch: darwin/amd64
Server:
Version: 1.9.1
API version: 1.21
Go version: go1.4.3
Git commit: a34a1d5
Built: Fri Nov 20 17:56:04 UTC 2015
OS/Arch: linux/amd64
Using Docker Toolbox and Kitematic GUI for the VM.
Here's a gist with all relevant info - https://gist.github.com/michaldudek/ae84153089bfc32b528f
Interestingly, when I run the same config through Vagrant VM (Ubuntu 14.04) (provisioned through docker / docker-compose plugin) it works.
michaldudek
on 26 Jan 2016
@michaldudek This is really odd.
I've been using the same compose file pointed to a virturalbox host for weeks. About 20 minutes ago I fired up a docker host on AWS and started seeing this issue.
$ docker version
Client:
Version: 1.9.1
API version: 1.21
Go version: go1.4.3
Git commit: a34a1d5
Built: Fri Nov 20 17:56:04 UTC 2015
OS/Arch: darwin/amd64
Server:
Version: 1.9.1
API version: 1.21
Go version: go1.4.2
Git commit: a34a1d5
Built: Fri Nov 20 13:20:08 UTC 2015
OS/Arch: linux/amd64
Gist with relevant information
nastycoder
on 26 Jan 2016
That's hilarious, in the three months the issue was open we were unable to reproduce the issue, but as soon as I closed it the problem happened to two people. Sorry for the bad timing you two, I'll re-open it now. Good luck!
christianbundy
on 26 Jan 2016
Yeah very peculiar timing.
I ended up rebuilding the AWS instance using docker-machine thinking something may have gotten foobar-ed during that process. Ended up experiencing the same results.
But fortunately the AWS environment is for QA purposes so I don't actually need the mounted volume.
I'll rebuild the virtualbox host this afternoon to see if I can reproduce the issue locally and hopefully be able to provided more information.
nastycoder
on 26 Jan 2016
I'm starting to suspect that this is an issue with Docker Machine, not Compose. Has anyone reproduced this issue _without_ Machine?
christianbundy
on 3 Feb 2016
A friend suggested to check /etc/exports and, alas, I only have Vagrant entries there, nothing related to Docker Machine. I don't know much about how NFS works (I assume Docker Machine is using NFS on Mac?), but I think there should be entries there?
What the friend also highlighted, is that when you have a dir shared via NFS, you can't share its subdir. The contrary might also be valid - when I have a dir shared I can't share it's parent / ancestor ?
My situation is like this: I have a bunch of Vagrant VM's related to projects located in ~/Developer/WebDeveloper/[a project name]. There are appropriate entries for all of them in /etc/exports. But from what I've heard (from the same friend, note that I'm not too knowledgable about Docker, just starting), Docker Machine (or boot2docker?) shares the whole home dir (~/) ? In which case the previous shares from Vagrant might block that operation?
Is there any log I can provide? Or a place to redirect this issue and let the Machine team investigate?
michaldudek
on 3 Feb 2016
I wonder if these may be related:
https://github.com/docker/machine/issues/2980
https://github.com/boot2docker/boot2docker/pull/1086
michaldudek
on 3 Feb 2016
Interesting development: I completely destroyed the default VM (through VirtualBox GUI) and launched Kitematic again and now everything works properly.
michaldudek
on 9 Feb 2016
I have already had some issues with Virtualbox shared folders in my OS X and boot2docker. Had to recreate the docker machine VM in order to get it working again. Now I'm having the same issue, where the volumes are not being mounted. But tried destroying the VM and creating it again. Didn't work for me this time.
andergmartins
on 14 Feb 2016
Apologies if I've misunderstood, but I think I have the same issue and I think the answer is here: http://stackoverflow.com/a/30512262/202168
It appears that, unfortunately, when using docker-machine the volumes are mounted _from the machine filesystem_ rather than from your local directory where you are running the compose.yml
Apparently there is a feature request on docker-machine here aiming to solve it.
anentropic
on 13 Mar 2016
And it seems simplest option currently is to manually copy the files to the machine using docker-machine scp command: http://stackoverflow.com/a/30688995/202168
anentropic
on 13 Mar 2016
@anentropic The trick is to have your Docker-Compose project under your user directory (/Users/... on Mac OS X or c:\users\... on Windows). These directories are automatically mounted into the Docker-Machine VM. If you do that, you shouldn't have to use rsync or scp.
nwinkler
on 14 Mar 2016
I do have everything under my user directory (OS X) but it didn't work. I was using a relative path in the compose yml
anentropic
on 14 Mar 2016
The user directory is only mounted for local virtualbox machines. I'm using a remote machine on digital ocean
anentropic
on 14 Mar 2016
Hey guys I'm having the same issue, the weird part is that it is working on osx and Windows but not on Debian.
This is my docker-compose.yml
django:
build: django
volumes:
- ./django/:/home/docker/django/
My Dockerfile
FROM python:2.7
RUN mkdir -p /home/docker/django/django/
ADD . /home/docker/django/django/
WORKDIR /home/docker/django/django/
CMD ["./command.sh"]
This is a light version of my project (django +react) you can find it at https://github.com/firetix/docker_bug
Running docker-compose build works properly but running docker-compose up result
[8] System error: exec: "./command.sh": stat ./command.sh: no such file or directory
Why does it work on both windows and osx but not Debian? /Users/ is shared folder on virutalbox GUI
in all operating systems. Any insight would be really helpfull
firetix
on 21 Apr 2016
I have the same problem here. It works fine on El Capitan but when I run it on the ubuntu docker-machine it has nothing in the volume.
This is my docker-compose volumes config.
I'm using the version 2.0
volumes:
- .:/share
Any ideas?
kaiomagalhaes
on 16 May 2016
@kaiomagalhaes I didn't find a solution to the problem using docker-machine But using the docker daemon directly on Linux distribution did solve the issue for me FYI.
firetix
on 16 May 2016
yeah @firetix I saw that it works on linux directly. My problem is that I'm trying to set my team to work with docker machine because of the advantages that it brings and to make it linear for all the team both who use MAC OS and Linux.
Thank you for you answer.
kaiomagalhaes
on 16 May 2016
Got this problem with the new Docker for Mac as well
MarcusSjolin
on 22 Jun 2016
Got the same problem.
I'm using kubuntu 16.04 as my Guest OS running docker-machine 0.7.0.
$ docker version:
Client:
Version: 1.11.2
API version: 1.23
Go version: go1.5.4
Git commit: b9f10c9
Built: Wed Jun 1 22:00:43 2016
OS/Arch: linux/amd64
Server:
Version: 1.11.2
API version: 1.23
Go version: go1.5.4
Git commit: b9f10c9
Built: Wed Jun 1 21:20:08 2016
OS/Arch: linux/amd64
$ docker-compose version
docker-compose version 1.8.0-rc1, build 9bf6bc6
docker-py version: 1.8.1
CPython version: 2.7.9
OpenSSL version: OpenSSL 1.0.1e 11 Feb 2013
My docker-compose.yml file:
version: '2'
services:
web:
build: .
volumes:
- .:/code
command: python web/manage_dev.py runserver 0.0.0.0:8000
ports:
- "8000:8000"
depends_on:
- db
db:
image: postgres:9.5
volumes:
- zippy-data:/var/lib/postgresql/data
ports:
- "5432:5432"
volumes:
zippy-data: {}
My Dockerfile:
FROM python:3.5
ENV PYTHONUNBUFFERED 1
RUN mkdir /code
WORKDIR /code
ADD requirements.txt /code/
RUN pip install -r requirements.txt
ADD . /code/
when I run $ docker-compose up I get the error:
$ docker-compose up
Creating network "zippymeals_default" with the default driver
Creating zippymeals_db_1
Creating zippymeals_web_1
Attaching to zippymeals_db_1, zippymeals_web_1
db_1 | LOG: database system was shut down at 2016-06-22 15:30:12 UTC
db_1 | LOG: MultiXact member wraparound protections are now enabled
db_1 | LOG: database system is ready to accept connections
db_1 | LOG: autovacuum launcher started
web_1 | python: can't open file 'web/manage_dev.py': [Errno 2] No such file or directory
zippymeals_web_1 exited with code 2
Any comment would be appreciated. I've read some people mention this is caused by the fact that docker-machine does not mount host directories correctly. I'm looking for workarounds, thank you.
reg3x
on 22 Jun 2016
I had the same issue. A co-worker installed docker (and docker-machine) using the OSX installer, and a "default" machine was set up for him, which mapped his "/Users" folder correctly.
My co-worker is on OSX and used the installed:
- The installer installed docker, docker-machine and docker-compose
- The installer set up a "default" machine, with "/Users" from the host being mapped to the default machine. You can verify this by logging into the machine (
docker-machine ssh) and seeing if the directory is populated as expected (ls /Users) - The volumes specified in docker-compose.yaml mapped correctly.
I'm on Ubuntu 14.04, and here is what I tried, and it didn't work:
- I installed docker from apt-get, and docker-machine / docker-compose from the docker website.
- I ran
docker-machine create --driver virtualbox defaultto set up a default machine. When I SSH into the machine, I can see that/homefrom the host has not been mounted. - The volumes specified in docker-compose are _not_ mapped correctly.
Here is what did work (my workaround):
# Create the "default" machine, but then stop it.
docker-machine create --driver virtualbox default
docker-machine stop
# Add a new shared folder through virtualbox
VBoxManage sharedfolder add default --name "userhome" --hostpath "/home/jason/"
# Turn the machine back on, and mount the share
docker-machine start
docker-machine ssh default 'sudo mkdir --parents /home/jason'
docker-machine ssh default 'sudo mount -t vboxsf userhome /home/jason'
# Check it works
docker-machine ssh default 'ls /home/jason'
# Start it up
docker-compose up
I hope this workaround helps someone,
jasononeil
on 6 Jul 2016
Also running into this :-( After docker-compose up -d, the mounts are empty; yet docker-compose run ... 'ls /the/folder' shows the content as expected. Environment:
Distributor ID: Ubuntu
Description: Ubuntu 14.04.3 LTS
Release: 14.04
Codename: trusty
Docker version 1.11.2, build b9f10c9
docker-compose version 1.7.1, build 6c29830
docker-py version: 1.8.1
CPython version: 2.7.6
OpenSSL version: OpenSSL 1.0.1f 6 Jan 2014
No docker-machine installed.
demobox
on 12 Jul 2016
I tried @jasononeil 's solution on my Macbook but it didn't work, now compose has shutted up completely:
(This is after running the commands of the workaround but with my /Users/wayne user path)
$ docker-machine start default
Starting "default"...
Machine "default" is already running.
$ docker-compose up -d
ERROR: Couldn't connect to Docker daemon - you might need to run `docker-machine start default`.
katywings
on 16 Jul 2016
I'm having this issue with the latest mac build 1.12.0-beta21.
I add a volume via docker-compose then it doesnt show up in the dockerfile.
Ermish
on 30 Jul 2016
@Ermish What do you mean by "it doesn't show up in the dockerfile"? If you mean it doesn't show up during the build process, that's to be expected - you can't mount volumes during a build.
@katywings It looks like you might need to run eval $(docker-machine env default). Even better, try out the just-released Docker for Mac, which handles volumes much better than Virtualbox.
aanand
on 1 Aug 2016
Hey @aanand, hopefully this helps:
If I add this to the dockerfile:
app:
...
volumes:
- ./host:/data
server:
...
volumes_from:
- app
In the server dockerfile, /data/ from the app volume does not exist.
Ermish
on 1 Aug 2016
So I had a similar issue recently. Here is how I fix it.
I was using Docker for Mac, and all files mount correctly in my local VM launched by VirtualBox. However those file don't exist when I deploy on AWS instance running Ubuntu.
The problem is caused by a misconfiguration in docker-compose.yml. Previously, it was like this:
app:
...
volumes:
- /path/on/mac:/path/in/container
...
To make it work, I did the following:
$ docker-machine scp /path/on/mac <ec2-instance>:/path/on/ubuntu
Then change the docker-compose.yml:
app:
...
volumes
- /path/on/ubuntu:/path/in/container
I think the reason is that docker-compose is loading the files on host machine, in this case the host is ec2 instance in Ubuntu, rather than the Mac client.
zhanghan177
on 2 Aug 2016
@Ermish That looks like a different problem. Could you open up a separate issue for it?
aanand
on 2 Aug 2016
I'm going to close this issue, as it's getting very noisy.
If you're having problems with a volume not mounting and you're using Docker Toolbox or any other form of Virtualbox VM, please try switching to Docker for Mac or Docker for Windows, both of which have now exited beta and have much better support for file syncing.
If you're having problems but you're not using Virtualbox at all, please open a new issue. Thanks!
aanand
on 2 Aug 2016
@aanand what's about linux?!
krayevidi
on 2 Aug 2016
Like I say, if you're having problems but not using Virtualbox, please open a new issue.
aanand
on 5 Aug 2016
@aanand Aanand, Sorry to write one more. However, while what you say may be ok on Docker for Windows, I just spent several hours this afternoon reading about the very slow syncing people have been consistently having for months with Docker for Mac.
Unless you at Docker have fixed that, and recently so I've missed the further discussion, it seems this goes in circles.
I personally could get by with pure Linux, or upgrade a machine to run the new kid on Windows if its teething is in appropriate state, but many in my community develop on Macs, so it would be good to have a clear answer of what the state of matters is.
Thank you,
Clive
p..s. to make clear why we want host volumes to work, since I know you guys want to hear always use cases, this is a classic one: a fine (actually) CMS with its big php area, full of Twig template files to edit as well as configuration and plugins in to develop, also its runtime cache and log areas. , And separately, there are all the assets, js and so forth materials, also in development, in the nginx web server public.
Host volumes are the only places appropriate for any of these, presume you'd agree.
narration-sd
on 14 Aug 2016
Hi there,
I've same issue on docker @OSX (see my version at the end of this post) leading to several annoying restart.
Then on Linux (see my version at the end of this post), I also noticed problem with volume but this time because it was mounted _after_ running the start command.
In that case my app was using the conf embedded in the image not the good one from my hosted volume. When looking inside the container after the app error to check the conf, I saw the correct configuration (the one from my host): so it was like the start command used the available conf at the time of the execution (ie: the one from the image) and after the volume was finally mounted. To avoid this volume issue I did rebuild my image to embed the good configuration.
This issue is really annoying and particularly if you have to manage volumes for database where it can be really dangerous depending database behavior. Is there any way to ask docker to not run the start command until all the volumes are mounted correctly ? Do you have some advices you can provide to avoid this issue ?
Kind regards,
╰─$ docker version
Client:
Version: 17.06.0-ce
API version: 1.30
Go version: go1.8.3
Git commit: 02c1d87
Built: Fri Jun 23 21:31:53 2017
OS/Arch: darwin/amd64
Server:
Version: 17.06.0-ce
API version: 1.30 (minimum version 1.12)
Go version: go1.8.3
Git commit: 02c1d87
Built: Fri Jun 23 21:51:55 2017
OS/Arch: linux/amd64
Experimental: true
╰─$ docker-compose version
docker-compose version 1.14.0, build c7bdf9e
docker-py version: 2.3.0
CPython version: 2.7.12
OpenSSL version: OpenSSL 1.0.2j 26 Sep 2016
╰─$ docker version
Client:
Version: 1.12.6
API version: 1.24
Go version: go1.6.2
Git commit: 78d1802
Built: Tue Jan 31 23:35:14 2017
OS/Arch: linux/amd64
Server:
Version: 1.12.6
API version: 1.24
Go version: go1.6.2
Git commit: 78d1802
Built: Tue Jan 31 23:35:14 2017
OS/Arch: linux/amd64
╰─$ docker-compose version
docker-compose version 1.10.0, build 4bd6f1a
docker-py version: 2.0.1
CPython version: 2.7.9
OpenSSL version: OpenSSL 1.0.1t 3 May 2016
mffrench
on 17 Jul 2017
I also noticed problem with volume but this time because it was mounted after running the start command.
Are you able to reproduce that? If so, please open an issue. I'm not really sure how that would happen, given that mounting a volume is part of the same step to create a container, not a separate API call.
thaJeztah
on 18 Jul 2017
I believe I have the same issue, I'm on windows 10.
version
Client:
Version: 17.06.0-ce
API version: 1.30
Go version: go1.8.3
Git commit: 02c1d87
Built: Fri Jun 23 21:30:30 2017
OS/Arch: windows/amd64
Server:
Version: 17.06.0-ce
API version: 1.30 (minimum version 1.12)
Go version: go1.8.3
Git commit: 02c1d87
Built: Fri Jun 23 21:51:55 2017
OS/Arch: linux/amd64
Experimental: true
and my docker compose file looks like this...
version: '2'
services:
wordpress:
image: wordpress:4.7-apache
ports:
- 8082:80
volumes:
- ./wp-content:/var/www/html/wp-content/themes
links:
- mariadb
environment:
WORDPRESS_DB_PASSWORD: wp
mariadb:
image: mariadb:10.1.22
ports:
- 8084:3306
volumes:
- ./mariadb:/var/lib/mysql
environment:
MYSQL_ROOT_PASSWORD: wp
So my question is this...why does docker not create my /wp-content and /mariadb directories automatically when i run docker-compose up -d? It does everything else but that.
Thats for any tips
blachawk
on 19 Jul 2017
I'm not on Windows myself, so can't verify if there's an issue; make sure disk sharing is enabled (https://docs.docker.com/docker-for-windows/#shared-drives).
So my question is this...why does docker not create my /wp-content and /mariadb directories automatically when i run docker-compose up -d? It does everything else but that.
Although docker _does_ indeed create those directories, best practice is to not rely on that. Automatically creating directories on the host when missing was a _bad_ decision in the past. When bind-mounting a host directory, effectively you're telling Docker: "here's content on the host (a directory, or a file), and I give a container permission to access that content" - not the other way round (i.e., not "start a container, and copy it's content to my host"). It's a seemingly "small" detail, but an important one in some cases.
This behaviour is the source of many "bug" reports. For example, using ./some-configuration-file.conf:/foo/config.conf will only work as expected if some-configuration-file.conf was present (and accessible by the docker daemon). If it's not found, docker creates a directory, and attempts to mount it as /foo/config.conf in the container (which will fail). Similar, ./oops-i-made-a-typo:/data will silently create a fresh, empty directory, and mount it in the container (resulting in surprising behaviour, because "files don't show up in the container")
Ideally, docker would _refuse_ to start the container if the file (or directory) was not found, so that you'd know the file (or directory) is not found on the host.
Worth knowing that when using Swarm services, or the --mount flag on docker run, auto-creation _is_ disabled, and does indeed produce such an error.
thaJeztah
on 19 Jul 2017
I am on Archlinux and get the same issue.
I tried to bind-mount my project python src folder in compose yaml. Try short syntax and long syntax, but there is not any files in /usr/src/app of the running container.
volumes:
- type: bind
source: .
target: /usr/src/app
volumes:
- '.:/usr/src/app'
md5-852f84756ae1dee2b40cc569f2a369f5
docker ssh <myvm>
df -Th
Filesystem Type Size Used Available Use% Mounted on
..
hosthome vboxsf 78.2G 25.1G 53.1G 32% /hosthome
...
md5-b6126bfa834fcc33c7852f896f1a53b8
Client:
Version: 17.10.0-ce
API version: 1.23
Go version: go1.9.1
Git commit: f4ffd2511c
Built: Wed Oct 18 23:08:56 2017
OS/Arch: linux/amd64
Server:
Version: 17.10.0-ce
API version: 1.33 (minimum version 1.12)
Go version: go1.8.3
Git commit: f4ffd25
Built: Tue Oct 17 19:05:23 2017
OS/Arch: linux/amd64
Experimental: false
$ docker-machine ls
NAME ACTIVE DRIVER STATE URL SWARM DOCKER ERRORS
vdev * virtualbox Running tcp://192.168.99.100:2376 v17.10.0-ce
wushuzh
on 4 Nov 2017
@christianbundy
I've had similar problem where, no matter how my python app was simple (imagine just hello.py that outputs 'hello world'), when running through docker-compose build && up commands I got error hello.py - not found.
It turned out that the issue was that virtualbox did not mount my folder (app) properly. This came as a rescue and resolved my problem in 5 minutes:
https://stackoverflow.com/questions/35005974/docker-ubuntu-virtualbox-volumes-directive-in-dockerfile-not-working/35013305#35013305
dusantrtica
on 18 Feb 2018
Try it running the docker-compose command with the -V option. That works for me when doing repeated compose up with changes in the composition.
Cheers.
IvensZambrano
on 11 Apr 2018
@thaJeztah many thanks on the explanation.
blachawk
on 11 Apr 2018
I had the same issue. turned out that shared files need to be enabled as well as running your application in the User folder.
I also had to restart my VM in Hyper V and restarted docker for windows after moving my project to C:/Users/...
hope this helps someone else
samwa
on 14 Aug 2018
Restarted docker for windows worked for me.
yang-zhang-syd
on 22 Oct 2018
Restarting Docker worked. I had the same problem on Docker for Mac Version: 18.06.1-ce-mac73 (26764)
lianhuilui
on 24 Oct 2018
I had a similar experience. I've developed under VB and rebooting system solved problem ¯_(ツ)_/¯
slava-nikulin
on 31 Oct 2018
I had a same problem on Windows 10 Pro.
Docker version is below.
Version 2.0.0.0-win81 (29211)
Channel: stable
Build: 4271b9e
-V option or restart docker didn't work for me.
I shutdowned Docker, deleted MobyLinuxVM.vhdx, and restarted Docker.
It works for me.
TakafumiFukuyo
on 3 Jan 2019
This happened to me because the project wasn't placed in C:/Users/...
Make sure your project is placed in that folder, otherwise the volumes cannot be mapped, since docker doesn't have access to those files.
waclock
on 14 Jan 2019
Thank @waclock so much,
You saved my life =))
minhlong
on 20 Feb 2019
Weird. Just experienced the same. Restarting docker fixed it.
felschr
on 4 Apr 2019
I had the same issue on ubuntu and restarting docker service also worked. This is indeed an issue with the docker daemon and not the compose.
amirmd76
on 25 Jun 2019
Docker for mac was running for too long. Restarting docker solved the mounting problem.
girisagar46
on 24 Jul 2019
@wushuzh did you ever figure this out on archlinux. I'm seeing something very familiar to what you describe.
ezmiller
on 9 Aug 2019
@wushuzh and @ezmiller -- My experience on Arch is to not use . or ~ when specifying host directories. It is kind of sucky having to type long paths but I use absolute paths where ever possible. This is probably not necessary, but I have avoided a lot of headaches this way.
32footsteps
on 21 Aug 2019
[Beginner here]
I was running into similar issues and I just wanted to post out what I did wrong in my particular case.
Took me a while to figure out my problem, but essentially I was mounting the volumes using
/mnt/c/MyUserName/.../:/path/to/docker/bind/mount
and starting the dockers from Windows. Had forgotten that I normally mount them from WSL.
Stupidly obvious, but still took me 20 minutes :D
NPellet
on 1 Oct 2020
Related issues
komuw
·
3Comments
29e7e280-0d1c-4bba-98fe-f7cd3ca7500a
·
3Comments
HackerWilson
·
3Comments
dazorni
·
3Comments
squeaky-pl
·
3Comments
Most helpful comment
Try it running the docker-compose command with the -V option. That works for me when doing repeated compose up with changes in the composition.
Cheers.