Cookiecutter-django: Error using docker with dev.yml

[mimischi]

I'm not able to start my docker image with the current dev.yml and Dockerfile-dev. I cloned the cookiecutter repo as described in the documentation. Those two files are therefore in the same state as this repo here.

The build performs without any problems

docker-compose -f dev.yml build

Starting the containers leads to an error though:

$ docker-compose -f dev.yml up
djangogeomat_postgres_1 is up-to-date
Starting djangogeomat_django_1
Cannot start container 63db2e44ef7a02477b57706ac1cfd31b9677257051f1493a628b524a7d716371: [8] System error: no such file or directory

I'm not experienced enough with Docker to know how to solve that problem. Using docker-compose.yml works, but this isn't the right solution for a development environment I think?

$ docker-compose --version
docker-compose version: 1.5.1
$ docker-machine --version
C:\Program Files\Docker Toolbox\docker-machine.exe version 0.5.0 (04cfa58)

[jayfk]

Your project directory isn't readable by the virtual machine where the docker deamon is running. I don't know how docker-toolbox works on windows, but you have to make sure that your project directory is mounted into the virtual machine in the same way as on your host machine.

The dev.yml configuration mounts the project directory into the container so that code changes can be detected automatically.

[mimischi]

Just to make sure that I'm on the right track: My actual django code is in a directory D:/Code/project

I now have to make sure, that the virtual machine is able to access the files in D:/Code/project. Is this correct?

[jayfk]

Yes. You could try to copy the project files in your user directory (that's mounted by default on OS X, I assume also on windows) and start from there. I think this will be a lot easier then mounting a custom directory since windows and the virtual machine are using a different directory naming scheme.

There's a good chance that docker-toolbox is working this out for your user directory automatically, so I'd start looking there.

[mimischi]

Ok. So now I'm getting an different error. Having my project inside ~/ or adding my normal project_dir to the VirtualBox shared folders will end up with:

$ docker-compose -f dev.yml up
Creating djangogeomat_postgres_1
Cannot start container 37937b9f4a1b66363340b358149d94574955e9c362553541388a10e93c04d604: mkdir /data: file exists

I followed the developing locally tutorial (http://cookiecutter-django.readthedocs.org/en/latest/developing-locally-docker.html) and setup the persistent /data partition. If I don't set this up, then I get the "System error: no such file or directory" error from above.

[mimischi]

Played around with it for a bit. So there are two possible scenarios (and it seems like the defined shared folders inside VirtualBox don't matter):

If I create a persistent data folder using (and restart the machine afterwards)

echo 'ln -sfn /mnt/sda1/data /data' >> /var/lib/boot2docker/bootlocal.sh

Then I get the "mkdir /data: file exists" message.

If I don't create the persistent data folder, then I get the "System error: no such file or directory" message from my first post.

Using this solution (https://github.com/pydanny/cookiecutter-django/issues/328#issuecomment-140186292) doesn't help and yields the same "mkdir /data: file exists" error.

[jayfk]

Played around with it for a bit. So there are two possible scenarios (and it seems like the defined shared folders inside VirtualBox don't matter):

Yeah, they don't matter in this case because the folder is intended to be used only by the virtual machine.

What does cat /var/lib/boot2docker/bootlocal.sh and ls -ln /data print out?

For background info: The underlying problem are docker file permissions in a mixed environment (that's why we have to create the /data folder in the first place). You can read more about that here.

[mimischi]

root@dev:/home/docker# cat /var/lib/boot2docker/bootlocal.sh

ln -sfn /mnt/sda1/data /data
root@dev:/home/docker# ls -ln /data
lrwxrwxrwx 1 0 0 14 Nov 17 16:50 /data -> /mnt/sda1/data

Creating the symlink works. Right now the first line of bootlocal.sh is empty, but this also doesn't seem to matter.

[jayfk]

That's odd. I've created a new machine according to the docs to confirm this is still working with the newest docker and had no problems. Can you sudo ls /mnt/sda1/data? If not, create the directory with sudo mkdir /mnt/sda1/data.

As a last resort you can trash the machine and create a new one from scratch with docker-machine rm

[burhan]

To make your life a whole heck of a lot easier, I recommend the following:

1. Use virtualbox to spin up an ubuntu instance.
2. Install docker on the ubuntu instance.
3. Copy your source code to the ubuntu instance.
4. Export a share from your ubuntu instance, which you mount in Windows as a normal network share.
5. Work as usual from Windows.

This is similar to what docker-machine and docker-toolbox are trying to do; they also create a common share directory that can be written on Windows, but I find that instead of struggling with permissions on Windows - having the directory already on the docker host, but mounted as a network share works better.

I am still playing around with docker-toolbox (docker-machine is not the issue) to see how to get a sure fire fail-safe way to get it setup on Windows.

[mimischi]

After starting to fumble around again, I finally found a solution to get it to work with Windows, but I haven't checked if it breaks it on Linux/OS X.

The solution to it can be found here: https://github.com/docker/compose/issues/2301#issuecomment-158025207

I changed the line:

ENTRYPOINT ["/entrypoint.sh"]

in Dockerfile-dev to:

ENTRYPOINT ["/bin/bash", "-e", "/entrypoint.sh"]

Then I had to remove the line

set -e

from entrypoint.sh, or otherwise it would throw an error (even without the "-e" in the ENTRYPOINT).

In VirtualBox I added a shared folder (Name: /mycode, Path: D:/path/to/my/code). The slash in front it "mycode" seems to be important. Also: don't check "Mount automatically", because this doesn't seem to do anything. Just make sure to check "Make permanent". The next part I got from here: http://stackoverflow.com/questions/30864466/whats-the-best-way-to-share-files-from-windows-to-boot2docker-vm
Logged into the docker-machine (docker-machine ssh ) and mounted the shared folder:

sudo mkdir /home/docker/myapp
sudo mount -t vboxsf /mycode /home/docker/myapp

Now my Windows folder with my code inside is accessible under /home/docker/myapp. The last thing I needed to do was update dev.yml to:

volumes:
  - /home/docker/mycode:/app

I haven't tested yet, if ".:/app" works, but there will probably be some tinkering I need to do, but will keep you updated.

Coming to the PostgreSQL /data bug: Your RTD doesn't cover creating the folder in /mnt/sda1/data, therefore it can't be mounted correctly and this error comes up. After creating it once (sudo mkdir /mnt/sda1/data), the persistant storage seems to work.

Also one reminder:

$ docker-compose run django python manage.py migrate
$ docker-compose run django python manage.py createsuperuser

I got stuck here for waaaaaay to long and didn't understand why it was always trying to install redis in the local development. You probably should say "docker-compose -f dev.yml", as one is still trying to get it to run locally and not in production :)

[mimischi]

OK. I just managed to get it working with

volumes:
   - .:/app

on Windows.

"." is a relative path and the Docker therefore always tries to find the SAME path inside the VM, to connect the /app folder to.
My project code actually is in C:/D/Daten/Programmierung/GitHub/myproject

Add the folder "C:/D/Daten/Programmierung/GitHub" to the VirtualBox share inside the VM as described above. Name it "/GitHub" (with the / in front!)

docker-machine ssh dev
mkdir -p /c/D/Daten/Programmierung/GitHub
mount -t vboxsf /GitHub /c/D/Daten/Programmierung/GitHub

Now all my projects in my Windows folder (C:/D/Daten/Programmierung/GitHub) are accesible inside the VM at /c/D/Daten/Programmierung/GitHub and this is exactly where Docker will try to connect the volume "/app" to, as it tries to map your working directory directly to the VM. (? not sure what it really does, but it looks like it..?)

My /var/lib/boot2docker/bootlocal.sh:

docker@dev:~$ cat /var/lib/boot2docker/bootlocal.sh
ln -sfn /mnt/sda1/data /data
mkdir -p /c/D/Daten/Programmierung/GitHub
mount -t vboxsf /GitHub /c/D/Daten/Programmierung/GitHub/

I will try out this two needed changes (entrypoint.sh and Dockerfile-dev) in OS X/Ubuntu and report back, if it breaks anything.

[jayfk]

Do you have a working setup now?

[mimischi]

Yes, so far tried it in Windows and OS X - and it works so far! As described above, there are some changes needed to be done to the files though.

Change

 ENTRYPOINT ["/entrypoint.sh"]

in Dockerfile-dev to:

ENTRYPOINT ["/bin/bash", "-e", "/entrypoint.sh"]

Remove

set -e

from entrypoint.sh. Create /mnt/sda1/data with

mkdir /mnt/sda1/data

and for an Windows environment update the bootlocal.sh to

docker@dev:~$ cat /var/lib/boot2docker/bootlocal.sh
ln -sfn /mnt/sda1/data /data
mkdir -p /c/D/Daten/Programmierung/GitHub
mount -t vboxsf /GitHub /c/D/Daten/Programmierung/GitHub/

The folder "/GitHub" is the share that I created in VirtualBox and it is pointing to my local folder "C:/D/Daten/Programmierung/GitHub" with the option "Make permanent" ticket (without auto-mount!). That folder contains all my project files and therefore I can use the same docker-machine for different projects.

Should I create a pull request for those few changes? I haven't tested those two file changes in Linux now, but as it's working on OS X it also should on Linux?

[jayfk]

Should I create a pull request for those few changes? I haven't tested those two file changes in Linux now, but as it's working on OS X it also should on Linux?

Maybe we could add a section Troubleshooting on Windows to the docs?

[pydanny]

:+1: to Troubleshooting on Windows.

[pydanny]

Is this resolved? Can I close this issue?

[mimischi]

Using Docker for Windows now, not having the problem anymore.

[michael-wise]

Edit: Yikes, just realized this thread isn't for docker, but cookiecutter-django. I feel this might add something--but definitely delete this comment as needed.

Experiencing this issue after repeatedly using docker-compose up on the rails docker tutorial, making changes to files in the shared volume, and executing various commands. Anyhow, restarting docker/cpu fixed this issue for me today, and once in the past. I have no encountered the issue otherwise.

[Stanton-B]

Hello, and thanks to everyone who figured this out, especially mimischi! Just a couple further instructions to help with anyone doing this on Linux for whatever reason (I'm on Linux Mint 18, which uses Ubuntu Xenial):

If you are following mimischi's instructions from above, you will first need to use VBoxManage from your local bash to create your shared folder on the VirtualBox VM. Reference.

Assuming your local project root folder is /home/UserName/ParentFolder/DjangoProjectRoot, and your VM is named dev (it will be named whatever your docker-machine is named, I think), use:

$docker-machine stop dev
$VBoxManage sharedfolder add dev --name /ParentFolder --hostpath /home/UserName/ParentFolder
$docker-machine start dev

After this you can SSH into the dev machine, and manually edit your bootlocal.sh script like so:

$docker-machine ssh dev

$sudo su (change to root)
$echo "ln -sfn /mnt/sda1/data /data" >> /var/lib/boot2docker/bootlocal.sh
$echo "mkdir -p /home/UserName/ParentFolder" >> /var/lib/boot2docker/bootlocal.sh
$echo "mount -t vboxsf /ParentFolder /home/UserName/ParentFolder" >> /var/lib/boot2docker/bootlocal.sh

Then ctrl+d out of of the VM into your local shell, use docker-compose to build the image (from your project root, e.g. /home/UserName/ParentFolder/DjangoProjectRoot), restart the VM using docker-machine, then you should be able to put the local server up using docker-compose:

eval $(docker-machine env dev) (important!)
docker-compose -f local.yml build
docker-machine restart dev
docker-compose -f local.yml up

This has worked for me on both a desktop and a laptop running Linux Mint 18.2.