Compose: python: can't open file 'app.py'

Created on 29 Jun 2015 · 10Comments · Source: docker/compose

Working through the compose tutorial and deploying to a proven working swarm running on Google Compute Engine (GCE):

https://docs.docker.com/compose/

I'm able to build/run the Dockerfile representing the python app standalone.

I've deployed and confirmed 'hello-world' to the swarm cluster.

I can run the composed app on my localhost.

I am unable to run the composed app either on a single instance (docker) or swam on GCE.

I'm perplexed as to the problem and to what further debugging steps I could try.

Here's the output running on a single instance on GCE. NB the lack of any reference in the output to the web app.

docker-compose up Recreating compose_redis_1... Recreating compose_web_1... Attaching to compose_redis_1 redis_1 | 1:C 29 Jun 16:12:57.109 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf redis_1 | _._ redis_1 | _.-``__ ''-._ redis_1 | _.-`` `. `_. ''-._ Redis 3.0.2 (00000000/0) 64 bit redis_1 | .-`` .-```. ```\/ _.,_ ''-._ redis_1 | ( ' , .-` | `, ) Running in standalone mode redis_1 | |`-._`-...-` __...-.``-._|'` _.-'| Port: 6379 redis_1 | | `-._ `._ / _.-' | PID: 1 redis_1 | `-._ `-._ `-./ _.-' _.-' redis_1 | |`-._`-._ `-.__.-' _.-'_.-'| redis_1 | | `-._`-._ _.-'_.-' | http://redis.io redis_1 | `-._ `-._`-.__.-'_.-' _.-' redis_1 | |`-._`-._ `-.__.-' _.-'_.-'| redis_1 | | `-._`-._ _.-'_.-' | redis_1 | `-._ `-._`-.__.-'_.-' _.-' redis_1 | `-._ `-.__.-' _.-' redis_1 | `-._ _.-' redis_1 | `-.__.-' redis_1 | redis_1 | 1:M 29 Jun 16:12:57.130 # Server started, Redis version 3.0.2 redis_1 | 1:M 29 Jun 16:12:57.132 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect. redis_1 | 1:M 29 Jun 16:12:57.132 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled. redis_1 | 1:M 29 Jun 16:12:57.132 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128. redis_1 | 1:M 29 Jun 16:12:57.132 * DB loaded from disk: 0.000 seconds redis_1 | 1:M 29 Jun 16:12:57.132 * The server is now ready to accept connections on port 6379

If I list the containers on that instance:

docker ps -a
CONTAINER ID        IMAGE               COMMAND                CREATED              STATUS                          PORTS               NAMES
8cf338d2d573        compose_web         "/bin/sh -c 'python    About a minute ago   Exited (2) About a minute ago                       compose_web_1       
7c8e99d581c7        redis               "/entrypoint.sh redi   About a minute ago   Up About a minute               6379/tcp            compose_redis_1

and the logs show that it's not finding the app.py:

docker logs compose_web_1
python: can't open file 'app.py': [Errno 2] No such file or directory

If I run the app on the swarm, I get "Error: image library/compose_web:latest not found":

docker-compose up
Creating compose_redis_1...
Building web...
Step 0 : FROM python:2.7
2.7: Pulling from python
64e5325c0d9d: Pull complete
bf84c1d84a8f: Pull complete
87de57de6955: Pull complete
6a974bea7c0d: Pull complete
4b708e6c0f6c: Pull complete
12fef6820370: Pull complete
69ca4f45dafd: Pull complete
11ea72ec71a1: Pull complete
192e879377de: Pull complete
a6ea00825040: Pull complete
e1f5c799d062: Pull complete
5d75f900f6bb: Pull complete
65c00af7c3f9: Already exists
python:2.7: The image you are pulling has been verified. Important: image verification is a tech preview feature and should not be relied on to provide security.
Digest: sha256:215b29097c20e32e9c74939e12ae0634260091eaaa96720a6a7110b876b9bf02
Status: Downloaded newer image for python:2.7
 ---> 65c00af7c3f9
Step 1 : ADD . /code
 ---> 6824583dd7d3
Removing intermediate container adad95e28bce
Step 2 : WORKDIR /code
 ---> Running in f1941e79af9c
 ---> 70145a0d969e
Removing intermediate container f1941e79af9c
Step 3 : RUN pip install -r requirements.txt
 ---> Running in bf1e3b181b4d
Collecting flask (from -r requirements.txt (line 1))
  Downloading Flask-0.10.1.tar.gz (544kB)
Collecting redis (from -r requirements.txt (line 2))
  Downloading redis-2.10.3.tar.gz (86kB)
Collecting Werkzeug>=0.7 (from flask->-r requirements.txt (line 1))
  Downloading Werkzeug-0.10.4-py2.py3-none-any.whl (293kB)
Collecting Jinja2>=2.4 (from flask->-r requirements.txt (line 1))
  Downloading Jinja2-2.7.3.tar.gz (378kB)
Collecting itsdangerous>=0.21 (from flask->-r requirements.txt (line 1))
  Downloading itsdangerous-0.24.tar.gz (46kB)
Collecting markupsafe (from Jinja2>=2.4->flask->-r requirements.txt (line 1))
  Downloading MarkupSafe-0.23.tar.gz
Building wheels for collected packages: flask, redis, Jinja2, itsdangerous, markupsafe
  Running setup.py bdist_wheel for flask
  Stored in directory: /root/.cache/pip/wheels/d2/db/61/cb9b80526b8f3ba89248ec0a29d6da1bb6013681c930fca987
  Running setup.py bdist_wheel for redis
  Stored in directory: /root/.cache/pip/wheels/b4/82/09/7713ff9bc6f6a58e41b0a89433aed5d2293fd5868f76adef94
  Running setup.py bdist_wheel for Jinja2
  Stored in directory: /root/.cache/pip/wheels/b3/c0/62/8908d15b90a7de8bd15e2e6e5f5ba1398c0d9940d62a6bf8f5
  Running setup.py bdist_wheel for itsdangerous
  Stored in directory: /root/.cache/pip/wheels/97/c0/b8/b37c320ff57e15f993ba0ac98013eee778920b4a7b3ebae3cf
  Running setup.py bdist_wheel for markupsafe
  Stored in directory: /root/.cache/pip/wheels/94/a7/79/f79a998b64c1281cb99fa9bbd33cfc9b8b5775f438218d17a7
Successfully built flask redis Jinja2 itsdangerous markupsafe
Installing collected packages: Werkzeug, markupsafe, Jinja2, itsdangerous, flask, redis
Successfully installed Jinja2-2.7.3 Werkzeug-0.10.4 flask-0.10.1 itsdangerous-0.24 markupsafe-0.23 redis-2.10.3
 ---> 773f58b40337
Removing intermediate container bf1e3b181b4d
Step 4 : CMD python app.py
 ---> Running in 388be301e6df
 ---> 058900b282c5
Removing intermediate container 388be301e6df
Successfully built 058900b282c5
Creating compose_web_1...
Error: image library/compose_web:latest not found

If I run docker-compose build before up, I get further but, when I run the app, I get a slightly different output (this time: web_1 | python: can't open file 'app.py') but the same behavior:

docker-compose up Creating compose_redis_1... Creating compose_web_1... Attaching to compose_redis_1, compose_web_1 redis_1 | 1:C 29 Jun 17:25:50.944 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf redis_1 | _._ redis_1 | _.-``__ ''-._ redis_1 | _.-`` `. `_. ''-._ Redis 3.0.2 (00000000/0) 64 bit redis_1 | .-`` .-```. ```\/ _.,_ ''-._ redis_1 | ( ' , .-` | `, ) Running in standalone mode redis_1 | |`-._`-...-` __...-.``-._|'` _.-'| Port: 6379 redis_1 | | `-._ `._ / _.-' | PID: 1 redis_1 | `-._ `-._ `-./ _.-' _.-' redis_1 | |`-._`-._ `-.__.-' _.-'_.-'| redis_1 | | `-._`-._ _.-'_.-' | http://redis.io redis_1 | `-._ `-._`-.__.-'_.-' _.-' redis_1 | |`-._`-._ `-.__.-' _.-'_.-'| redis_1 | | `-._`-._ _.-'_.-' | redis_1 | `-._ `-._`-.__.-'_.-' _.-' redis_1 | `-._ `-.__.-' _.-' redis_1 | `-._ _.-' redis_1 | `-.__.-' redis_1 | redis_1 | 1:M 29 Jun 17:25:50.945 # Server started, Redis version 3.0.2 redis_1 | 1:M 29 Jun 17:25:50.945 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect. redis_1 | 1:M 29 Jun 17:25:50.945 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled. redis_1 | 1:M 29 Jun 17:25:50.945 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128. redis_1 | 1:M 29 Jun 17:25:50.945 * The server is now ready to accept connections on port 6379 web_1 | python: can't open file 'app.py': [Errno 2] No such file or directory

docker ps
CONTAINER ID        IMAGE               COMMAND                CREATED              STATUS              PORTS               NAMES
9a8678b2abdd        compose_web         "/bin/sh -c 'python    About a minute ago   Up About a minute                       swarm-150629-node-00/compose_web_1                                                                                                                                            
a9a2e0bdc159        redis               "/entrypoint.sh redi   About a minute ago   Up About a minute   6379/tcp            swarm-150629-node-00/compose_redis_1

docker logs 9a86
python: can't open file 'app.py': [Errno 2] No such file or directory

Full disclosure: I'm a Google employee in cloud but not a developer

kindocs

Source

DazWilkin

Most helpful comment

when running an app on a remote host, you'll need to remove any volumes entries that mount anything in from a local directory

Could you add this to the tutorial?

mweggen on 10 Feb 2016

👍12

All 10 comments

Alright, the issue appears to primarily be a consequence of the ADD/WORKDIR in the Dockerfile:

ADD . /code
WORKDIR /code

Conflicting with the volume mapping in the docker-compose.yml

  volumes:
   - .:/code

I may have misread the instructions but had both.

After many attempts to understand why this seems to work on my localhost but not on GCE, by stopping the conflict (either renaming the ADD destination or renaming|removing the volumes reference), the code now works.

If volumes is used, the ADD command should be redundant, shouldn't it?

The problem appears to be with volumes and the self-contained Dockerfile works.

DazWilkin on 30 Jun 2015

👍4

Yep: when running an app on a remote host, you'll need to remove any volumes entries that mount anything in from a local directory.

The reason we promote both adding your code to the image _and_ mounting it as a volume is so that doing a docker-compose build gets you an image that's ready to deploy to a remote server, but whose code can still be mounted when developing locally so it can be live-reloaded.

If you want to use Compose to both develop an app locally and deploy it to the cloud, the best practice is to have a file for each - docker-compose.yml for local development and something else (e.g. remote.yml) for remote deployment - and use the -f flag to pass the alternate file in when deploying.

You can use the extends option to reduce duplication between both files.

aanand on 1 Jul 2015

👍8

If I can't use the "build" directive in service definitions when compose is pointed at a remote host, that needs to be documented.

allingeek on 8 Aug 2015

@allingeek You _can_ use the build directive against a remote machine (the local directory is tar-ed and sent to the server, just like with docker build).

You _can't_ use volumes with host paths on your local machine when the daemon is remote, either with Compose or plain docker run. We could perhaps note that in the YAML reference for volumes.

aanand on 10 Aug 2015

when running an app on a remote host, you'll need to remove any volumes entries that mount anything in from a local directory

Could you add this to the tutorial?

mweggen on 10 Feb 2016

👍12

I just got caught out by this running the docker bday tutorial on 1.12.0-rc2-beta16 (build 4760) on Windows. Had to comment out:

    #volumes:
    # - ./voting-app:/app

    #volumes:
    #  - ./result-app:/app

carwyn on 20 Jun 2016

👍5 ❤1

Check for selinux policy, by disabling selinux policy, it's working fine for me

surajnarwade on 4 Oct 2016

👍2

I encounter the same problem using Win8.1 and run this tutorial under D:\DockerWorkspace\LearnDockerCompose\composetest, I find that it is Windows limit.

Notice the Tip in this section Step 3: Define services in a Compose file.

So I move this project to the Users directory (cd ~) and then it works fine. Or remove the volumes declaration in the docker-compose.yml.

May this words can help the follows, forgive my pool English ):