Compose: docker-compose up seems to get the wrong image... ?

Created on 17 May 2016  ยท  31Comments  ยท  Source: docker/compose

I am doing something pretty simple. I have a docker-compose.yml file:

  tsdb:
    build:
      context: ./tsdb_prod
    ....

and in the Dockerfile in that directory I have:

FROM influxdb:0.13
COPY file_prod /etc/file_prod

and I have the file_prod file in that directory.

running docker-compose build with --no-cache to rebuild my image after changes:

0s ๐Ÿ‘  $> docker-compose build --no-cache

[snip]

Building tsdb
Step 1 : FROM influxdb:0.13
 ---> c7a58aa7ce3b
Step 2 : COPY file_prod /etc/file_prod
 ---> ff2aa86452fd
Removing intermediate container 91c3eafd328c
Successfully built ff2aa86452fd
2s ๐Ÿ‘  $> docker run --rm -it ff2aa86452fd /bin/bash
root@0e660d6a1784:/# ls /etc/file_prod
/etc/file_prod

But when I do my docker-compose up I see

2s ๐Ÿ‘  $> docker-compose up
Recreating task_tsdb_1

[snip]

tsdb_1   | run: open server: open service: open "/etc/file_prod": no such file or directory

I'm pretty new to Docker. I'm sure I'm missing something stupid here. Is it building one image but starting another? I do see

2s ๐Ÿ‘  $> docker ps -a
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS                      PORTS               NAMES
f742363da3aa        58fcf34dd924        "/entrypoint.sh influ"   44 minutes ago      Exited (1) 34 minutes ago                       task_tsdb_1

[snip]

Not sure what's going on with that. I would appreciate anyone's insight.

Most helpful comment

Weird thing is, if I run the app with docker run, it's fine. If I run it with docker-compose up, it's not. It's almost as if docker-compose were picking the wrong image to spin the container from - an intermediary image, maybe?

All 31 comments

OK. I found something last night that seemed to explain and fix this, and now I have the same problem again this morning. Grr.

more of my docker-compose.yml:

version: '2'
services:

  tsdata:
    build:
      context: ./tsdb_prod
    volumes:
      - /var/lib/influxdb
    command: "true"

  tsdb:
    build:
      context: ./tsdb_prod
    restart: always
    environment:

[snip]

    volumes_from:
      - tsdata
    ports:

[snip]

My docker-compose build output:

3s ๐Ÿ‘  $> docker-compose build --no-cache
Building tsdata
Step 1 : FROM influxdb:0.13
 ---> c7a58aa7ce3b
Step 2 : COPY forelight_data.pem /etc/file_prod
 ---> 22d88415e029
Removing intermediate container 115d46c1e419
Successfully built 22d88415e029
Building tsdb
Step 1 : FROM influxdb:0.13
 ---> c7a58aa7ce3b
Step 2 : COPY forelight_data.pem /etc/file_prod
 ---> 78a43a107f4e
Removing intermediate container fa2aba9694e4
Successfully built 78a43a107f4e
35s ๐Ÿ‘  $> docker run --rm -it 78a43a107f4e /bin/bash
root@ea6546277008:/# ls /etc/file_prod
/etc/file_prod

My docker-compose up output:

1s ๐Ÿ‘  $> docker-compose up
Creating task_tsdata_1
Creating task_tsdb_1
Attaching to task_tsdata_1, task_tsdb_1
[snip]
tsdb_1   | run: open server: open service: open "/etc/file_prod": no such file or directory

same dockerfile as above.

so the file is there in the image, but the container docker-compose produces doesn't have it... ?

Can you try running docker-compose down -v --rmi then run build / up again?

I can and I will, after I deal with a couple other things. Though that
sounds like it will destroy my data volume. Which is fine... now. Any
solution that will involve that is pretty much out after this week, though.

Does compose get wedged this way often? If that's the case I'm inclined to
go with a host directory instead of a data volume.

On Wed, May 18, 2016 at 4:35 PM Joffrey F [email protected] wrote:

Can you try running docker-compose down -v --rmi then run build / up
again?

โ€”
You are receiving this because you modified the open/close state.
Reply to this email directly or view it on GitHub
https://github.com/docker/compose/issues/3472#issuecomment-220165334

No - sorry about the misunderstanding. I'm not proposing it as a solution _per se_, just as a debug step.

On somewhat related note, your config could potentially be made quite simpler by using named volumes instead of the volumes_from trickery you're currently using. And it would allow you to manage volumes independently of containers, too.

HTH

Nope. Same message.

I went back and deleted all my images except my copy of the base influxdb image. Then I did docker-compose build and up and got the same message. I list my images now and get

4s ๐Ÿ‘  $> docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
task_tsdata         latest              2fba08d97ba7        24 seconds ago      234.7 MB
task_tsdb           latest              2fba08d97ba7        24 seconds ago      234.7 MB
influxdb            0.13                c7a58aa7ce3b        7 days ago          234.7 MB

so it's finding my dockerfile from the context, finding the InfluxDB image, but not copying the file... ?

Looking at the named volumes doc... the volumes_from 'trickery' is suggested by a lot of sources. Not that I'm attached to it (this week, anyway). Having an explicit way to declare a volume does look cleaner.

Though I will say, the 'external' parameter is confusing. If they mean it mounts on the Docker host, then where? It looks like it puts it in the same place specified in the referring service. But then if there are multiple services referring to the same volume trying to locate it in different places does it error out? That seems inconsistent with Docker's design in general. I suspect I'm misunderstanding something.

ok closer...

0s ๐Ÿ’ฉ  $> docker-compose build
Building tsdata
Step 1 : FROM influxdb:0.13
 ---> c7a58aa7ce3b
Step 2 : COPY file_prod /etc/file_prod
 ---> Using cache
 ---> 0914f84be4a1
Successfully built 0914f84be4a1
Building tsdb
Step 1 : FROM influxdb:0.13
 ---> c7a58aa7ce3b
Step 2 : COPY file_prod /etc/file_prod
 ---> Using cache
 ---> 0914f84be4a1
Successfully built 0914f84be4a1
0s ๐Ÿ‘  $> docker images -a
REPOSITORY          TAG                 IMAGE ID            CREATED              SIZE
task_tsdata         latest              0914f84be4a1        About a minute ago   234.7 MB
task_tsdb           latest              0914f84be4a1        About a minute ago   234.7 MB
influxdb            0.13                c7a58aa7ce3b        7 days ago           234.7 MB

so it's copying the file (and I can make it bomb if I introduce a misspelling in the filename in my dockerfile).

But the file isn't showing up in my container.

And also I don't see a clear way to answer the question: 'what image is container x based on?'

aha, inspect. No surprises... but no explanations either.

0s ๐Ÿ‘  $> docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS                              PORTS                NAMES
01bf4a397710        task_tsdb           "/entrypoint.sh influ"   52 minutes ago      Restarting (1) 25 minutes ago       8083/tcp, 8086/tcp   task_tsdb_run_3
0s ๐Ÿ‘  $> docker inspect 01bf4a397710 | grep Image
        "Image": "sha256:0914f84be4a19ba5ad25f7404ede741c06afe63ddb06374d855df438c33a925a",
0s ๐Ÿ‘  $> docker images -a
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
task_tsdata         latest              0914f84be4a1        About an hour ago   234.7 MB
task_tsdb           latest              0914f84be4a1        About an hour ago   234.7 MB
influxdb            0.13                c7a58aa7ce3b        7 days ago          234.7 MB

and when I

0s ๐Ÿ‘  $> docker run --rm -it 0914f84be4a1 /bin/bash
root@6e93d6b39419:/# ls /etc/file_prod
/etc/file_prod
root@6e93d6b39419:/# exit

and then I do docker-compose up and see the same message.

That's weird. I'll take a closer look and update when I have more info.

Looking at the named volumes doc... the volumes_from 'trickery' is suggested by a lot of sources. Not that I'm attached to it (this week, anyway). Having an explicit way to declare a volume does look cleaner.

Yeah, named volumes didn't exist and volume management through the API is a pretty recent feature, so that was indeed the conventional solution for a long time.

Though I will say, the 'external' parameter is confusing.

In that context, external just means it's not managed by Compose (i.e. it will not attempt to create or destroy it at any point, and its name will not be namespaced).

actually, that's kind of ideal. I can create a volume with docker volume and use that, and then when I get compose working I can just use external. Nice.

More messing around:

I started messing with docker volume, and found I had a huge list of volumes I had created in the course of my messing around. So I l deleted all of those, and cleaned out all my containers and images, except the base one I got from Docker Hub, redid build and up, and... got the same result.

Then I switched to your recommended volume configuration, and upgraded my Docker client from 1.10 to 1.11.1, and cleaned everything out again, containers, images, volumes, and... got the same result.

Then I deleted everything including my base image and started again and... got the same result.

So I am out of bullets. I'm going back to my direct docker invocations (which work) for now and using those. If you want I can send you my dockerfile directly--I'd rather not post it here. See if you can reproduce it.

Sure, you can send it to [redacted]. I've tried to create a simple reproduction case but everything worked fine.

I'm having the same problem here.

Further, if I docker run image, the file is there. If I docker-compose up, the file is not there.

Exact same behavior on the latest beta on OSX and on Windows.

PS C:\Users\pedro> docker-compose version
docker-compose version 1.7.1, build 0a9ab35
docker-py version: 1.8.1
CPython version: 2.7.11
OpenSSL version: OpenSSL 1.0.2d 9 Jul 2015

@pedrovanzella InfluxDB as well?

Nope, it's a rails app. Base image is rails:latest.

@shin- found it, in my case. In his words from an offline email

The quotes around the path weren't being parsed out, and as such influxdb was trying to access "/etc/..., which, unsurprisingly, isn't a directory.

I had a bunch of fields quoted that didn't want to be quoted in docker-compose.yml. That was the one that was bombing for me, but there were others.

Anything to add, @shin- ?

@pedrovanzella Make sure the paths are correct - that also means checking for quotes that aren't parsed out, like John pointed out. If it's a different issue, please create a new ticket with more details so we can look into it.

I'm going to close this as the original issue has been solved.

Weird thing is, if I run the app with docker run, it's fine. If I run it with docker-compose up, it's not. It's almost as if docker-compose were picking the wrong image to spin the container from - an intermediary image, maybe?

that also means checking for quotes that aren't parsed out

In the Dockerfile or in docker-compose.yml? There's nothing quoted there.

I also tested on a simpler app, by copying first the Gemfile and then the rest of the app. On Windows, running with docker-compose, the image fails to run because Rails can't find the Gemfile. Running with docker run it runs fine. On OSX, it runs fine either way.

I think you need to post something that we could look at to help diagnose
the problem. Describing it isn't enough. (maybe anonymize some stuff for
security's sake).

On Wed, May 25, 2016 at 9:16 AM Pedro Vanzella [email protected]
wrote:

that also means checking for quotes that aren't parsed out

In the Dockerfile or in docker-compose.yml? There's nothing quoted there.

I also tested on a simpler app, by copying first the Gemfile and then the
rest of the app. On Windows, running with docker-compose, the image fails
to run because Rails can't find the Gemfile. Running with docker run it
runs fine. On OSX, it runs fine either way.

โ€”
You are receiving this because you modified the open/close state.
Reply to this email directly or view it on GitHub
https://github.com/docker/compose/issues/3472#issuecomment-221589627

I've got the same issue with docker-compose 1.7.1. The server has two websites that are supposed to be behind the dynamic nginx-proxy. They are in /var/www/production-site/ and /var/www/development-site/. Both are the same git repo just checked out at different commits. Each has docker/docker-compose.yml and docker/.env. The name of a container in docker-compose is dynamic:

version: "2"
services:
  web:
    build:
      context: .
      dockerfile: Dockerfile
    container_name: app-${MODE}
...

i.e app-dev and app-prod; the suffix being taken from .env.

I have two ssh terminals open in each folder and I'm trying to docker-compose up and down these two websites independently. docker-compose config gives different results in different folders as expected, $pwd assures me that I'm in the right folders.

Surprisingly when I docker-compose up -d --build I'm getting Recreating app-dev or Recreating app-prod randomly regardless of what folder I'm in. Can this be to do with some caching?

Tried supplementing my docker-compose.yml with image name:

version: "2"
services:
  web:
    image: my-project/my-app-${MODE}
    build:
      context: .
      dockerfile: Dockerfile
    container_name: app-${MODE}
...

Result of docker images after running docker-compose build --no cache in both folders (/var/www/production-site/docker /var/www/development-site/docker); the only difference is $MODE in .env:

REPOSITORY                              TAG                 IMAGE ID            ...
my-project/my-app-dev                   latest              920eae94d96a        ...
my-project/my-app-prod                  latest              bd6d87daa129        ...

However running docker-compose up in the two dirs does not let me see both containers in docker ps: it is either app-dev or app-prod, starting one kills another. What else can it be if not over-caching by docker-compose up?

Similar complaints here: https://github.com/docker/compose/issues/1883

@kachkaev This isn't a bug, and has nothing to do with caching.

Both your directories are named the same thing (docker), so when you run docker-compose up in either directory you're working on the same project. Each time you're updating the configuration of the same service (web) to change both the image name and container name.

It looks like what you actually want is two separate apps, each with its own image and container. In that case, I suggest doing the following:

  1. Either rename each directory so they're different or use the --project-name option or COMPOSE_PROJECT_NAME environment variable to set a different project name for each app. For example, myappdev and myappprod.
  2. Remove the container_name option from both Compose files - it's unnecessary.

If you run docker-compose up in both directories, you should now have two containers: myappdev_web and myappprod_web.

Thanks for your reply @aanand!

Both your directories are named the same thing (docker), so when you run docker-compose up in either directory you're working on the same project.

This is quite strange. I was expecting docker to figure out the name of the project using the full path, which is different (/var/www/production-site/docker && /var/www/development-site/docker) - just looking at the name of one parent dir seems a bit naive. What is the reason for not considering the full path here?

I'll try --project-name tonight when get to the laptop, thanks for the hint! Is there a way to define project name inside docker-compose.yml as a derivative of a variable in .env? E.g. myapp${MODE}. The main point of using docker-compose for me is to reduce the length of commands to type (I could just go for docker-run otherwise in this case).

I was expecting docker to figure out the name of the project using the full path, which is different (/var/www/production-site/docker && /var/www/development-site/docker) - just looking at the name of one parent dir seems a bit naive. What is the reason for not considering the full path here?

Mostly historical. We used to depend entirely on container names for keeping track of what containers belonged to what project, and so we had to squash a reasonably unique identifier into each container name - hence, a munged directory basename. It's a very difficult thing to change while preserving backwards compatibility.

Is there a way to define project name inside docker-compose.yml as a derivative of a variable in .env?

No, though it's a very old discussion - see https://github.com/docker/compose/issues/745 - but putting a COMPOSE_PROJECT_NAME=myappdev line in .env will work.

@aanand awesome, it worked! Summary of changes:

  1. New line in .env:

```
# /var/www/development-site/docker
COMPOSE_PROJECT_NAME=development-site

# /var/www/production-site/docker
COMPOSE_PROJECT_NAME=production-site
```

  1. Removal of image and container_name from docker-compose.yml

Images and containers no longer conflict and I see developmentsite_web_1 and productionsite_web_1 in docker ps. Awesome!

I am having the same problem, my image is a simple golang image. I can run docker-compose build without error, but the docker run works well while docker-compose up fails all the time.

I have read all your comments above, it seems not useful to me though.

Error message

โžœ  gray-test git:(master) โœ— docker-compose up
Starting graytest_server_1
Attaching to graytest_server_1
server_1  | sh: 1: /go/bin/app: not found

It tell me that /go/bin/app is not found, however the file exists if I run docker run -it [IMAGE NAME] /bin/bash and check the directory.

docker-compose.yml

server:
  build: './src'
  ports:
    - 8988:8988
  volumes:
    - .:/go

By the way, the volumes configuration is not making any sense here either. I think this roots for the same cause of failure of docker-compose up.

Dockerfile

FROM golang:1.8

WORKDIR /go/src/
COPY . .

RUN go-wrapper download graypublish/*  # "go get -d -v ./..."
RUN go-wrapper install  graypublish/*  # "go install -v ./..."

RUN ls /go/bin

CMD ["sh", "-c", "/go/bin/app"] # ["app"]

EXPOSE 8988

Directory structure

โ”œโ”€โ”€ README.md
โ”œโ”€โ”€ docker-compose.yml
โ””โ”€โ”€ src
    โ”œโ”€โ”€ Dockerfile
    โ””โ”€โ”€ graypublish
        โ”œโ”€โ”€ api
        โ”‚ย ย  โ””โ”€โ”€ graypublish.go
        โ”œโ”€โ”€ app
        โ”‚ย ย  โ””โ”€โ”€ main.go
        โ””โ”€โ”€ db
            โ”œโ”€โ”€ db.go
            โ””โ”€โ”€ tracks.go

Can you please give some suggestion?

I am having the exact same issue. But I think it is the 'docker' version. I am using Docker toolbox under OSX. When the VM is a boot2docker.vm image, it works fine. When the VM is a coreos, then the exact same build fails. The files are missing from the /usr/src folder, expect for the node_modules directory.

Here is the Dockerfile:

FROM node:10.5.0

RUN npm install --global yarn
RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app
ADD package.json /usr/src/app
RUN yarn install
ADD . /usr/src/app

If I use 'docker run image -it /bin/bash' then all of the files are there, if I use 'docker-compose run server /bin/bash' then the only directory under /usr/src/app is node_modules.

Again, this is only an issue when the VM is coreos which is using

docker -v

Docker version 18.03.1-ce, build 9ee9f40

core@my_vm01 ~ $ docker version
Client:
Version: 18.03.1-ce
API version: 1.37
Go version: go1.9.4
Git commit: 9ee9f40
Built: Thu Apr 26 04:27:49 2018
OS/Arch: linux/amd64
Experimental: false
Orchestrator: swarm

Server:
Engine:
Version: 18.03.1-ce
API version: 1.37 (minimum version 1.12)
Go version: go1.9.4
Git commit: 9ee9f40
Built: Thu Apr 26 04:27:49 2018
OS/Arch: linux/amd64
Experimental: false

I experienced a similar issue, and found this discussion. In my case, I was using a busybox image to persist some data. The issue was resolved by force removing this image and docker-compose build'ing anew.

I had this issue. For anybody getting a similar thing, I realised I was accidentally binding the local folder (using docker-compose) as a volume into the docker container like this:

  volumes:
    - ".:/home/node"

Which I must have done early-on, trying to get live-reload working or something. This was confusing the heck out of docker-compose and was showing a "container" version of the filesystem when putting RUN ls -al statements in Dockerfile and a "host" version of the filesystem when using CMD ls -al resulting in completely different outputs. Check to make sure you aren't doing something similar.

Leaving a comment here in case it helps someone else. docker-compose stop followed by docker-compose build and docker-compose start isn't good enough, you have to docker-compose down, docker-compose build and docker-compose up because down/up is when docker actually maps images to containers. If you don't use down/up docker-compose will use the old pre-existing containers.

This is probably obvious to seasoned docker users, but a casual like me didn't understand this nuance and it very well could be the trap many of the commenters on this thread are accidentally falling into.

Was this page helpful?
0 / 5 - 0 ratings