Compose: Should recreate use the same data volume when the image has changed

[dnephin]

Currently if we re-create a container that has a data volume, and the contents of that data volume were changed by the image, we re-use the old data-volume (because we don't want to lose data).

In my experience (and some related github issues) it seems like this is unexpected behaviour. If an image defines a data volume, and it changes the contents of that data volume, we should use the new one, not the old one.

[aanand]

When you say the image changes the contents of the data volume, do you mean that a new version of the image contains different files at that volume's path?

If so, I don't know how we would detect that.

[dnephin]

Right. The only idea I had was that if the image id changed, we could use the new volume, but that doesn't necessarily mean the data in the volume has changed.

[aanand]

Given that there's a pretty simple way to override it and get a fresh volume with the contents from the new image - just rm SERVICE and then up - I'm inclined to say we shouldn't try to be clever.

Furthermore, it'd be disastrous if, for example, we discarded* your database just because you'd updated your postgres image.

*we don't technically throw away the volume, but it's obscure and difficult enough to get to that we might as well have.

[dnephin]

I'm ok with that. I'm going to close this issue, we can always re-open it if we want to explore adding a flag or something.

[kriestof]

+1 In my case I have two containers - frontend and nginx:
I build my frontend app and in Dockerfile put it to volume /dist which is shared with nginx container. Below I show docker-compose config.

nginx:
  restart: always
  build: ./nginx/
  ports:
    - "80:80"
  volumes_from:
    - frontend
frontend:
  build: ./frontend/
  volumes:
    - /dist

At first sight it wasn't so obvious for me that building frontend for second time will update it's own volume but won't update volume in nginx. It would be nice to at least mention it in documentation. I think making flag which will imply if it should remove old volume before new build will be cool.

Current workaround is to rm services. I'm not sure if one can always do that. For instance in case when she would like to remove some volumes and leave some other untouched.

[dchersey]

I wrestled with this when exposing my Rails public/assets folder (containing precompiled static assets) to Nginx in another service linked to my rails app. In my case, I have 2 separate apps, each with their own server and static assets living behind one nginx server (with different locations specified). Each app is built separately via its own Dockerfile, published to my docker-repo, then pulled into my docker-compose stack as an image.

I wound up creating a separate shared, named volume for each app's public folder (app1_public, app2_public) and referencing them both from the nginx service and the respective app.

app1:
   image: repo/app1:latest
    ports:
      - "3000"
    depends_on:
      - 'postgres'
      - 'redis'
    volumes:
      - 'app1_public:/app/public'
    command: puma -C config/puma.rb

app2:
   image: repo/app2:latest
    ports:
      - "3000"
    depends_on:
      - 'postgres'
      - 'redis'
    volumes:
      - 'app2_public:/app/public'
    command: puma -C config/puma.rb

volumes:
  redis:
  postgres:
  app1_public:
  app2_public:

This all worked great on the first build/up. When I update the app1 or app2 images and docker-compose pull app1 or docker-compose pull app2 the apps update but the static assets contained in their app/public directories do not get updated into the shared volumes, and therefore are not available to nginx.

My solution was to add a docker volume rm app1_public and docker volume rm app2_public to the script that updates my services.

e.g:

docker-compose pull app1
docker-compose pull app2
docker volume rm app1_public
docker volume rm app2_public
docker-compose up -d

Not too bad, but it forces me to have an external script vs just using the magic of compose.
What would be nice is a volumes driver or other option that refreshes updates inside the app1 container to the shared volume at either build, pull or launch time. We obviously don't want this for all containers (leave my Postgres volume alone please!)

Thanks for listening, and I hope this solution helps others (was a bit arduous to piece together as there is much documentation on what compose does and doesn't do with volumes but no suggestions outside this issue on how to achieve what I want). And this thread suggested removing the service, which did not remove the shared volumes; I finally google 'remove docker volume' and discovered the docker volume command set.