Compose: Hooks to run scripts on host before starting any containers

[Jamie-]

This is clearly a common problem lots of people have been facing (even since 2014, #468), there's pile of closed issues for similar functionality to be added before and they have been closed, I believe, entirely un-reasonably.

Please see #1341 for a very concise argument as to why this functionality is useful, and judging by the reactions to most of the comments, it is quite a popular feature the community would like added.

Now it's over 2 years since #1341 was closed I believe hook-like functionality should be reconsidered.

Is your feature request related to a problem? Please describe.

There are many examples in #1341 already but I'll add my most recent use case for this.

I have a number of containers that are spun up, using compose, for development which require a shared data directory. I also need to access that directory on my host. Inside each of my containers a Python program is started as a specific user (as to mimic production as accurately as possible). Currently I mount this volume on each of my containers in docker-compose like so:

volumes:
  - "/tmp/data-var:/var/data"

However /tmp/data-var doesn't exist on my host (this is a shared development project), so it's created by docker for me, as root. Therefore my Python programs, running as non-root, cannot write to it.

Before docker-compose up starts any containers, I'd like to call something like mkdir /tmp/data-var && chown +w /tmp/data-var. Then on docker-compose down after all containers are destroyed I'd like to remove the temp data directory rm -rf /tmp/data-var.

I understand this _could_ be accomplished in other ways, please see the alternatives section below as to why these suck.

Describe the solution you'd like

I'd like to have two bash scripts, say pre-up.sh and post-down.sh and add them to be called via docker-compose with something like the following in my docker-compose.yml

version: "3"
pre-up: "./pre-up.sh"
post-down: "./post-down.sh"
services:
  service1:
    build: .
    volumes:
      - "/tmp/data-var:/var/data"

Other possible hooks people might find useful:

• post-up: Called after containers have all started
• post-stop: Called after containers have been stopped (either with ^C or docker-compose stop)
• pre-down: Called before destroying containers and networks (with docker-compose down)
• etc.

When calling these, compose should block at the specified point until the script has returned with an exit code of 0, and itself stop with a non-zero exit code if the script exits with a non-zero code.

Describe alternatives you've considered

There are alternatives for _my example_ use case, and equally good reasons they're a bad fit.

1. Calling a script on container start

I could have an ENTRYPOINT ["start.sh"] which sets the correct permissions on the directory, then my Python run command be specified via CMD ["python", ...] and have start.sh finally call exec "$@". However this is a waste as the first container to get started, and every container restart after would do the same thing, it only needs to be done once before any containers start.

Equally, it wouldn't solve my post-down: "./post-down.sh" use case.

2. Wrapping it up in a different script

I could write a wrapper script that calls docker-compose up, as that's been suggested many times in other issues. Come on... we're all using compose because it's concise, neat, tidy and simple to use. Everything is specified in one place which makes it easy for beginners to understand and read what's going on. Compose itself is essentially a standard when using docker with multiple containers.

3. Compose events

Though my understanding of events is lacking, due to how complex it is for what I really want to do. This is a poor way to achieve the goal I described, just like many other issues that were raised, then all pointed to #1510 (compose events). Events are reactive, this needs to be proactive, but more importantly, events do not block, and for many people, like me, blocking is essential.

[phedders]

Good luck... with kubernetes. Docker inc doesnt care.

[ndeloof]

Volume permission is very common issue with docker, and as long as you use bind mounts you even tell the engine "I'm in charge for this one, just expose inside container" and get into obvious permission issues. Using named volumes, which are created with owner set to the first container to use them, would help solve this issue.

What you describe as a proposed solution is pretty comparable to Kubernetes init containers, this is something we should consider for a future version of docker-compose. Main constraint is that compoes file format is not only used by docker-compose so such a move will require some coordination with docker stack command
and compose-on-kubernetes.
cc @chris-crone

[jdiegosierra]

Would be nice to run different scripts before start any services, in my case I need create some folders and depends of the service I want to start.

[TomaszGasior]

@jdiegosierra Can't you just do this in entrypoint script?

[jdiegosierra]

@TomaszGasior I mean I have to create folders into my host. And they dependeds of the service. As far I understand how entrypoint.sh works, it is for run some commands into de container right?

[TomaszGasior]

@jdiegosierra If you need to create directories inside directories shared between host and container, you can create them inside container's entrypoint by wrapping original entrypoint into your own.

[jdiegosierra]

@TomaszGasior Yeah I know :D But is not my case...

My case is I'm sharing the folder of a project with the container except the dist folder. I want the container has its own dist folder and my host project its own dist folder in order to develop using docker or without it. So in my docker-compose I have this:

volumes:
  - ../../frontend:/opt/app
  - ../../frontend/dist
  - /opt/app/dist <-here is the problem

Also in my docker image I have this:

RUN groupadd appuser
RUN useradd -r -u 1001 -g appuser appuser
... build stuff
RUN chown appuser:appuser /opt/app -R

Into my container the dist folder has the appuser permissions and its builded files so its okey for me. The problem is outside the container. docker.-compose has created a folder called dist with root permissions so if i want to build my project in my host i cant because permissions. However if I create dist folder in my host with appuser permissions before start docker-compose all works as I want and dist folder in my host is empty with appuser permissions so I can build my project also in my host and it doesnt conflict with the dist folder into the container.

[TomaszGasior]

docker-compose has created a folder called dist with root permissions so if i want to build my project in my host i cant because permissions

As I understand, what you need is to create directory from inside of container's entrypoint but with permissions like it would be created from your host. If there is any directory inside your container with permissions from your host, you may want to use stat and chown.

Let's see my example. I have PHP application. composer, PHP package manager, creates vendor directory with all app dependencies. I want to run it from container entrypoint but with permissions like it would be ran from my host. Check it out: https://github.com/TomaszGasior/RadioLista-v3/blob/bf5692d3d767afcfa7c1ccf46109c4f653c85b1c/container/php-fpm/entrypoint.sh#L8

Basically what I am doing here is I ran composer command with permissions (user and group) the same as parent folder has. You may take some inspiration from this. For example, you may create folder with mkdir, then change its permissions by chown to permissions of different directory of your project which is created on your host — you can get them using stat.

[jdiegosierra]

Is not exactly what I need. I need to create a empty folder in my host called dist with appuser permissions before start the service in order to docker-compose doesnt create a dist folder with root permissions.

btw, I appreciate your help :)

[TomaszGasior]

Is not exactly what I need. I need to create a empty folder in my host called dist with appuser permissions before start the service in order to docker-compose doesnt create a dist folder with root permissions.

Or you may let docker create that folder with root permissions and then inside your entrypoint just fix that wrong permissions using method which I described. :) It's possible if you have any other directory created in host by your host user, available inside container. Just stat the second one and chown first one. Your host user don't have to exist inside your container — chown will accept non-existing user/group ID returned by stat.

Something like: chown $(stat -c '%u:%g' /from-host) /in-container.
/from-host — directory created by your user in host OS, /in-container — directory created by dockerd with wrong permission.