Compose: Feature Request: Init Containers for Docker Compose

[soapergem]

Is your feature request related to a problem? Please describe.

Kubernetes has a concept called Init Containers which currently has no direct parallel in Docker Compose. There are certain applications designed to take advantage of Init Containers which cannot be correctly tested locally using Docker Compose due to this limitation.

Describe the solution you'd like

The Docker Compose docs includes an article on Startup Order for Containers but this doesn't capture the essence of this problem. That article stresses, repeatedly, that there is no good way for Docker Compose to determine when a container is truly "up," so dependent applications should be designed with checks. However Init Containers are a totally different animal, as they are designed to start up and terminate before the next container runs. Termination is, in fact, very easy to check for and should be thought of as a different use case entirely.

In other words, there should be a solution in Docker Compose to declare that a container is dependent on a special class of "Init Containers" which will start up, run, and terminate before launching the rest. All you'd really have to do is add a flag in the configuration - something like initContainer: true

Describe alternatives you've considered

The only "alternatives" aren't really alternatives at all. You can override the default entrypoint / commands of the dependent containers to wait some fixed interval, but this is sloppy guesswork and does not represent good application design. You could write your application code in the dependent containers to check that other containers (with certain names, maybe?) at one time existed and are no longer running... but you see where this is going, no one does this, and indeed no one _should_ try to do this, because it's sloppy, unreliable, and bad design. On the other hand, it would be very easy to check, from a Docker Compose standpoint, that a container has terminated before launching the next.

Additional context

An example of an application that makes use of Kubernetes' Init Containers is vault-env, by Banzai Cloud. They have an in-depth article on it here. Essentially, they have an Init Container application which copies itself to a shared volume as a means of injecting secrets from Hashicorp Vault, and then the default entrypoint / command of the dependent container is updated to call the binary in the shared volume. Currently with Docker Compose this is not possible, because it doesn't wait for the first container to finish executing before trying to launch the second, so the application doesn't copy to the shared volume in time. Again, you can hack around this by changing the entrypoint to a long string involving bash -c sleep 3; /shared-volume/vault-env ... but this is an absolute mess.

Kubernetes has Init Containers. Docker Compose needs them too!

[cartermckinnon]

I agree; init containers are an elegant way to achieve much more flexible, reliable lifecycle management for a stack. The alternatives are distasteful.

I would rather write a docker-compose.yml than a helm chart, but (in the interest of time) using a local Kubernetes cluster may be a better solution. You can set one up with a checkbox in Docker for Mac, and you'd be able to use all your k8s files instead of having to maintain a docker-compose.yml.

[DaveBRH]

Coming from Kubernetes I was just looking for this feature. The company am I working for now does not have a Kubernetes implementation yet. And for now just a Docker installation is fine. But I am missing this feature.

[cartermckinnon]

Having had more experience with initContainers in production, there are some downsides. Specifically, if an initContainer is not successful, how often is it retried? In Kubernetes, the Pod enters CrashLoopBackoff (exponential backoff). This can cause your stack to be slow to stabilize when the initContainer finally succeeds. An Operator is one way to conduct a complex lifecycle without this sort of delay. I'm not sure what an "operator" would look like in a bare Docker setup -- a shell script? 😂

[blackandred]

This feature would be nice to see in compose. It is very helpful, so some kind of "Maintenance page" could be easily implemented in place of apps that are not up yet. Often applications do not wait for linked services such as database, cache or other linked application.

[AndreasGB]

I'd also love for this feature to be implemented as well.

My usecase is generating config files containing per-deployment secrets that two containers expect.

[dfcowell]

I worked around this by defining a shared volume that my init container writes to and then sleeps forever, which my dependent containers poll.

version: '3.7'
services:

  bootstrap:
    build: ./
    volumes:
      - ./:/app
      - bootstrap:/bootstrap
    command: ./bootstrap.sh

  dependent-1:
    build: ./
    depends_on:
      - bootstrap
    volumes:
      - ./:/app
      - bootstrap:/bootstrap
    command: /app/start.sh dependent-1

  dependent-2:
    build: ./
    depends_on:
      - bootstrap
    volumes:
      - ./:/app
      - bootstrap:/bootstrap
    command: /app/start.sh dependent-2

volumes:
  bootstrap:

bootstrap.sh:

#! /bin/ash

if test ! -f "/bootstrap/bootstrap"; then
  lerna bootstrap

  # This provides an init-contianer-like experience on docker-compose, which
  # doesn't natively support init containers.
  today=$(date +"%Y-%m-%d")

  echo "${today}" > /bootstrap/bootstrap
fi

sleep infinity

start.sh:

#! /bin/ash

set -e

until test -f "/bootstrap/bootstrap"; do
  >&2 echo "Waiting for bootstrap - sleeping"
  sleep 3
done

cd "./packages/$1"

yarn start

Won't work for all init container use-cases, but it solves a subset of them.

[fksimon]

We have a few scenarios where we want to move to init containers.

• certificate generation
• database provisioning
• common configuration

Right now we use the approach above which is to have a flag that tells the init container to not shutdown in compose environments, but run normal in kubernetes environment.

[blackandred]

I'm developing a docker (compose based) environment framework/template with features such as service discovery, automatic SSL, rolling updates (zero-downtime deployments) and I also plan to implement startup priority + init containers.

I'm preparing a 2.0 version rewritten in Python where I plan to implement init containers within month or two (possibly in 2.1) - subscribe for releases :)

https://github.com/riotkit-org/riotkit-harbor/tree/migrate_to_rkd