Compose: Execute a command after run v2

[reduardo7]

Reopen #1809

Hi,

It will be very helpful to have something like "onrun" in the YAML to be able to run commands after the run. Similar to docker/docker#8860

mongodb:
    image: mongo:3.0.2
    hostname: myhostname
    domainname: domain.lan
    volumes:
        - /data/mongodb:/data
    ports:
        - "27017:27017" 
    onrun:
        - mongodump --host db2dump.domain.lan --port 27017 --out /data/mongodb/dumps/latest
        - mongorestore -d database /data/mongodb/dumps/latest/database

After the _mongodb_ start, It will _dump_ db2dump.domain.lan and restore it.

When I will stop and then start the container, onrun part will no be executed to preserve idempotency.

[shin-]

lol dude, that's not how this works.

@dnephin and I have clearly explained in #1809 why we won't add this to Compose. Opening a new issue won't change that.

[dopry]

@shin-, @dnephin we do appreciate all the work you do. #1809 is rather contentious and hard to follow. So opening a new issue isn't necessarily a bad idea if it keeps the focus in the right place...

Let's set aside the technical reasons that have been outlined, which I don't buy for a moment. Let's think about the Docker ecosystem as a whole. Docker is facing stiff competition from Kubernetes which makes is much easier to swap between underlying container implementations. Kubernetes has historically been proven in production but didn't support development well. Docker/docker-compose has been a great tool for development, but, frankly, Docker doesn't quite have the same track record as Kubernetes at scale. Specifically, container and service lifecycle management are required in production, Kubernetes has facilities for that, https://kubernetes.io/docs/tasks/configure-pod-container/attach-handler-lifecycle-event/, https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/.

Now there is minikube, which make setting up a Kubernetes environment for local development much simpler.

While Docker has enjoyed a strong following with developers thanks to fig/docker-compose up until now, in my experience sysadmins have struggled with the transition to production. Now with minikube, as a manager and architect I see a path from development to production that uses the same environment and configuration.

Given your reluctance (@dnephin, @shin-), I'm no longer confident that docker will be able to deliver the flow from development to production using containers that I have long sought for my team. So as a manager I have to strongly consider minikube and kubernetes.

Frankly, this resistance isn't good for the docker ecosystem as a whole. I hope the management at docker, guys like @shykes and @iaincgray, take notice that this is an important issue and that the way you treat the community and their requests isn't beneficial to the platform, customers, or the ecosystem.

[reduardo7]

@shin- , @dnephin and all team: we do appreciate all the work you do! You can delete this ticket.
In the future, perhaps another solution to the proposed requirement may be found, possibly in another version, not necessarily for the current implementation.

Thank you very much to all who work hard in this project!

[omeid]

@shin- That is a very inappropriate and childish way of responding to an issue that many many users disagree with.

What has become with Docker? this is very unprofessional.

[shin-]

@omeid Unprofessional? You mean like this comment of yours?

Listen, I understand that a negative answer to a feature request can be frustrating, but we've already said our piece and creating more issues and posting more comments at this point is unlikely to change this. We've additionally suggested different ways to achieve the desired result. Maybe they don't work quite like you want, in which case, feel free to fork the project into your own tool that does what you feel it should, but demanding that we do what you want is absolutely preposterous, and no amount of badgering is going to make it happen.

@reduardo7 @dopry Thank you for the kind words. I'm sorry we don't see eye to eye on this.

[SvenDowideit]

I too want something that will onrun in my compose file

__BUT__, neither containers, nor compose have a way to know what onrun means. This is why the operator pattern exists, and why I made the examples in https://github.com/docker/compose/issues/1809#issuecomment-362126930

it __is__ possible to do this today - in essence, you add an onrun service that waits until whatever other services are actually ready to interact with (in gitlab's case, that takes quite a bit of time), and then do whatever you need to do to co-ordinate things.

If there _is_ something that doesn't work with that, please tell us, and we'll see if we can figure out something!

[dopry]

@shin-, it is unfortunate. The experience you craft for developers and systems administrators directly affects adoption of Docker. Docker isn't faring so well. I figured you'd want to contribute to the success of the project you work on, rather than it's failure.

If I'm not mistaken you're a paid representative of Docker Inc. In that case, the onus of professionalism on you. For the rest of us code monkies, I think civility is a hard enough bar to achieve some days.

[omeid]

@shin- That comment is true, @dnephin always brings up dopey as a solution, while, all the users want is to make docker-compose a better tool.

lol dude, that's not how this works.

But that comment of yours, is rude. You don't treat people like that.

[yosefrow]

I too want something that will onrun in my compose file

BUT, neither containers, nor compose have a way to know what onrun means.

As I see it onrun per service, means when the first container process starts. In a larger number of cases, the container is only running one process anyway, as this is the recommended way of running containers.

The issue of cross-platform support was solved earlier, as the command can be completely OS agnostic through docker exec, in the same way that RUN does not have to mean a linux command in Dockerfile.
https://docs.microsoft.com/en-us/virtualization/windowscontainers/manage-docker/manage-windows-dockerfile

[omeid]

Listen, I understand that a negative answer to a feature request can be frustrating, but we've already said our piece and creating more issues and posting more comments at this point is unlikely to change this. We've additionally suggested different ways to achieve the desired result. Maybe they don't work quite like you want, in which case, feel free to fork the project into your own tool that does what you feel it should, but demanding that we do what you want is absolutely preposterous, and no amount of badgering is going to make it happen.

@shin- , I highly suggest you reconsider your tone, no one has asked you, or even docker explicitly to do the work.

While it is true that the project is maintained by Docker, it is largely based on community contributions, from the days before it was a Docker project up until now, nothing around this issue is exceptional to that. So once a consensus has been reached, I am sure many people would be interested to contribute the code.

It seems like you fail to understand that contributions to projects often start with an issue that ends up with deciding on a solution before someone contributes the code and if all issues related to this problem is promptly closed, it would never be resolved.

As for _go fork it_, that is the lowest level of reasoning, please don't go that low.

[shin-]

Folks,

I want to apologize to you for my attitude yesterday - I lost my cool and wrote things in a way that was insensitive and hurtful, and that wasn’t right. I shouldn’t have let my emotions get the better of me and I shouldn’t have antagonized fellow community members. I hope you can forgive me and that we can keep working together on making Compose better moving forward.

While we appreciate the passionate feedback on this matter, our position at Docker on this specific issue remains the same. We are committed to craft a better experience for developers and understand the need for the feature. That said, we do not believe Compose is the right place for this feature to exist. We’re going to work with folks internally to figure out what the best way is to deliver this, and I’ll commit to keeping you and the rest of the community in the loop through GitHub and/or other appropriate channels.

Thank you for your understanding.

[dopry]

@shin-, thanks. Apology accepted. I'm glad to hear the issue is being considered.

If the internal team is open to suggestions, it would be really nice to see lifecycle hooks implemented swarm/stacks. Then it could exist in docker-engine in a way that is cluster aware and independent of the CLI interfaces.

That would make moving toward docker stack as a replacement for docker-compose in the developer flow a real possibility. If toolkit were modified to install the VM in swarm mode, Docker could begin recommending developers use docker stack. It would create a more consistent experience from development to production. It would eliminate redundant tooling in the chain. It would reduce the volume of code that needed maintenance and support.

It would be a strong justification to limit docker-compose support to just maintenance and bug fixes, so resources can be allocated to docker engine and docker stack to hasten the transition.

[robertlagrant]

@shin- the PR folks who draft such responses should be thankful that you keep them in a job :D

On-topic: can anyone propose how docker-compose could actually detect that a container is running its main process to the extent that commands can be run against it? E.g. should it wait for the CMD to come back, a bit like running a command in a shell prompt? Will that be useful enough for a lot of cases?

[dopry]

@robertlagrant, I don't think CMD is appropriate as many commands don't return, they just run until the container is killed or they fail in some other manner.

Instead, I would propose a successful healthcheck as the trigger that a service is ready. A healthcheck script could be written to await start-up or to make a connection to the service before returning success. Timeout and retries are handled by healthcheck already as a bonus.

Compose will need to listen to the docker events, until it can resolve the onrun declarations. Which may be at the heart of what @shin- and @dnephin don't want to implement in compose. It would obviously be better if docker itself handled state and state changes.

[ruiztulio]

Hi guys,

I know is not the proper solution to the initial issue #1809 but I actually have to use Kubernetes for this (I had a little faith in docker-compose devs and hope that given the popularity of this feature they will reconsider it).

With the init containers I managed to initialize our app as mentioned in some comments here, that is initialize the database (users and stuff) create an admin user for djago and other stuff

I'd rather to have such feature integrated with compose since is simpler and we could use it from dev to production w/o any additiontal stuff, but it is what it is.

Note: I'm still into this feature

[omeid]

@robertlagrant it is not a problem that docker-compose should solve, it should allow running a list of commands in serial and kill the container if any fails, then you can simply do things like this:

services:
  api:
    image: node:alpine-x.x.x
    init:
      - ./inits/wait_for_api.sh
      - ./inits/run_seed.sh

Simple. :)