Compose: Support docker exec command

[k4nar]

I've just learned about the new docker exec command. Wouldn't it be great to add it to fig ?

I think it really makes sense with fig, as you could run fig up and then exec some commands (like a shell) to debug your running containers if something happen.
Of course this is already possible by running docker exec -it your_container cmd, but I think that it wouldn't hurt to have fig exec service cmd.

I'll try to give it a shot and submit a PR.

[bfirsh]

:+1:

Thanks @k4nar! Contribution docs are here: https://github.com/docker/fig/blob/master/CONTRIBUTING.md#tldr

[k4nar]

I think we'll have to wait for https://github.com/docker/docker-py/pull/363 to land.

[imsickofmaps]

Just an FYI as I clicked through this link, docker/docker-py#363 has landed

[k4nar]

I'm working on it. I have something working for simple commands, like fig exec web echo "Hello World!", but I don't know yet how to make it works with applications using TTYs.
I'm looking at dockerpty but I think it will require some modifications, as for now it only handles the client.start method.
If anyone with more experience than me on the subject wants to help it's welcome :) . You can reach me here or on IRC.

[pikeas]

+1!

[Stono]

+1

[andradei]

:+1:

[hadim]

+1

[hadim]

It would also be nice to be able to define exec command in fig.yml.

[baptistedonaux]

+1 Yes I would it !

[Grokzen]

+1

[jhftrifork]

+1

[dnephin]

docker exec is already "supported" by the docker cli

docker exec <container name>

This really wouldn't change with docker-compose, you'd just use the service name instead of the container name. It doesn't seem to me like exec is really a feature of composition. In the same way that docker-compose doesn't duplicate docker inspect or docker import. Those are very much operations for individual containers, so they don't really need to exist in compose.

[Vrakfall]

+1

Well I think this would be a usage improvement. It implies shorter commands (= more user friendly) and less pain when it comes to using long container names. As some fig commands like kill or rm are already doing the same job than docker while shortening the command lines, why not implement it for commands like exec or inspect (e.g.).

It would, personally, ease my repetitive tasks. (Because you know, at work, you can't always have the choice of the paths or directories names.)

[dnephin]

In the case of kill or rm at least they are "group" operations, in that you can run them on every container in the composition. This is not the case for exec.

I suspect, in the long term, that eventually the docker cli will start to understand docker-compose.yml and be able to support service names (this was the case in some of the earlier proposals for compose). Although that may still be a long way off.

It seems like there are already a lot of ways of dealing with long names that it doesn't really need to be part of compose. You can always use an environment variable (export PRJT="verylongdockercomposeprojectname"; docker exec $PRJT_service_1) or a bash alias, or a bash script. You can even use tab completion with docker.

[Vrakfall]

You're right, I didn't see it like that.

Well, I'm still thinking about some magic that could do "fig exec". You said something about grouping. What if it would actually be able to group the exec command and be able to run a magic cat or tail -f in all the services (e.g. to add some logs to the console prompt). Another use case could be to run a automated script put in every container (e.g. a sanity check .sh or some daily checks) in one command. It could be neat, combined to a bash script that would e.g. call it everyday.

This is an idea I juste had. What do you think about it ?
(I personally don't need it in a close future.)

[aanand]

I'm in favour of supporting commands like exec and inspect - their docker equivalents aren't hellish, but it's a pain having to type out the full container name - and once we switch to using labels, container names will be a lot less predictable.

[thedug]

+1

I'm often:

1) running fig ps to get list of container names,
2) running docker exec -it container

It would be nice to have a command that worked like fig run but didn't start a new container

[k4nar]

@aanand @dnephin : Next week I'll have some time to work on this. Last time I checked I was stuck at using dockerpty with an exec command.

[NodeGuy]

I'm trying to use docker exec in a script and, as far as I can tell, I can't know the container name in advance because fig generates it automagically. That's why we need fig exec.

Ah, I just figured out that I can do this:

docker exec $(fig ps -q service) command

[dnephin]

You don't have to use the hex id, you can use the name <project>_<service>_1. You know the service name, and the project name can either be set (using $FIG_PROJECT_NAME or -p) or you can assume the default.

[NodeGuy]

The last time I checked manually, the name was <project>_<service>_4. How can I know the final digit?

[dnephin]

Ah, normally it will be _1, if you're seeing it use higher numbers it means you're either using fig scale, or there are old containers sticking around using the lower numbers. I forget exactly the condition that causes this, but if you docker rm__xall the containers with earlier numbers, you should see it using_1` consistently.

[mikehaertl]

+1

Actually I thought, that's what fig run would already do and was surprised to find out, that it will always run the command in a new container instead.

[lcamacho]

+1

[schmunk42]

+1

[chiedo]

I agree with @mikehaertl. What would be great would be to be able to run a command like the following:

docker-compose exec [name of service] command

As apposed to having to do the following :

docker exec [container name] command

[torte]

+1

[chiedo]

I went ahead and created a terminal command to the same effect. Just add the below to your ~/.bash_profile or ~/.bashrc

# Allows you to use docker-compose-exec {service} {command} to run a command on the service
docker_compose_exec() {
  docker exec -it $(docker-compose ps | grep $1 | awk '{print $1}') ${@:2}
}
alias docker-compose-exec=docker_compose_exec

With the above, you can run a command such as the following

docker-compose-exec web bash

[ioggstream]

Simple patch to support "fig execute" https://github.com/ioggstream/fig/tree/support-docker-exec-command-553

I appreciate feedback before PR

[k4nar]

Does it work with commands needing a TTY ? I had a similar patch, but I couldn't handle TTY creation.

[ioggstream]

@k4nar I added support for all docker-exec parameters: check if it works for you.

In case it doesn't, can you please:

• show me your patch?
• show a testcase to check your TTY issue?

It is still need timeout handling and a way to show command output somewhere

[osteenbergen]

Hi, as I'm building an other feature (trigger after scaling) I needed the docker exec command so I made a patch #1180

It allows you to run a command against a service or all services. What isn't possible is interactive as I consider that too complex and cumbersome, but --detach is allowed.

[Vrakfall]

Thanks a lot. That'll be very useful.

[chiedo]

Thanks, The interactive portion is key for my use case though :(

[osteenbergen]

Interactive is quite complex as a service can have multiple containers. You can't send the input to all containers, which leaves doing things one at a time.

Lets say we have 3 containers belonging to the "web" service and we execute:

$ docker-compose execute -i web /bin/bash

Do we get a terminal to the first and when we close it it opens a terminal to the second and so on?

Or do you want to have a command like:

$ docker-compose execute -i --container=2 web /bin/bash

This is just a shortcut for the following and adding that to compose doesn't make sense to me. Using numbers sounds like guess work and executing a command on a single node could affect the service that it provides.

$ docker exec `docker-compose ps -q | sed '2!d'` /bin/bash

So can you give some insight into the exact use case you are trying to solve? Does it have to be interactive or can you use tooling like expect scripting to solve it?

[chiedo]

Hey @osteenbergen,
Thanks for your great points! My request came from my newness to Docker (a little under a month). I was not aware that multiple containers for the same service could be up at once.

That explains the intermittent bugs I was receiving while using the following

# Allows you to use docker-compose-exec {service} {command} to run a command on the service
docker_compose_exec() {
  docker exec -it $(docker-compose ps | grep $1 | awk '{print $1}') ${@:2}
}
alias docker-compose-exec=docker_compose_exec

# To use run something like: docker-compose-exec web bash

For now, I will just continue to use docker exec. I have rigged up the following set up so that once my docker web container starts, it waits to be entered interactively using the following in the container and outputs the command needed to connect to it interactively with bash (purely lazy I know).
https://gist.github.com/chiedojohn/e3ff751fed1b641c2ee9/d51ef7f1286d9e3a5adffd74cec5b4e6b1d6e731

I know the idea is to not need to enter your container with bash and limit it to one process usually but this solution is for development environments only makes development much faster as I can run any commands on my container as if it were an ssh connection (for the most part) for running test installations of packages before adding them to my Dockerfile, etc.

Thanks for your response and explanation

[sandric]

Well, we can use docker exec instead of compose's one, and it is actually just wrapping around, but you actually can not automate it simply - because first you need to get full container's name, which is created via composer's naming policy, so there's actually is point to implement it, imho. Is it alive, smbdy implementing it?

[k4nar]

There is some work for non-interactive mode (like compose exec <service> ls) but interactive mode (compose exec bash) is pretty hard.
Also there is the question of the behavior of exec when the service has several instances.

[sandric]

@k4nar hm, must say I don't really understand what is interactive mode then, and how you can execute command without specifying service, which in turn run command on particular container. No, I meant that exactly compose exec <service> ls feature would be very handy, and it seems not much problems implementing it to me.

[harobed]

+1

[djdefi]

This would be cool!

[langri-sha]

@osteenbergen, would it make sense to be able to implicitly reference which instance we want to attach to?

These could be equivalent: sudo docker-compose exec foo cmd, sudo docker-compose exec foo_1 cmd, sudo docker-compose exec myproj_foo_1 cmd.

[unbalancedparentheses]

+1

[lcacciagioni]

+1

[MiLk]

:+1:

[Skyross]

Want to see this feature because often we need do some stuff with linked database engines (like creating/migration). Someone find workaround solution already?

[balor]

+1

[TomasTomecek]

Already managed to do a basic implementation ^

[painhardcore]

+1 waiting 4 this feature.

[gnoymmij]

+1 would be a useful and handy feature.

[andrefreitas]

+1, very useful

[timsuchanek]

+1, because doing docker-compose run for each single command takes a lot of time

[langri-sha]

@timsuchanek I installed Kitematic recently; I haven't used any GUI to interface with Docker before, but I welcomed this addition to my toolbox for it's ability to select a running container and start a shell that runs docker exec to the selected container. I find it a decent intermediary solution until we get proper support.

[knight-of-design]

+1

[krasi-georgiev]

+1

[vlastv]

+1

[skalfyfan]

+1 to this as well? Is it still being considered or looked at?

[TomasTomecek]

@skalfyfan Yes, I just got patch to docker-py merged recently: https://github.com/docker/docker-py/pull/858 now I need to finish dockerpty https://github.com/d11wtq/dockerpty/pull/48 and then finally finish compose https://github.com/docker/compose/pull/2023 (and sen https://github.com/TomasTomecek/sen/issues/9) eta: January

[thasmo]

+1

[timgriffiths]

+1

[kunalvjti]

+1

[theotheo]

+1

[pawelkaczor]

+1

[Neschur]

+1

[Ehekatl]

+1

[stupied4ever]

+1

[michabbb]

just a note about this +1 thing:

since github doesn´t care about real needs for features like +1, it´s sad, but there are alternatives

• http://feathub.com/
• http://poll.gitrun.com/
• https://www.zenhub.io
• https://zapier.com/zapbook/github/uservoice/

[qcho]

+1 Waiting for exec to be part of compose! :D

[docteurklein]

Just a quick note about a zsh function I declared in my ~/.zshenv:

dce() { docker exec -it "${COMPOSE_PROJECT_NAME}_${1}_1" ${@:2} }

It's usable like that:

dce <service-name> bash

You'll need to export COMPOSE_PROJECT_NAME for that.

[ptahchiev]

+1 ..I need to create solr indexes after the server is up:

docker exec -it nemesis-solr bin/solr create_core -c products

[michaelarnauts]

One-liner to the rescue:

docker exec -it $(docker-compose ps | grep <service> | cut -d" " -f 1) <command>

[krasi-georgiev]

Yeah right , like you will ever remember that when you need it :)

[michaelarnauts]

No need to remember it. I have all my test/deployment statements in one handy Makefile.

[roytruelove]

@aanand can you comment on the reason for closing this one? I see workarounds in the thread but no 'won't fix'es

[docteurklein]

@roytruelove it's implemented since Feb 29, as stated by by aanand in PR 2023.

[thesobercoder]

+1 Much needed.

[thaJeztah]

@thesobercoder did you actually read the comments directly before yours? This was implemented over two years ago; https://github.com/docker/compose/issues/2023