Compose: docker-compose exec is laggy

[luispabon]

There seems to be an substantial initial delay running commands through docker-compose exec that isn't there when going straight through docker:

[development@localhost ~/P/app] (dev) time docker-compose exec php-fpm echo 

1.12user 0.07system 0:01.27elapsed 93%CPU (0avgtext+0avgdata 39440maxresident)k
0inputs+35568outputs (0major+38964minor)pagefaults 0swaps

vs

[development@localhost ~/P/app] (dev) time docker exec app-php-fpm echo

0.00user 0.00system 0:00.07elapsed 22%CPU (0avgtext+0avgdata 21400maxresident)k
0inputs+0outputs (0major+2537minor)pagefaults 0swaps

This seems to be related with startup time to docker-compose itself:

[development@localhost ~/P/app] (dev) time docker-compose --version
docker-compose version 1.12.0, build b31ff33
1.10user 0.05system 0:01.18elapsed 98%CPU (0avgtext+0avgdata 39356maxresident)k
0inputs+35568outputs (0major+35516minor)pagefaults 0swaps

Running 10 containers at the time of running commands above, 5 of which on the project I'm running these from.

Host: Ubuntu 17.04
Docker: 17.03.1-ce, build c6d412e
Docker-compose: docker-compose version 1.12.0, build b31ff33
CPU: Intel(R) Core(TM) i5-4200M CPU @ 2.50GHz, mostly idle
RAM: 16GB (~5GB free at the time of running commands above)

[shin-]

This might be due to the pip version check that we fixed in #4698

Can you try 1.13.0 RC1 and let me know if the issue persists?

[luispabon]

I'm afraid the issue is still there:

[development@localhost ~/app] (master) time docker-compose-Linux-x86_64 exec webserver echo 

1.14user 0.07system 0:01.28elapsed 94%CPU (0avgtext+0avgdata 39080maxresident)k
0inputs+35592outputs (0major+39011minor)pagefaults 0swaps
[development@localhost ~/app] (master) time docker-compose-Linux-x86_64 --version
docker-compose version 1.13.0-rc1, build 38af513
1.10user 0.08system 0:01.21elapsed 98%CPU (0avgtext+0avgdata 39184maxresident)k
0inputs+35592outputs (0major+35775minor)pagefaults 0swaps

[shin-]

Thank you for checking. Do you see any difference if you install using pip instead of using the binary? The binary is created with PyInstaller, which extracts required program data into your /tmp folder when you execute it - is there anything specific about the configuration of your temporary dir that could cause the observed slowdowns?

[seeruk]

Coming across this later on, I'm also having this issue. Using the binary or not doesn't make a difference for me, on either 1.12, or 1.13:

Installed via pacman:

$ time docker-compose version
docker-compose version 1.12.0, build unknown
docker-py version: 2.2.1
CPython version: 3.6.1
OpenSSL version: OpenSSL 1.1.0e  16 Feb 2017
0.76user 0.01system 0:00.79elapsed 98%CPU (0avgtext+0avgdata 35968maxresident)k
0inputs+0outputs (0major+30041minor)pagefaults 0swaps

Binary:

$ time /usr/local/bin/docker-compose version
docker-compose version 1.13.0, build 1719ceb
docker-py version: 2.2.1
CPython version: 2.7.13
OpenSSL version: OpenSSL 1.0.1t  3 May 2016
0.55user 0.00system 0:00.57elapsed 98%CPU (0avgtext+0avgdata 33880maxresident)k
0inputs+0outputs (0major+20270minor)pagefaults 0swaps

Using 1.10 is considerably faster. I hadn't upgraded the version on my laptop for a while, but installing a fresh system on my desktop has led to me noticing this. 1.10 is very quick on my desktop by comparison.

[shin-]

Are you using Ubuntu on both machines as well?

On Thu, May 18, 2017, 3:18 PM Elliot Wright [e-mail]
wrote:

Coming across this later on, I'm also having this issue. Using the binary
or not doesn't make a difference for me, on either 1.12, or 1.13:

$ time docker-compose version
docker-compose version 1.12.0, build unknown
docker-py version: 2.2.1
CPython version: 3.6.1
OpenSSL version: OpenSSL 1.1.0e 16 Feb 2017
0.76user 0.01system 0:00.79elapsed 98%CPU (0avgtext+0avgdata 35968maxresident)k
0inputs+0outputs (0major+30041minor)pagefaults 0swaps

$ time /usr/local/bin/docker-compose version
docker-compose version 1.13.0, build 1719ceb
docker-py version: 2.2.1
CPython version: 2.7.13
OpenSSL version: OpenSSL 1.0.1t 3 May 2016
0.55user 0.00system 0:00.57elapsed 98%CPU (0avgtext+0avgdata 33880maxresident)k
0inputs+0outputs (0major+20270minor)pagefaults 0swaps

Using 1.10 is considerably faster. I hadn't upgraded the version on my
laptop for a while, but installing a fresh system on my desktop has led to
me noticing this. 1.10 is very quick on my desktop by comparison.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/docker/compose/issues/4748#issuecomment-302557155,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABCVnBdpo9pscuz6lK0pHsqX_4ohtVvfks5r7MOvgaJpZM4NC30G
.

[seeruk]

I'm using Arch, same machine for both examples. It's a pretty fresh install, I'm not at that PC now, but if you want any information about the environment just let me know and I'll get it on here ASAP (e.g. tonight).

Edit: On Ubuntu on my laptop currently, it looks like this, with the binary:

$ time docker-compose version
docker-compose version 1.13.0, build 1719ceb
docker-py version: 2.2.1
CPython version: 2.7.13
OpenSSL version: OpenSSL 1.0.1t  3 May 2016
0.35user 0.02system 0:00.38elapsed 97%CPU (0avgtext+0avgdata 26212maxresident)k
0inputs+35592outputs (0major+17366minor)pagefaults 0swaps

Which is a fair bit faster than Arch above (despite the hardware in my PC being more powerful), but still around 3-4 times slower than 1.10 was.

And again, installed via pip yields similar results:

$ time ./docker-compose version
docker-compose version 1.13.0, build 1719ceb
docker-py version: 2.3.0
CPython version: 2.7.12
OpenSSL version: OpenSSL 1.0.2g  1 Mar 2016
0.32user 0.02system 0:00.35elapsed 98%CPU (0avgtext+0avgdata 55404maxresident)k
0inputs+0outputs (0major+21592minor)pagefaults 0swaps

This is:

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 16.04.2 LTS
Release:    16.04
Codename:   xenial

And:

$ docker version
Client:
 Version:      17.05.0-ce
 API version:  1.29
 Go version:   go1.7.5
 Git commit:   89658be
 Built:        Thu May  4 22:10:54 2017
 OS/Arch:      linux/amd64

Server:
 Version:      17.05.0-ce
 API version:  1.29 (minimum version 1.12)
 Go version:   go1.7.5
 Git commit:   89658be
 Built:        Thu May  4 22:10:54 2017
 OS/Arch:      linux/amd64
 Experimental: false

(I'll get this same info for Arch tonight).

[luispabon]

Apologies, just coming back to this. Installing from pip doesn't make any difference, numbers same as I reported.

I am on Ubuntu 17.04:

[development@localhost ~] docker version
Client:
 Version:      17.03.1-ce
 API version:  1.27
 Go version:   go1.7.5
 Git commit:   c6d412e
 Built:        Mon Mar 27 17:17:43 2017
 OS/Arch:      linux/amd64

Server:
 Version:      17.03.1-ce
 API version:  1.27 (minimum version 1.12)
 Go version:   go1.7.5
 Git commit:   c6d412e
 Built:        Mon Mar 27 17:17:43 2017
 OS/Arch:      linux/amd64
 Experimental: false

[development@localhost ~] lsb_release -a
LSB Version:    core-9.20160110ubuntu5-amd64:core-9.20160110ubuntu5-noarch:security-9.20160110ubuntu5-amd64:security-9.20160110ubuntu5-noarch
Distributor ID: Ubuntu
Description:    Ubuntu 17.04
Release:    17.04
Codename:   zesty

[MajPay]

Just noticed the same issue, running docker on ubuntu 16.04.3

Docker version 17.12.0-ce, build c97c6d6
docker-compose version 1.16.1, build 6d1ac21

Even docker-compose --help is 15 times slower than docker --help

So i assume docker-compose is just slow

i utilize a docker container to build templates that require php, so this delay is a real pain in the ass. I want to use docker-compose over docker because in docker-compose i can say

docker-compose exec servicename command

Where in docker i would have to use the container name or id - and both of them depend on the directory name i guess, so it would be a different command in different dev environments

docker exec appname_servicename_1 command

My build using docker: 5.1s (real), 1.7s (user), 0.4s (system)
My build using docker-compose: 13.7s (real), 8.5s (user), 1.4s (system)

It would be great to have similar performance in both variants

[alberto56]

For now I work around this by using:

CONTAINER=$(docker-compose ps -q service)
docker exec $CONTAINER /bin/bash -c 'echo "hello"'
docker exec $CONTAINER /bin/bash -c 'echo "world"'
docker port $CONTAINER 80 
docker port $CONTAINER 80

instead of

docker-compose exec service /bin/bash -c 'echo "hello"'
docker-compose exec service /bin/bash -c 'echo "world"' 
docker-compose port service 80
docker-compose port service 80

For example, on mac OS (see further down for an example with Ubuntu):

$ docker-compose -v
docker-compose version 1.24.1, build 4667896b
$ docker -v
Docker version 19.03.2, build 6a30dfc
$ time docker-compose exec service /bin/bash -c 'echo "hello"'
hello

real    0m0.990s
user    0m0.457s
sys 0m0.158s
$ time docker-compose exec service /bin/bash -c 'echo "hello"'
hello

real    0m0.898s
user    0m0.455s
sys 0m0.164s

$ time docker-compose port service 80
0.0.0.0:8088

real    0m0.675s
user    0m0.456s
sys 0m0.147s

$ time docker-compose port service 80
0.0.0.0:8088

real    0m0.657s
user    0m0.424s
sys 0m0.146s

Calling docker-compose exec twice and docker-compose port takes a total of 3.217 seconds.

One can get around this by using Docker directly, like this:

$ time CONTAINER=$(docker-compose ps -q service)

real    0m0.674s
user    0m0.422s
sys 0m0.131s

$ time docker exec $CONTAINER /bin/bash -c 'echo "hello"'
hello

real    0m0.252s
user    0m0.036s
sys 0m0.017s

$ time docker exec $CONTAINER /bin/bash -c 'echo "hello"'
hello

real    0m0.222s
user    0m0.032s
sys 0m0.017s

$ time docker port $CONTAINER 80
0.0.0.0:8088

real    0m0.473s
user    0m0.043s
sys 0m0.129s

$ time docker port $CONTAINER 80
0.0.0.0:8088

real    0m0.073s
user    0m0.032s
sys 0m0.021s

With Docker this takes 1.694 seconds, including the call to convert a docker-compose service to a docker container UUID; this is 47.34% faster.

I tried the same thing on Ubuntu:

$ docker-compose -v
docker-compose version 1.14.0, build c7bdf9e
$ docker -v
Docker version 17.09.0-ce, build afdb6d4

$ time docker-compose exec service /bin/bash -c 'echo "hello"'
hello

real    0m0.611s
user    0m0.448s
sys 0m0.084s

$ time docker-compose exec service /bin/bash -c 'echo "hello"'
hello

real    0m0.667s
user    0m0.472s
sys 0m0.092s

$ time docker-compose port service 80
0.0.0.0:32769

real    0m0.613s
user    0m0.484s
sys 0m0.076s

$ time docker-compose port service 80
0.0.0.0:32769

real    0m0.541s
user    0m0.424s
sys 0m0.064s

This took a total of 2.423 seconds. Now, using Docker directly:

$ time CONTAINER=$(docker-compose ps -q service)

real    0m0.656s
user    0m0.488s
sys 0m0.108s

$ time docker exec $CONTAINER /bin/bash -c 'echo "hello"'
hello

real    0m0.074s
user    0m0.008s
sys 0m0.008s

$ time docker exec $CONTAINER /bin/bash -c 'echo "hello"'
hello

real    0m0.075s
user    0m0.000s
sys 0m0.016s

$ time docker port $CONTAINER 80
0.0.0.0:32769

real    0m0.019s
user    0m0.004s
sys 0m0.012s

$ time docker port $CONTAINER 80
0.0.0.0:32769

real    0m0.017s
user    0m0.012s
sys 0m0.000s
$ Connection to 35.231.72.193 closed by remote host.
Connection to 35.231.72.193 closed.

A total of 0.841 seconds, a 65.29% improvement over docker-compose.

Since it is possible to work around the slowness by fetching the container UUID from docker-compose and then calling docker directly for exec and port, perhaps Docker Compose could cache Docker UUID, so that behind the scenes, it would take a similar amount of time to call docker-compose exec (or port) as docker exec (or port).

[elifiner]

I wrote a (mostly) drop-in bash function to redirect docker-compose exec calls to docker exec to make them faster. It assumes all commands are interactive and always passes -it and could probably be improved to properly handle -T. The idea is to take the first positional parameter and convert it into a container id. The rest of the options (except -T / -it) seem to be the same for both docker-compose exec and docker exec.

# replacing slow "docker-compose exec"
docker-compose-exec() {
    local ARG=''
    local ARGS=''
    local COUNT=0
    local DIR=$(basename $(pwd))
    for ARG in "$@"; do
        case $ARG in
            -*)
                ARGS="$ARGS $ARG"
            ;;
            *)
                # first positional arg is container name, replace with container id
                if [ $COUNT == 0 ]; then
                    ARG=$(docker ps -q --filter "name=${DIR}_${ARG}")
                fi
                ARGS="$ARGS $ARG"
                COUNT=$((COUNT+1))
            ;;
        esac
    done
    docker exec -it $ARGS
}

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale[bot]]

This issue has been automatically closed because it had not recent activity during the stale period.

[salarmehr-cbs]

I know that this ticket is for some years ago, but I still face in the latest version of Docker. 😐 (Windows-host)