Compose: docker-compose super slow on macOS Catalina

[benlumia007]

when i do docker-compose up -d, it takes a while to start the process, same when i do docker-compose restart.

in other words, there's a delay of few seconds before it begins to restart containers or stopping containers.

[glours]

Hello @benlumia007
Can you give more details, for example the output for docker-compose --log-level debug up -d or docker-compose --log-level debug restart -d

[benlumia007]

is not about the output of docker-compose, is when i type the following

docker-compose up -d and press enter, it just delays for 5 - 10 seconds before it shows restarting or stopping

[benlumia007]

let me do some testing, i'll comment later , it just could be my eyes are deceiving me lol

[ndeloof]

Are you using https://github.com/docker/compose/blob/master/contrib/completion/bash/docker-compose for command line completion ?

[skazancev]

same problem, macos catalina (10.15 19A583), docker 2.1.0.3 (38240), docker-compose 1.24.1

[benlumia007]

so this is what it supposed to do, when you docker-compose up -d or docker-compose restart

but for macOS Catalina there is a delay between

benlumia007@aatrox:~/WordPress$ docker-compose restart
````
delays 5 seconds, then 

Restarting docker-nginx ... done
Restarting docker-phpfpm ... done
Restarting docker-mysql ... done
Restarting docker-mailhog ... done
benlumia007@aatrox:~/WordPress$
````

[glours]

Can you post the config of your compose file (without any sensitive data of course)
docker-compose config
Just to try to reproduce with the same kind of application (number of containers, eventually network config ...)

[benlumia007]

benlumia007@aatrox:~/WordPress$ docker-compose config
services:
  mailhog:
    container_name: docker-mailhog
    image: benlumia007/mailhog
    ports:
    - published: 1025
      target: 1025
    - published: 8025
      target: 8025
    restart: always
  mysql:
    command: --default-authentication-plugin=mysql_native_password
    container_name: docker-mysql
    environment:
      MYSQL_ROOT_PASSWORD: root
    image: benlumia007/mysql:latest
    ports:
    - published: 3306
      target: 3306
    restart: always
    volumes:
    - db_data:/var/lib/mysql:rw
  nginx:
    container_name: docker-nginx
    depends_on:
    - phpfpm
    image: benlumia007/nginx:latest
    ports:
    - published: 80
      target: 80
    - published: 443
      target: 443
    restart: always
    volumes:
    - /home/benlumia007/WordPress/config/nginx:/etc/nginx/conf.d:rw
    - /home/benlumia007/WordPress/certificates:/etc/nginx/certs:rw
    - /home/benlumia007/WordPress/sites:/var/www/html:rw
  phpfpm:
    container_name: docker-phpfpm
    image: benlumia007/php:7.3-fpm
    ports:
    - target: 9000
    restart: always
    volumes:
    - /home/benlumia007/WordPress/sites:/var/www/html:rw
version: '3.5'
volumes:
  db_data: {}

[benlumia007]

it was pretty good, to begin with when using macOS Mojave, it runs fine, then Catalina just delays a few seconds.

I don't think the issue is how it is config, I think its just the docker itself may not be tested fully under macOS Catalina

[carlosonunez]

I am experiencing the same issue. docker-compose can take between 5-60+ seconds to do anything, despite docker commands being executed instantaneously. This makes Compose pretty unusable, tbh.

I didn't see anything coming from docker-compose during startup during tcpdump captures except for UDP traffic within Compose. I did see a lot of tccd and XprotectService sprinkled in during fs_usage traces while docker-compose runs, though I'm not sure if other processes are triggering this activity or not. Trying to invoke docker-compose --log-level debug also incurs a delay.

Interestingly, docker-compose runs very quickly after turning off any networking, so it's definitely something network-related...I just can't see it.

One clue is that whenever I SIGHUP Compose, I get errors that seem to be coming from PyInstaller. This makes me think that running Compose directly from within Python would be much faster and that the issue is with PyInstaller doing...things.

[carlosonunez]

I was able to work around this by installing docker-compose from Pip instead of using the precompiled one:

$: pip install docker-compose

This should set your default Compose to the Python one, but double-check just to make sure:

$: file $(which docker-compose) # should NOT be a Mach-O executable

[benlumia007]

yeah but it stil doesn't solve the problem whom is using the compiled version. I'll try that too. but this should be a bug for catalina.

[carlosonunez]

I think it's a PyInstaller bug as well :(

[wodCZ]

Running into same issue. docker commands are instant:

➜  sites time docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
docker ps  0.03s user 0.08s system 49% cpu 0.232 total

Running docker-compose from Docker for Mac is terribly slow, even when running just version command outside of any docker-compose project:

➜  sites time docker-compose-bundled version
docker-compose version 1.24.1, build 4667896b
docker-py version: 3.7.3
CPython version: 3.6.8
OpenSSL version: OpenSSL 1.1.0j  20 Nov 2018
docker-compose-bundled version  0.43s user 0.14s system 24% cpu 2.338 total

Then I installed docker-compose using pip3 (pip3 install docker-compose --user) and the result is a bit better (but still so slow given that I'm only running version command)

➜  sites time docker-compose version
docker-compose version 1.24.1, build 4667896
docker-py version: 3.7.3
CPython version: 3.7.3
OpenSSL version: LibreSSL 2.8.0
docker-compose version  0.60s user 0.14s system 80% cpu 0.931 total

Of course other subcommands are slow too, I thought using version as an example would eliminate an issue with docker-compose.yml file.
I'm running macOS Catalina Version 10.15.

I've tried turning off network, also tried popular /etc/hosts fix from https://github.com/docker/compose/issues/3419#issuecomment-221793401 - no success in any case.

[diogocnunes]

I have a same issue running on macOS Cataina. If i run the docker-compose version, this take some time, but works, like example below:

If I run under a proxy, the commands doesn't work. It's freeze (this image below it's a animated GIF, not a static image.... i wait for 30 seconds before CTRL+C... watch it...)

I try run the same commands, under proxy, on macOS Mojave and it's works fine...

[ndeloof]

I was able to reproduce this issue by updating my own OSX machine to Catalina.
I noticed when running any docker-compose command, even unkown ones (which just show the usage help) I get a request sent to https://api.apple-cloudkit.com. If I switch off my Internet connexion this request is skipped and compose run at normal speed. I assume using a proxy will just make this request hang some way and block docker-compose to run.

I've requested help on the PyInstaller mailing list to diagnose this issue.

Internally tracked as https://docker.atlassian.net/browse/COMPOSE-96

[benlumia007]

@diogocnunes: what software do you use the screen capture.

[carlosonunez]

@ndeloof How did you see those requests? I didn't see anything like that when I did a tcpdump capture. I used: sudo tcpdump 'port 80 or port 443'

[diogocnunes]

@diogocnunes: what software do you use the screen capture.

I use Kap (https://getkap.co/)

[diogocnunes]

I was able to reproduce this issue by updating my own OSX machine to Catalina.
I noticed when running any docker-compose command, even unkown ones (which just show the usage help) I get a request sent to https://api.apple-cloudkit.com. If I switch off my Internet connexion this request is skipped and compose run at normal speed. I assume using a proxy will just make this request hang some way and block docker-compose to run.

I've requested help on the PyInstaller mailing list to diagnose this issue.

Internally tracked as https://docker.atlassian.net/browse/COMPOSE-96

Adding https://api.apple-cloudkit.com to /etc/hosts, pointing to localhost, under proxy, docker-compose is working again ...

I don't know what the real impact of this, but it's one less problem right now :)

[ndeloof]

How did you see those requests?

I used charles proxy

I reported this issue on https://groups.google.com/forum/#!msg/pyinstaller/2MvcMaDFXyo/GhpVfaJJCQAJ and was able to reproduce using a minimalist "print hello" python application. Hope this will help maintainers to understand the root cause and propose a fix.

[justinpage]

When I run the following command, under our company's VPN, I notice the following behavior:

http://localhost:None "POST /v1.38/networks/test_default/connect HTTP/1.1" 200 0
Pending: set()
Pending: set()
Pending: set()
Pending: set()
Pending: set()
etc...
etc...

This goes on for some time until it finally connects

Pending: set()
Pending: set()
Pending: set()
Pending: set()
http://localhost:None "POST /v1.38/containers/cd03d59f314f95ff1b7c6aa38352bd506028ee542dc3d44f1fa1a29a3007f28b/start HTTP/1.
1" 204 0

This slowdown seems to happen in regards to the following command:

https://docs.docker.com/engine/api/v1.38/#operation/NetworkConnect

Not sure if this is related but I wanted to call this out in case this helps anyone.

[benlumia007]

@diogocnunes
I downloaded the getkap, but it won't work, i keep getting unable to start recording. are you on catalina right.

[diogocnunes]

@diogocnunes
I downloaded the getkap, but it won't work, i keep getting unable to start recording. are you on catalina right.

Yes. I use Catalina. Did you select the recording area before you started?

[mediaessenz]

Maybe this help someone else: https://github.com/Homebrew/homebrew-core/issues/45687#issuecomment-547102000

[kliyes]

I was able to work around this by installing docker-compose from Pip instead of using the precompiled one:

$: pip install docker-compose

This should set your default Compose to the Python one, but double-check just to make sure:

$: file $(which docker-compose) # should NOT be a Mach-O executable

works for me, thanks.
ps. if a virtualenv activated, you should deactivate it first after pip install docker-compose

[rfay]

5.8 seconds to do docker-compose --version on Catalina:

$ time docker-compose --version
docker-compose version 1.25.0-rc4, build 8f3c9c58

real    0m5.762s

[rfay]

@benlumia007 (or anyone with privs to do so) could you please edit the title to be more descriptive? Something like "docker-compose super slow on macOS Catalina" - thanks!

[mayconmfl]

@benlumia007 I have done added api.apple-cloudkit.com to /etc/hosts, pointing to localhost.
Now it's faster.

My /etc/hosts:

````
...

Added by Docker Desktop

To allow the same kube context to work on the host and the container:

127.0.0.1 kubernetes.docker.internal
127.0.0.1 api.apple-cloudkit.com
...

[wodCZ]

I have downloaded compose binary 1.25.1-rc1, which should include the potential fix from #7010 and it still runs slow:

➜  ~ time docker-compose --version
docker-compose version 1.25.1-rc1, build d92e9bee
docker-compose --version  0.49s user 0.14s system 20% cpu 3.178 total
➜  ~ time docker-compose --version
docker-compose version 1.25.1-rc1, build d92e9bee
docker-compose --version  0.49s user 0.15s system 19% cpu 3.340 total
➜  ~ time docker-compose --version
docker-compose version 1.25.1-rc1, build d92e9bee
docker-compose --version  0.49s user 0.15s system 22% cpu 2.932 total
➜  ~ time docker-compose --version
docker-compose version 1.25.1-rc1, build d92e9bee
docker-compose --version  0.51s user 0.15s system 20% cpu 3.244 total

Latest version installed using pip as mentioned earlier in this issue is considerably faster (except the first invocation):

➜  ~ time docker-compose --version
docker-compose version 1.25.0, build b42d419
docker-compose --version  0.45s user 0.18s system 40% cpu 1.566 total
➜  ~ time docker-compose --version
docker-compose version 1.25.0, build b42d419
docker-compose --version  0.32s user 0.07s system 96% cpu 0.403 total
➜  ~ time docker-compose --version
docker-compose version 1.25.0, build b42d419
docker-compose --version  0.31s user 0.07s system 97% cpu 0.392 total
➜  ~ time docker-compose --version
docker-compose version 1.25.0, build b42d419
docker-compose --version  0.30s user 0.07s system 98% cpu 0.370 total

[ndeloof]

@wodCZ fix is imlemented in the docker-compose-Darwin-x86_64.tgz "folder-style" installation, the single binary is kept for backward compatibility and pre-catalina users who don't need this fix.

[zeshan77]

@benlumia007 I have done added api.apple-cloudkit.com to /etc/hosts, pointing to localhost.
Now it's faster.

My /etc/hosts:

...
# Added by Docker Desktop
# To allow the same kube context to work on the host and the container:
127.0.0.1 kubernetes.docker.internal
127.0.0.1 api.apple-cloudkit.com
...

This reduced waiting time from 18s to 2s for me. Thanks for sharing!

Before:

After:

[ndeloof]

@zeshan77 can you give doker-compose 1.25 tar.gz release a try ?
etc/hosts hack can only be considered a temporary workaround

[wodCZ]

@ndeloof the .tgz version seems to work much better than the bundled or the binary:

time ./docker-compose --version
docker-compose version 1.25.1-rc1, build d92e9bee
./docker-compose --version  0.50s user 0.07s system 97% cpu 0.587 total
time ./docker-compose --version
docker-compose version 1.25.1-rc1, build d92e9bee
./docker-compose --version  0.50s user 0.08s system 97% cpu 0.593 total
time ./docker-compose --version
docker-compose version 1.25.1-rc1, build d92e9bee
./docker-compose --version  0.48s user 0.07s system 98% cpu 0.567 total
time ./docker-compose --version
docker-compose version 1.25.1-rc1, build d92e9bee
./docker-compose --version  0.48s user 0.07s system 97% cpu 0.568 total

the version installed using pip still seems to be faster though:

time docker-compose --version
docker-compose version 1.25.0, build b42d419
docker-compose --version  0.38s user 0.10s system 97% cpu 0.487 total
time docker-compose --version
docker-compose version 1.25.0, build b42d419
docker-compose --version  0.38s user 0.09s system 97% cpu 0.491 total
time docker-compose --version
docker-compose version 1.25.0, build b42d419
docker-compose --version  0.38s user 0.09s system 97% cpu 0.480 total

[BirkhoffLee]

@wodCZ fix is imlemented in the docker-compose-Darwin-x86_64.tgz "folder-style" installation, the single binary is kept for backward compatibility and pre-catalina users who don't need this fix.

Can you elaborate on "folder-style" installation? How do I do that? Running the executable in that archive is same slow (20+ second for me).

[ndeloof]

pyinstaller used by docker-compose to bake a binary executable can both create a single-file binary, or a folder with binary + all libraries.
In the former case, unpacking at execution time trigger Catalina security scanning, so slowness. In the later, scan will occur on first run, but then executable is flagged by system as "_allready scanned_"

so, unpacking the tar.gz and running docker-compose version from there should fix the issue (you can create a symlink to binary in folder)

[Yelfive]

I was able to work around this by installing docker-compose from Pip instead of using the precompiled one:

$: pip install docker-compose

This should set your default Compose to the Python one, but double-check just to make sure:

$: file $(which docker-compose) # should NOT be a Mach-O executable

@carlosonunez I installed with pip, but no executable found anywhere, what did I miss please?

[ulyssessouza]

@Yelfive As mentioned by @ndeloof in https://github.com/docker/compose/issues/6956#issuecomment-561507679 we now release in two flavors for macOS. Please refer to the releases page and get the docker-compose-Darwin-x86_64.tgz from the latest one.

[Yelfive]

@ulyssessouza Thx for your reply, finally I got it to work. I previously tried all versions of the binary release, and missed the tgz part. Thank you again.

[SteffenAnders]

We have solved the problem by installing docker-compose as a container.
See https://docs.docker.com/compose/install/ → "Alternative Install Options".

sudo curl -L --fail https://github.com/docker/compose/releases/download/1.26.0/run.sh -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose

docker-compose --version