Compose: "docker-compose exec" does not write to STDERR

[felixhummel]

This is similar to #3267, but for exec.

When you exec something, both STDOUT and STDERR are on STDOUT.

I created an example repository to reproduce this.

My use case is quite simple:
I have a Mysql instance and want to run a database dump, piping its output to the host, e.g.

docker-compose exec mysql mysqldump -ufoo -pbar baz > /var/backups/foo.sql

Having its warning messages (from STDERR) inside the dump file is a problem.

PS: Fun fact: On Postgres, I did not stumble upon this problem, because pg_dump rocks! ;)

[shin-]

You need to use -T to disable TTY allocation.

[felixhummel]

This did not work for me. See https://github.com/felixhummel/compose-bug-stderr#the-minus-t-flag

versions:

docker-compose version 1.11.2, build dfed245
Docker version 17.06.2-ce, build cec0b72

[shin-]

Indeed, my bad. I'll take a look at our options here when I have some time.

[teneri66]

Quick workaround for now can be ignoring stderr in container. So instead of

docker-compose exec mysql mysqldump -ufoo -pbar baz > /var/backups/foo.sql

you can have:

docker-compose exec mysql sh -c "mysqldump -ufoo -pbar baz 2>/dev/null" > /var/backups/foo.sql

[shin-]

This should work with 1.19.0 RC1 - please try it out and let me know!

[felixhummel]

It works beautifully. Thanks!

felix@locutus /tmp $ sudo wget -q https://github.com/docker/compose/releases/download/1.19.0-rc1/docker-compose-Linux-x86_64 -O /usr/local/bin/docker-compose
felix@locutus /tmp $ sudo chmod +x /usr/local/bin/docker-compose
felix@locutus /tmp $ [[ $(which docker-compose) == /usr/local/bin/docker-compose ]] && echo OK
OK

felix@locutus /tmp $ git clone -q https://github.com/felixhummel/compose-bug-stderr.git
felix@locutus /tmp $ cd compose-bug-stderr/

felix@locutus master /tmp/compose-bug-stderr $ docker-compose build
Building app
Step 1/4 : FROM alpine:3.6
3.6: Pulling from library/alpine
605ce1bd3f31: Pull complete
Digest: sha256:3d44fa76c2c83ed9296e4508b436ff583397cac0f4bad85c2b4ecc193ddb5106
Status: Downloaded newer image for alpine:3.6
 ---> 77144d8c6bdc
Step 2/4 : COPY bin/main /usr/local/bin/main
 ---> 59fb09ea7d21
Step 3/4 : COPY bin/dump /usr/local/bin/dump
 ---> b55f922132f0
Step 4/4 : CMD ["/usr/local/bin/main"]
 ---> Running in 72fcccafbee4
Removing intermediate container 72fcccafbee4
 ---> 50621f018d01
Successfully built 50621f018d01
Successfully tagged composebugstderr_app:latest

felix@locutus master /tmp/compose-bug-stderr $ docker-compose up -d
Creating network "composebugstderr_default" with the default driver
Creating composebugstderr_app_1 ... done

felix@locutus master /tmp/compose-bug-stderr $ docker exec $(docker-compose ps -q app) /usr/local/bin/dump > /dev/null
Hello stderr!

felix@locutus master /tmp/compose-bug-stderr $ docker exec $(docker-compose ps -q app) /usr/local/bin/dump 2> /dev/null
Hello stdout!

felix@locutus master /tmp/compose-bug-stderr $ docker exec $(docker-compose ps -q app) /usr/local/bin/dump
Hello stderr!
Hello stdout!

[timwhite]

@felixhummel Am I missing something? Your attempt with 1.19.0 appears to use docker exec instead of docker-compose exec for the stdout/stderr test.
I'm on docker-compose 1.21.2, and still have this issue where both stdout and stderr from the container are combined.

[felixhummel]

@timwhite you're right, but at least using -T looks good:

$ docker-compose --version
docker-compose version 1.21.2, build a133471
$ docker-compose exec -T app /usr/local/bin/dump > /tmp/stdout
Hello stderr!
$ cat /tmp/stdout 
Hello stdout!

[greg0ire]

Furthermore, if you use docker exec -t, it doesn't work either.

You need to use -T to disable TTY allocation.

Looks like allocating a TTY implies not having this work properly, and it also looks like this will not be fixed, here is why: https://github.com/moby/moby/issues/19696#issuecomment-174832094

So this isn't really a docker-compose issue, unless maybe you think allocating a tty should be opt-in (right now it is opt-out).