Compose: "docker-compose run -T" inconsistently outputs stdout / stderr?

[cjerdonek]

I've noticed that docker-compose run -T seems to behave inconsistently from run to run with respect to outputting stderr and stdout.

For example, on repeated invocations of the following command:

$ docker-compose run -T web python3 -c "import sys; print(\"err\\n\", file=sys.stderr); print(\"stdout\\nstdout\\nstdout\\nstdout\\n\")"

the "err" usually doesn't print out (but sometimes does), and the "stdout" messages usually (but don't always) print.

This is a follow-up to issue #3239.

[cjerdonek]

I'm using Compose version 1.5.1. I'm hesitant to upgrade and try on the latest version though because of breaking changes the last time I tried.

[dnephin]

I've tried this a few dozen times with both master and 1.7.0rc1. I always see both streams, I always see stderr on stderr and stdout on stdout. I never see any missing output. Sometimes the order is inverted, but that's to be expected with separate streams that use different buffers.

I can't reproduce the issue.

[cjerdonek]

Okay, other things I can try are -- (1) try this on a stripped-down container, and (2) try with the newest release of compose. I should be able to try (1) within the next couple days. (2) may take a bit longer.

[dnephin]

I used the python:3 image to test it.

You can always just download the docker-compose binary to a local path, you don't have to install it to the system right away if you just want to test it out.

[cjerdonek]

Good points. Thanks. Yes, I don't have to upgrade Docker toolbox. I'll try to try within the next few days.

[cjerdonek]

Okay, I tried reproducing with compose 1.7.0rc1 and the plain python:3 image. About a third of the time, the err stream doesn't output, but the stdout stream always does. With compose 1.5.1 and the plain python:3 image, I observed the same as what I originally reported. So it looks like going from 1.5.1 to 1.7.0rc1 makes things a bit better, but there is still an issue with flakiness for me.

[dnephin]

Are you sure it's not just an issue with the order? Sometimes it's output after the stdout lines.

Are you running these tests on OSX? It may be an OSX issue, I was running the tests on Ubuntu.

[cjerdonek]

No, it's not just the order. The total output is only 5 or 6 lines, so it's easy to see all the output. And yes, it's Mac OS X (should have made that clear from the beginning).

[tuxpiper]

FWIW: we've also noticed this on Mac OS X, while using our in-house tool that wraps docker-compose execution. Piping the output of docker-compose run results in silence (or just partial output). This happens both using -T and without it.

Interestingly, in our case 1.6.0 exhibited the problem, then it worked fine with 1.6.2 , and now 1.7.0 exhibits the problem again

[shadiakiki1986]

My 2 cents: I came across this issue while looking for a reason why when I run a docker-compose build --no-cache service > /dev/null it outputs Building service to stderr instead of stdout. Coincidentally, I had seen a similar issue 30 mins earlier in mongoexport. The problem there was that exported n records was showing up on stderr instead of stdout. That was fixed in mongo 3.3.6, versus 3.2 which had the issue. It made me think that perhaps a newer version of docker-compose would also have addressed this. I'm at docker-compose version 1.7.1, build 0a9ab35

[cjerdonek]

My 2 cents: I came across this issue while looking for a reason why when I run a docker-compose build --no-cache service > /dev/null it outputs Building service to stderr instead of stdout.

@shadiakiki1986 What you describe seems unrelated because this issue is about docker-compose run and you're describing something about docker-compose build. Also, it seems like "Building service" _should_ go to stderr rather than stdout since that is a progress message.

[allcode]

We've run in, what seems like, the same issue with docker-compose 1.7.1, build 0a9ab35 on Mac OS X El Capitan - 10.11.5 (15F34).

When running a bash script inside the container, like such:

#!/usr/bin/env bash

for ((i=1; i<51; i++)); do
    echo "Line ${i}"
    sleep 0.1
done

a random amount of lines will be missing from stdout, when docker-compose is executed like such:

#!/usr/bin/env bash

sh -c "docker-compose run -T --rm test bash script.sh 1> stdout.log 2> stderr.log"

Both with and without the -T flag.
When docker-compose is ran directly from terminal, it will only have partial output with the -T flag.
When removing the sleep 0.1 from the bash script, it will always output all lines.

We've tested it with alpine:latest, debian:latest and debian:wheezy without any extras (except bash).

Using:

• docker-machine version 0.7.0, build a650a40
• Docker version 1.11.2, build b9f10c9
• VirtualBox version 5.0.20 r106931

[bcb]

Similar to what @allcode said - adding sleep 1 to the top of my script gives output, without that I get nothing.

[stefansedich]

I am seeing the exact same running a ruby script like this:

loop do
puts "Hello\n"
sleep 1
end

No output at all, if I run a container and run the script inside the container directly it is fine, I have sometimes seen it work but it seems very flakey, changing STDOUT.sync to true makes it work but docker compose is the only thing experiencing this, running anywhere is fine.

I am running 1.8.1.

[shouze]

Ok, experiencing the very same issue and having an intersting use case:

$ docker-compose run -w /home/node -T --rm --no-deps --entrypoint sh opening_hours -c 'tar -cf - node_modules' > /tmp/archive1.tar
$ docker run -w /home/node --rm --entrypoint sh rezzza/opening_hours:1.0.1-node -c "tar -cf - node_modules" > /tmp/archive2.tar
$ ls -l /tmp/archive*
-rw-r--r--  1 shouze  wheel  74481780  8 déc 15:46 /tmp/archive1.tar
-rw-r--r--  1 shouze  wheel  74711040  8 déc 15:50 /tmp/archive2.tar

So, I can't dump to stdout a tar with docker run, of course if I try to extract it I have messages saying it's a damaged tar file with docker-compose run whereas all is ok with a docker run.

Docker version 1.12.3, build 6b644ec
docker-compose version 1.8.1, build 878cff1
Boot2Docker version 1.12.3, build HEAD : 7fc7575 - Thu Oct 27 17:23:17 UTC 2016

In the mean time... I've tested with docker for mac and I don't have the issue. So, to sum up:

• Same docker versions in both case, working with docker run every time
• Same docker-compose version, but working only with docker for mac, not with boot2docker

So... looks like related more to boot2docker, probably an issue related to the way stdout/stderr are streamed down?

[paddie]

Experiencing the same with docker-compose version:

docker-compose version 1.9.0, build 2585387
docker-py version: 1.10.6
CPython version: 2.7.12
OpenSSL version: OpenSSL 1.0.2j  26 Sep 2016

and docker version:

Client:
 Version:      1.13.0-rc2
 API version:  1.25
 Go version:   go1.7.3
 Git commit:   1f9b3ef
 Built:        Wed Nov 23 17:40:58 2016
 OS/Arch:      darwin/amd64

Server:
 Version:             1.13.0-rc2
 API version:         1.25
 Minimum API version: 1.12
 Go version:          go1.7.3
 Git commit:          1f9b3ef
 Built:               Wed Nov 23 17:40:58 2016
 OS/Arch:             linux/amd64
 Experimental:        true

Using the docker for mac seems to work, but introducting docker-compose into the mix seems to break it up locally. I'm seeing prints from a single service, but not another.

[shouze]

@paddie yes I've switched to docker for mac, on my side it's essentially to be handy with the IDE, I use this to bring back dependencies installed into builded containers to my host.

[shin-]

Is anyone still seeing the issue with Docker 4 Mac + a recent (1.11+) version of Compose?

[allcode]

@shin- This particular problem no longer occurs for us when using Docker for Mac beta.

[costa]

I see a similar problem: $ docker-compose exec -T ... spits both stdin and stdout to stdout whereas the same command yields correct outputs with $ docker exec ....

[shin-]

@costa Can you open a separate issue?

[costa]

@shin- I can confirm my issue is resolved in the latest version as well, thank you very much.