Buildkit: docker build command outputs general information lines to stderr instead of stdout when buildKit is enabled.

[pratiksanglikar]

Description

docker build commands output general information lines to standard error instead of standard output when buildKit is enabled.

This is particularly problematic for extra tooling we have around docker which depend on lines on stderror to determine if the command was successful.

Following is the output of docker build command which is returned on the stderr instead of stdout. The process has exitcode=0 so, it shouldn't ideally put these lines on stderr.

Steps to reproduce the issue:

1. Make sure to have Docker version >= 19.03
2. Enable BuildKit by setting environment variable - DOCKER_BUILDKIT=1
3. Run docker build command on any project.
4. Verify the output of the command is on stderr instead of stdout.

Describe the results you received:
General information lines are present on stderr.

#2 [internal] load .dockerignore
#2 transferring context: 35B done
#2 DONE 0.1s

#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 32B done
#1 DONE 0.1s

#3 [internal] load metadata for mcr.microsoft.com/dotnet/core/aspnet:3.0-bu...
#3 DONE 0.1s

#4 [base 1/2] FROM mcr.microsoft.com/dotnet/core/aspnet:3.0-buster-slim@sha...
#4 resolve mcr.microsoft.com/dotnet/core/aspnet:3.0-buster-slim@sha256:b0fd451ed5dcd9c86a5c5364b478ebfb411beada0740369127fa53bca365650f done
#4 DONE 0.0s

#5 [base 2/2] WORKDIR /app
#5 CACHED

#6 exporting to image
#6 exporting layers done
#6 writing image sha256:96324b5520b19fef0877dffb950a9f7b62ab23dcc65d3f1fb1cb09dabf113c7b
#6 writing image sha256:96324b5520b19fef0877dffb950a9f7b62ab23dcc65d3f1fb1cb09dabf113c7b 0.0s done
#6 naming to docker.io/library/webapplication7:dev done
#6 DONE 0.1s

Describe the results you expected:
General information lines should be present on stdout when there is no error.

Additional information you deem important (e.g. issue happens only occasionally):

Output of docker version:

Client: Docker Engine - Community
 Version:           19.03.2
 API version:       1.40
 Go version:        go1.12.8
 Git commit:        6a30dfc
 Built:             Thu Aug 29 05:26:49 2019
 OS/Arch:           windows/amd64
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          19.03.2
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.8
  Git commit:       6a30dfc
  Built:            Thu Aug 29 05:32:21 2019
  OS/Arch:          linux/amd64
  Experimental:     true
 containerd:
  Version:          v1.2.6
  GitCommit:        894b81a4b802e4eb2a91d1ce216b8817763c29fb
 runc:
  Version:          1.0.0-rc8
  GitCommit:        425e105d5a03fabd737a126ad93d62a9eeede87f
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

Output of docker info:

Client:
 Debug Mode: false
 Plugins:
  app: Docker Application (Docker Inc., v0.8.0)
  buildx: Build with BuildKit (Docker Inc., v0.3.0-5-g5b97415-tp-docker)

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 1
 Server Version: 19.03.2
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 894b81a4b802e4eb2a91d1ce216b8817763c29fb
 runc version: 425e105d5a03fabd737a126ad93d62a9eeede87f
 init version: fec3683
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 4.9.184-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 2
 Total Memory: 1.952GiB
 Name: docker-desktop
 ID: AVON:LYQZ:TO6V:5KVI:QJDV:E5OH:5EYZ:L6TJ:3AZL:HISV:F5QX:DEFA
 Docker Root Dir: /var/lib/docker
 Debug Mode: true
  File Descriptors: 28
  Goroutines: 43
  System Time: 2019-10-02T21:27:32.2913151Z
  EventsListeners: 1
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: true
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false
 Product License: Community Engine

Additional environment details (AWS, VirtualBox, physical, etc.):
Physical machine.
Running docker commands through Containers Tools for Visual Studio.

[pratiksanglikar]

Related issue - https://github.com/microsoft/DockerTools/issues/213

[tonistiigi]

Treating prints to stderr as error is not correct. Stdout is used for actual build results (eg. you can pipe a tarball of the built files) or for id with -q. https://www.jstorimer.com/blogs/workingwithcode/7766119-when-to-use-stderr-instead-of-stdout

[bwateratmsft]

In what scenario is that a thing for Docker? The build output isn't files, it's images, and trying to cram an image into STDOUT...I don't even think that would be possible.

Using STDERR for all logging output, while arguably technically correct, is highly irregular, and likely to cause more problems than solve.

[mdonoughe]

It would be nice to be able to build a container and capture the image ID at the end (like -q) but still have the build output as a separate stream.

[tonistiigi]

@mdonoughe --iidfile

@bwateratmsft

The build output isn't files, it's images

Build output is whatever you define in --output, eg. for stdout docker build -o - . > t.tar

[marc-guenther]

It would be nice to be able to build a container and capture the image ID at the end (like -q) but still have the build output as a separate stream.

How would one do that? Seems not to be possible, normal docker build always writes everything to stdout, and buildkit doesn't write anything to stdout. And --iidfile wants a physical file, so process substitition doesn't work either. :(