Cli: Permission denied errors using buildkit when Dockerfile is in directory with restricted files

[bossmc]

Steps to reproduce the issue:

1. Put a Dockerfile in /tmp
2. Ensure there's some non-readable/non-accessible folder in /tmp too (e.g a systemd PrivateTmp folder)
3. From somewhere else (e.g. ~/) run DOCKER_BUILDKIT=1 docker build -f /tmp/Dockerfile .

Describe the results you received:

$ DOCKER_BUILDKIT=1 docker build -f /tmp/Dockerfile .
[+] Building 0.1s (2/2) FINISHED                                                                                                                                                                                  
 => [internal] load .dockerignore                                                                                                                                                                            0.0s
 => => transferring context: 55B                                                                                                                                                                             0.0s
 => ERROR [internal] load build definition from Dockerfile                                                                                                                                                   0.0s
 => => transferring dockerfile: 144B                                                                                                                                                                         0.0s
------
 > [internal] load build definition from Dockerfile:
------
failed to solve with frontend dockerfile.v0: failed to resolve dockerfile: failed to build LLB: error from sender: open /tmp/systemd-private-59aef507f5954ec69c62db609c0164a3-ModemManager.service-4ohfhM: permission denied

Describe the results you expected:

The image is built successfully without trying to open other files/folders in /tmp.

Additional information you deem important (e.g. issue happens only occasionally):

Only happens with DOCKER_BUIILDKIT=1, if I'm not using buildkit, everything works fine.

Output of docker version:

$ docker version
Client: Docker Engine - Community
 Version:           19.03.5
 API version:       1.40
 Go version:        go1.12.12
 Git commit:        633a0ea838
 Built:             Wed Nov 13 07:29:52 2019
 OS/Arch:           linux/amd64
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          19.03.5
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.12
  Git commit:       633a0ea838
  Built:            Wed Nov 13 07:28:22 2019
  OS/Arch:          linux/amd64
  Experimental:     true
 containerd:
  Version:          1.2.10
  GitCommit:        b34a5c8af56e510852c35414db4c1f4fa6172339
 runc:
  Version:          1.0.0-rc8+dev
  GitCommit:        3e425f80a8c931f88e6d94a8c831b9d5aa481657
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

Output of docker info:

$ docker info
Client:
 Debug Mode: false
 Plugins:
  app: Docker Application (Docker Inc., v0.8.0)
  buildx: Build with BuildKit (Docker Inc., v0.3.1-tp-docker)

Server:
 Containers: 11
  Running: 0
  Paused: 0
  Stopped: 11
 Images: 88
 Server Version: 19.03.5
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: b34a5c8af56e510852c35414db4c1f4fa6172339
 runc version: 3e425f80a8c931f88e6d94a8c831b9d5aa481657
 init version: fec3683
 Security Options:
  apparmor
  seccomp
   Profile: default
 Kernel Version: 5.4.7-050407-generic
 Operating System: Linux Mint 19.2
 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 7.773GiB
 Name: antares
 ID: GEU5:P3UF:ZNZ4:JAS3:667E:A5EN:L3FO:IMQ5:7JZB:2GL6:PHXX:TB32
 Docker Root Dir: /var/lib/docker
 Debug Mode: true
  File Descriptors: 23
  Goroutines: 36
  System Time: 2020-01-10T18:02:30.961875272Z
  EventsListeners: 0
 Username: bossmc
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: true
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: No swap limit support

Additional environment details (AWS, VirtualBox, physical, etc.):

Running on a VirtualBox VM.

[bossmc]

Further investigation reveals that the docker client is the one failing to open the folder:

$ strace docker build ...
[...]
[pid  9740] openat(AT_FDCWD, "/tmp/systemd-private-59aef507f5954ec69c62db609c0164a3-ModemManager.service-4ohfhM", O_RDONLY|O_CLOEXEC <unfinished ...>
[pid  9740] <... openat resumed> )      = -1 EACCES (Permission denied)
[...]

[bossmc]

Workaround (from @tonistiigi - thanks!) is to do:

$ DOCKER_BUILDKIT=1 docker build -f - . < /tmp/Dockerfile
[+] Building 0.1s (3/3) FINISHED                                                                                                                                                                                  
 => [internal] load build definition from Dockerfile                                                                                                                                                         0.1s
 => => transferring dockerfile: 56B                                                                                                                                                                          0.0s
 => [internal] load .dockerignore                                                                                                                                                                            0.1s
 => => transferring context: 34B                                                                                                                                                                             0.0s
 => exporting to image                                                                                                                                                                                       0.0s
 => => writing image sha256:71de1148337f4d1845be01eb4caf15d78e4eb15a1ab96030809826698a5b7e30

[tonistiigi]

@tiborvass We should make a temporary copy of Dockerfile always, (not only when set from stdin) and send the temp copy instead.

[thaJeztah]

Also related https://github.com/docker/cli/issues/1938

ping @tiborvass PTAL