Cli: Error building from a git repository

[jkfran]

Description

docker build seems to not work as expected when building from a git repository.

Steps to reproduce the issue:

1. export DOCKER_BUILDKIT=1
2. docker build --pull --build-arg BUILD_ID=test -t test https://github.com/canonical-web-and-design/snapcraft.io.git

Describe the results you received:

[+] Building 2.5s (12/27)                                                                                                   
 => CACHED [internal] load git source https://github.com/canonical-web-and-design/snapcraft.io.git                     0.0s
 => resolve image config for docker.io/docker/dockerfile:experimental                                                  0.8s
 => CACHED docker-image://docker.io/docker/dockerfile:experimental@sha256:de85b2f3a3e8a2f7fe48e8e84a65f6fdd5cd5183afa  0.0s
 => [internal] load metadata for docker.io/library/ubuntu:focal                                                        0.3s
 => [internal] load metadata for docker.io/library/node:12-slim                                                        0.5s
 => [yarn-dependencies 1/4] FROM docker.io/library/node:12-slim@sha256:7676294fed76a8127254821edc6891a3fa304dec73117e  0.0s
 => CACHED [python-dependencies 1/4] FROM docker.io/library/ubuntu:focal@sha256:8bce67040cd0ae39e0beb55bcb976a824d996  0.0s
 => CANCELED [python-dependencies 2/4] RUN apt-get update && apt-get install --no-install-recommends --yes python3-pi  0.7s
 => ERROR [stage-4  2/10] ADD . .                                                                                      0.0s
 => CACHED [yarn-dependencies 2/4] WORKDIR /srv                                                                        0.0s
 => CACHED [yarn-dependencies 3/4] ADD package.json .                                                                  0.0s
 => CANCELED [yarn-dependencies 4/4] RUN --mount=type=cache,target=/usr/local/share/.cache/yarn yarn install           0.8s
------
 > [stage-4  2/10] ADD . .:
------
failed to solve with frontend dockerfile.v0: failed to solve with frontend gateway.v0: rpc error: code = Unknown desc = fail
ed to build LLB: cannot replace to directory /var/lib/docker/overlay2/zkrx9qomlbu5zohuwjhzcenwx/merged/run with file

Describe the results you expected:

The image to be built.

Additional information you deem important (e.g. issue happens only occasionally):

Output of docker version:

Docker version 19.03.8, build afacb8b7f0

Output of docker info:

Client:
 Debug Mode: false

Server:
 Containers: 148
  Running: 0
  Paused: 0
  Stopped: 148
 Images: 258
 Server Version: 19.03.8
 Storage Driver: overlay2
  Backing Filesystem: <unknown>
  Supports d_type: true
  Native Overlay Diff: true
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 
 runc version: 
 init version: 
 Security Options:
  apparmor
  seccomp
   Profile: default
 Kernel Version: 5.4.0-37-generic
 Operating System: Ubuntu 20.04 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 15.33GiB
 Name: fran-ThinkPad-X1-Carbon-7th
 ID: 6TJF:F6W3:EBAT:XTVP:5IVJ:VQAZ:HAK7:2C6Z:Y3I5:CFCX:X6LG:VRDC
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: No swap limit support

Additional environment details (AWS, VirtualBox, physical, etc.):

Physical

[thaJeztah]

Doesn't look like a bug, but an issue with the Dockerfile; in this step; https://github.com/canonical-web-and-design/snapcraft.io/blob/4e5c3ba395c4324896d13211289fd2480756b5c0/Dockerfile#L34-L36

FROM ubuntu:focal

ADD . .

You're copying the whole repository to the root of an ubuntu:focal image, however, there's a /run directory in that image;

docker run --rm ubuntu:focal sh -c 'ls -la | grep run'
drwxr-xr-x   1 root root 4096 Apr 24 01:07 run

And the files you're copying have a _file_ named run at the root; https://github.com/canonical-web-and-design/snapcraft.io/blob/4e5c3ba395c4324896d13211289fd2480756b5c0/run

The build is failing because you cannot replace a directory with a file. If this build worked before (without BuildKit), that may be due to a bug in the classic builder, that allowed situations like this (see https://github.com/moby/moby/issues/41035)

I'm closing this issue because this is not a bug, but feel free to continue the conversation