Compose: docker-compose build fails copy from previous stage but docker build succeeds

[AleF83]

Description of the issue

I'm trying to build image using docker-compose and it's fails to copy from previous stage.
When I'm running this build using docker build it succeeds.

Context information (for bug reports)

docker-compose version 1.20.1, build 5d8c71b
docker-py version: 3.1.4
CPython version: 3.6.4
OpenSSL version: OpenSSL 1.0.2n  7 Dec 2017

Client:
 Version:       18.03.0-ce
 API version:   1.37
 Go version:    go1.9.4
 Git commit:    0520e24
 Built: Wed Mar 21 23:06:22 2018
 OS/Arch:       darwin/amd64
 Experimental:  true
 Orchestrator:  kubernetes

Server:
 Engine:
  Version:      18.03.0-ce
  API version:  1.37 (minimum version 1.12)
  Go version:   go1.9.4
  Git commit:   0520e24
  Built:        Wed Mar 21 23:14:32 2018
  OS/Arch:      linux/amd64
  Experimental: true

api:
    build: .
    environment:
      ASPNETCORE_ENVIRONMENT: Development

Steps to reproduce the issue

1. docker-compose build -f docker-compose.yml

Observed result

Build fails

Expected result

Build succeeded

Stacktrace / full error message

Building api
Step 1/16 : FROM debian:stretch-slim as csproj-files
 ---> dd99abd0503e
Step 2/16 : COPY . /src
 ---> Using cache
 ---> 59df2c35ad19
Step 3/16 : RUN find /src -type f -not -name "*.csproj" -delete && find /src -type d -empty -delete
 ---> Using cache
 ---> b4ae90c81dc4

Step 4/16 : FROM microsoft/aspnetcore-build:2.0.5-2.1.4 as source
 ---> e9127190cede
Step 5/16 : WORKDIR /src
 ---> Using cache
 ---> 89ecec8c69f2
Step 6/16 : COPY --from=csproj-files /src .
ERROR: Service 'api' failed to build: COPY failed: stat /var/lib/docker/overlay2/d315f23cde90af1f7d022aa28a2554a1ba8cd20e30639b7828754481f99ee04d/merged/src: no such file or directory
error An unexpected error occurred: "Command failed.

Additional information

I tried with swarm orchestrator too. The same behaviour.

System Version: macOS 10.13.4 (17E199)
Kernel Version: Darwin 17.5.0

[shin-]

What does your working docker build command look like? Your Dockerfile fails the same way for me with both docker-compose and docker:

$ docker build --no-cache .
Sending build context to Docker daemon  3.072kB
Step 1/7 : FROM debian:stretch-slim as csproj-files
 ---> dd99abd0503e
Step 2/7 : COPY . /src
 ---> 9add57906714
Step 3/7 : RUN find /src -type f -not -name "*.csproj" -delete &&     find /src -type d -empty -delete
 ---> Running in 3bd46ee81612
Removing intermediate container 3bd46ee81612
 ---> 561d81b4eb0d
Step 4/7 : FROM microsoft/aspnetcore-build:2.0.5-2.1.4 as source
 ---> e9127190cede
Step 5/7 : WORKDIR /src
Removing intermediate container 10aef2c5478e
 ---> 4681b8a28539
Step 6/7 : COPY --from=csproj-files /src .
COPY failed: stat /var/lib/docker/aufs/mnt/32dcd51817bbd2a58ecc21ac3b69d9ba376a2c3f9e3b303c0920175c37051ab3/src: no such file or directory

[AleF83]

Nope.
docker command works. Only docker-compose fails.

Docker:

docker build --no-cache .
Sending build context to Docker daemon  413.7kB
Step 1/16 : FROM debian:stretch-slim as csproj-files
 ---> dd99abd0503e
Step 2/16 : COPY . /src
 ---> a1c0f5796c4d
Step 3/16 : RUN find /src -type f -not -name "*.csproj" -delete && find /src -type d -empty -delete
 ---> Running in 88077ad8a9f3
Removing intermediate container 88077ad8a9f3
 ---> 6cb8f565b3b0
Step 4/16 : FROM microsoft/aspnetcore-build:2.0.5-2.1.4 as source
 ---> e9127190cede
Step 5/16 : WORKDIR /src
Removing intermediate container 7cb8b5f457ea
 ---> b7f6acde3327
Step 6/16 : COPY --from=csproj-files /src .
 ---> ba2463fd9681

Docker-compose:

docker-compose -f docker-compose.yml build api
Building api
Step 1/16 : FROM debian:stretch-slim as csproj-files
 ---> dd99abd0503e
Step 2/16 : COPY . /src
 ---> Using cache
 ---> e878f63c39e5
Step 3/16 : RUN find /src -type f -not -name "*.csproj" -delete && find /src -type d -empty -delete
 ---> Using cache
 ---> 5ee83e7f7ccc

Step 4/16 : FROM microsoft/aspnetcore-build:2.0.5-2.1.4 as source
 ---> e9127190cede
Step 5/16 : WORKDIR /src
 ---> Using cache
 ---> b7f6acde3327
Step 6/16 : COPY --from=csproj-files /src .
ERROR: Service 'api' failed to build: COPY failed: stat /var/lib/docker/overlay2/f73c5df62c188ebb55ec12fc82fd38fa9a2ef0d2336f6143ce8fc04a84d4eadb/merged/src: no such file or directory

[olivere]

Just FYI: I see the exact same error with the COPY command in a project that used to work before upgrading to Docker Edge 18.03.0-ce-mac58 (23607). That Docker version comes with Compose version 1.20.1. After downgrading to Compose 1.19.0, the problem disappeared. Maybe someone with the same problem can confirm?

[vovimayhem]

Maybe it's related - I'm seeing docker build cache-missing a layer when I previously built an image using docker-compose build...

[AleF83]

Reproduced in docker-compose 1.21.0 as well.

[olivere]

I see this even in 1.21.1 and 1.21.2 (Build a133471) here.

[jojo1981]

I have the exact same error and using docker-compose 1.21.1, build 5a3f1a3

[shin-]

Thank you for corroborating. If one of you can share a repo that reproduces the issue, that would go a long way in helping me identify the cause and pushing a fix.

[olivere]

Hmm... after updating to Docker Edge 18.05.0-ce-mac66 (24545), build e4c62d3168, I cannot reproduce any more.

I was on Docker Edge 18.03.0-ce-mac58 (23607) before and that failed with basically all repositories in all projects. I had to downgrade to Docker Compose 1.19.0 to get it to work with 18.03.0-ce. I was always getting that no such file or directory error mentioned above in one of the comments, I think it was always in a COPY step.

I'm sorry for the confusion. I don't know how the Docker tools work together in detail, so I hope the above makes any sense to you.

[AleF83]

I've upgraded to 18.05.0-ce-mac66 (24545) and the issues still happens for me.
@shin- Try to run yarn docker-compose build api on Tweek project repo (branch gateway).

[AleF83]

I've upgraded to version 1.22.0 and it works for me!