Buildkit: Failed to compute cache key in newer version

[DRoet]

This is a docker issue but it seems to be related to BuildKit only. this is something that was still working in docker ~19.03.10 but stopped functioning in 20.10.0+. I managed to bring down my DockerFile to a minimal repro:

This works (A.DockerFile):

FROM php:7.4.13-cli

COPY --from=composer:2.0.8 /usr/bin/composer /usr/local/bin/composer

This also works (B.DockerFile):

FROM php:7.4.13-cli

COPY --from=mlocati/php-extension-installer /usr/bin/install-php-extensions /usr/bin/

This no longer works (C.DockerFile):

FROM php:7.4.13-cli

COPY --from=mlocati/php-extension-installer /usr/bin/install-php-extensions /usr/bin/
COPY --from=composer:2.0.8 /usr/bin/composer /usr/local/bin/composer

Output from running A and C after eachother:

C:\Users\Test>set "DOCKER_BUILDKIT=1" & docker build -f A.Dockerfile .
[+] Building 3.6s (7/7) FINISHED
 => [internal] load build definition from A.Dockerfile                                                                                                                                                           0.0s
 => => transferring dockerfile: 132B                                                                                                                                                                             0.0s
 => [internal] load .dockerignore                                                                                                                                                                                0.0s
 => => transferring context: 2B                                                                                                                                                                                  0.0s
 => [internal] load metadata for docker.io/library/php:7.4.13-cli                                                                                                                                                2.9s
 => CACHED FROM docker.io/library/composer:2.0.8                                                                                                                                                                 0.0s
 => => resolve docker.io/library/composer:2.0.8                                                                                                                                                                  0.5s
 => CACHED [stage-0 1/2] FROM docker.io/library/php:7.4.13-cli@sha256:c099060944167d20100140434ee13b7c134bc53ae8c0a72e81b8f01c07a1f49d                                                                           0.0s
 => [stage-0 2/2] COPY --from=composer:2.0.8 /usr/bin/composer /usr/local/bin/composer                                                                                                                           0.1s
 => exporting to image                                                                                                                                                                                           0.1s
 => => exporting layers                                                                                                                                                                                          0.0s
 => => writing image sha256:ea6d75bc9ad24e800c8083e9ea6b7774f2bd9610cb0e61b3640058c9c7fe34c6                                                                                                                     0.0s

C:\Users\Test>set "DOCKER_BUILDKIT=1" & docker build -f C.Dockerfile .
[+] Building 1.0s (8/8) FINISHED
 => [internal] load build definition from C.Dockerfile                                                                                                                                                           0.0s
 => => transferring dockerfile: 221B                                                                                                                                                                             0.0s
 => [internal] load .dockerignore                                                                                                                                                                                0.0s
 => => transferring context: 2B                                                                                                                                                                                  0.0s
 => [internal] load metadata for docker.io/library/php:7.4.13-cli                                                                                                                                                0.2s
 => FROM docker.io/mlocati/php-extension-installer:latest                                                                                                                                                        0.0s
 => => resolve docker.io/mlocati/php-extension-installer:latest                                                                                                                                                  0.0s
 => => sha256:ccf3a05d8241580ad9d2a6c884a735bb248e90942ab23e0f8197f851a999ddac 526B / 526B                                                                                                                       0.0s
 => CACHED FROM docker.io/library/composer:2.0.8                                                                                                                                                                 0.0s
 => [stage-0 1/3] FROM docker.io/library/php:7.4.13-cli@sha256:c099060944167d20100140434ee13b7c134bc53ae8c0a72e81b8f01c07a1f49d                                                                                  0.0s
 => CACHED [stage-0 2/3] COPY --from=mlocati/php-extension-installer /usr/bin/install-php-extensions /usr/bin/                                                                                                   0.0s
 => ERROR [stage-0 3/3] COPY --from=composer:2.0.8 /usr/bin/composer /usr/local/bin/composer                                                                                                                     0.0s
------
 > [stage-0 3/3] COPY --from=composer:2.0.8 /usr/bin/composer /usr/local/bin/composer:
------
failed to compute cache key: "/usr/bin/composer" not found: not found

This doesn't happen consistently in my build, sometimes everything builds fine and there are no issues.
I'm using windows 10 (20H2) and the latest version of Docker Desktop that includes Docker version 20.10.2, build 2291f61, but I have also seen this happen on Linux with the same version

[yogevyuval]

Expiriencing the same thing, any update?

[adrubesh]

Same issue here. Experiencing the issue using both docker-compose build and docker compose build. Disabling buildkit for docker compose (COMPOSE_DOCKER_CLI_BUILD=0) works for a temporary workaround for now.

[wizu]

Same issue with COPY --from=mlocati/php-extension-installer /usr/bin/install-php-extensions /usr/local/bin/ and BuildKit

[max-sixty]

Here's another example:

FROM python:3.8-slim-buster as base

COPY --from=hairyhenderson/gomplate:v3.8.0 /gomplate /usr/local/bin/gomplate
COPY --from=bitnami/kubectl:1.20 /opt/bitnami/kubectl/bin/kubectl /usr/local/bin/kubectl
COPY --from=argoproj/argocli:v2.12.8 /bin/argo /usr/local/bin/argo

This seems to work with two but not three of them (!). As above, only on buildkit.

~And it works if those images are pulled separately prior to building.~ If those images are pulled prior to building, the build reports as successful, but some of the files are not copied in. The inconsistent silent failure is not ideal...

[summera]

Also experiencing this issue with buildkit. Without buildkit, builds fine.

[matti]

when will this fix land in docker for mac?

[matti]

docker for mac update check says that 3.1.0 is the latest, but the latest is 3.2.0

[DRoet]

Docker for mac 3.2.0 has docker engine v20.10.3, the fix for this issue is included with docker v20.10.4 so you'll need to wait until the next release.

[newtorob]

I'm on 20.10.5 on mac and still running into this issue.

[carlydf]

I'm on 20.10.5 on mac and still running into this issue.

same here

[DRoet]

Seems to be resolved for me on both Windows/Linux. Maybe create a new issue with an example if you are still having issues.

[alexdruhet]

I'm on 20.10.5 on mac and still running into this issue.

I'm experiencing the exact same issue

[Plasma]

A brand new VS solution today breaks with "not found" in the path.

[emma-campbell]

I'm on 20.10.5 on mac and still running into this issue.

I'm in the same boat -- running into the same issue.

[RedactedProfile]

On Windows, 20.10.5 and encountering this issue

[barkurguru]

Hello

I am also facing the same issue here.. Is there any fix for this please..

---

> [2/2] COPY ./target/employee-jdbc-0.0.1-SNAPSHOT.jar .:

failed to compute cache key: "/target/employee-jdbc-0.0.1-SNAPSHOT.jar" not found: not found

Thanks
Guru

[yogevyuval]

@thaJeztah I see this issue is closed but many people are still expiriencing the issue. It is because the fix is not rolled out yet, or is it because the problem is still not solved?

[Plasma]

I'm not sure if this fixes it for you: https://stackoverflow.com/a/66611529/554037

Try running:

docker build -f PROJECT_DIR\Dockerfile .

[jonyhy96]

Disable buildkit works for me.
Try

DOCKER_BUILDKIT=0 docker build ...

[thehim]

I'm using version 20.10.5 of the Docker Engine on a Windows machine and I'm seeing this error using a very basic Dockerfile. Any updates?

[MarcoMartins86]

Disable buildkit works for me.
Try

DOCKER_BUILDKIT=0 docker build ...

This flag helped me to get the real error and then I solved it from there when using docker-compose. With the original error, I would not reach this fast to the cause of it.
With flag (improved error):

COPY ../../../server/src/test/resources/elasticsearch/agents/agents_mapping.json /tmp/agents_mapping.json
COPY failed: forbidden path outside the build context: ../../../server/src/test/resources/elasticsearch/agents/agents_mapping.json ()

Without flag (original error):

COPY ../../../server/src/test/resources/elasticsearch/agents/agents_mapping.json /tmp/agents_mapping.json:
failed to compute cache key: "/server/src/test/resources/elasticsearch/agents/agents_mapping.json" not found: not found

[uplinkgit]

Windows 20.10.5 users, here is how I fixed it:

1. Locate daemon.json (should be in c:\Users\CURRENT_USER\.docker folder).
2. Add the following property: "features": {"buildkit": false}. It should look something like:

{
  "registry-mirrors": [],
  "insecure-registries": [],
  "debug": true,
  "experimental": false,
  "features": {"buildkit": false}
}

Cheers.

[BrendonW]

It looks like Docker for [Mac, Windows] have some crippling regressions in buildkit. As far as I can tell, they are using a build version greater than this fix and also #1159 which ends up making it very difficult to build anything non-trivial using caching.

[gwynnarth]

Docker Desktop on Mac v3.3.0 (62916), Engine: 20.10.5 still has this issue.
Only thing that helps is disabling Buildkit... This issue should be reopened, IMHO.