Buildkit: Not invalidating cache when renaming a file and COPY wildcard

[kekru]

Hi there, first of all: thanks to this great piece of software!

I think I found a wrong caching behavior when using COPY with a wildcard source and renaming a file.
Tested on Docker Desktop for Windows 3.1.0, Docker Engine version 20.10.2

I have these two files:

run.sh

#!/bin/bash
set -e

docker builder prune -af

echo "Hello World" > content.txt
echo "Local dir before first build"
ls
echo ""
docker build . --progress=plain -t testimage
echo "Content of first image"
docker run --rm -it testimage ls test/

echo ""
echo "#######################"
echo "Local dir before second build"
mv content.txt content2.txt
ls
echo ""

docker build . --progress=plain -t testimage

echo ""
echo "#######################"
echo "Content of second image"
docker run --rm -it testimage ls test/

rm content2.txt

Dockerfile

FROM alpine:3.13.2
RUN mkdir /test
COPY ./* /test/

Now when running DOCKER_BUILDKIT=1 ./run.sh I want the following:
The first build adds "content.txt" to the Dockerfile, the second one renames "content.txt" to "content2.txt".
I expect that Buildkit invalidates the cache in the second build and adds the "content2.txt" correctly, but it reuses the cache from the first build.

This is the output of DOCKER_BUILDKIT=1 ./run.sh

$ DOCKER_BUILDKIT=1 ./run.sh
Deleted build cache objects:
6gnh4bheysec55o46m54yhtbs
wd0aj4eyaqn9yvfheb6akkows
7svl9b3u8et5dqgjx66sqeb2d
3c9wwdy5erg4ybtb6fxrwy2rg
yfhh43xkzfpyo9ixlertf9ezn
ngmd43p08542mrqd7edt3ixs3

Total reclaimed space: 10.7kB
Local dir before first build
content.txt  Dockerfile  run.sh

#1 [internal] load build definition from Dockerfile
#1 sha256:88c2a8dc447a3b6f461bbf0afe55acc70a3223574d4f056f4c4010552593abb2
#1 transferring dockerfile: 88B done
#1 DONE 0.0s

#2 [internal] load .dockerignore
#2 sha256:b7d345d3151619959a0c4f179384fa4e7df776474a7a5811958c7b1d49043e56
#2 transferring context: 2B done
#2 DONE 0.1s

#3 [internal] load metadata for docker.io/library/alpine:3.13.2
#3 sha256:74437c134bdd4eeebefdd4e9840fe80d11b87372b077755a28e5ee354e1befdc
#3 DONE 0.0s

#4 [1/3] FROM docker.io/library/alpine:3.13.2
#4 sha256:363517c98a88a5f639269fcb7cdb12f04b5cc44e5454dc778152adedd469084e
#4 resolve docker.io/library/alpine:3.13.2 done
#4 DONE 0.1s

#6 [internal] load build context
#6 sha256:5d16faa5b3418e443682f9abe7d2c73aa10113310755a537684d73854c74d4e2
#6 transferring context: 706B done
#6 DONE 0.1s

#5 [2/3] RUN mkdir /test
#5 sha256:ed63ec1b034438327d23a957c83f200507a32273d4c28cb9298c05f7a77afd96
#5 DONE 0.3s

#7 [3/3] COPY ./* /test/
#7 sha256:1ccdccc27a3aa7a016f44bfb19b8b088144908c03e76477697779245a3050795
#7 DONE 0.1s

#8 exporting to image
#8 sha256:e8c613e07b0b7ff33893b694f7759a10d42e180f2b4dc349fb57dc6b71dcab00
#8 exporting layers 0.1s done
#8 writing image sha256:c36ed49650e1db21a828cd2e7c349e6664f3cc8c72c89d691e8f0458816dc538 done
#8 naming to docker.io/library/testimage done
#8 DONE 0.1s
Content of first image
Dockerfile   content.txt  run.sh

#######################
Local dir before second build
content2.txt  Dockerfile  run.sh

#1 [internal] load build definition from Dockerfile
#1 sha256:95f58a89fd58d375700dbe4a8e17504bc6ed243b8095464263cea5a4b8cd3b67
#1 transferring dockerfile: 31B done
#1 DONE 0.0s

#2 [internal] load .dockerignore
#2 sha256:c3c952e1aca0d6bf91463f400e52a37c31eca12249363148b3f917aad817f288
#2 transferring context: 2B done
#2 DONE 0.0s

#3 [internal] load metadata for docker.io/library/alpine:3.13.2
#3 sha256:74437c134bdd4eeebefdd4e9840fe80d11b87372b077755a28e5ee354e1befdc
#3 DONE 0.0s

#4 [1/3] FROM docker.io/library/alpine:3.13.2
#4 sha256:363517c98a88a5f639269fcb7cdb12f04b5cc44e5454dc778152adedd469084e
#4 DONE 0.0s

#6 [internal] load build context
#6 sha256:2d30b6ccda83a15c20fe1aa608beded4330bf9bf3509a1fd3e6064909862597e
#6 transferring context: 110B done
#6 DONE 0.0s

#5 [2/3] RUN mkdir /test
#5 sha256:ed63ec1b034438327d23a957c83f200507a32273d4c28cb9298c05f7a77afd96
#5 CACHED

#7 [3/3] COPY ./* /test/
#7 sha256:3b761065f0da406cb5e12846dda1dc28d6593b313353ce8bfce583ccc4a8170d
#7 CACHED

#8 exporting to image
#8 sha256:e8c613e07b0b7ff33893b694f7759a10d42e180f2b4dc349fb57dc6b71dcab00
#8 exporting layers done
#8 writing image sha256:c36ed49650e1db21a828cd2e7c349e6664f3cc8c72c89d691e8f0458816dc538 done
#8 naming to docker.io/library/testimage done
#8 DONE 0.0s

#######################
Content of second image
Dockerfile   content.txt  run.sh

As you see, the second build reuses the cache and it contains "content.txt" and not "content2.txt".

If I do one of the following, then everything works correctly:

• Running without Buildkit DOCKER_BUILDKIT=0 ./run.sh
• or changing the content of "content2.txt"

So the problem occurs if I use COPY with wildcard in combination with renaming a file, that is matched by that wildcard

Maybe this is somehow related to #1853

[tonistiigi]

@kekru Thanks for the report. Patch in #2018