Kaniko: COPY command in Dockerfile does not copy files in directory with dot at start

[Vrtak-CZ]

COPY command in Dockerfile does not copy files in directory with dot at start. For example files in node_modules/.bin/ eg node_modules/.bin/babel.

I'm using gcr.io/kaniko-project/executor:debug image (gcr.io/kaniko-project/executor@sha256:3998f220d46ae8e7eccc17cc13cddd5b82bee2087510910477563a844097d5b1).

[priyawadhwa]

Hey @Vrtak-CZ, I tried a simple Dockerfile that tries to copy over one dot file (.foo) and kaniko was able to build it successfully.

$ ls -a foo
.  ..  bar  .foo

$ cat Dockerfile 
FROM scratch
COPY foo foo

$ docker run -v $HOME/.config/gcloud:/root/.config/gcloud -v $(pwd):/workspace gcr.io/kaniko-project/executor@sha256:3998f220d46ae8e7eccc17cc13cddd5b82bee2087510910477563a844097d5b1 -d gcr.io/priya-wadhwa/test:test
time="2018-08-31T00:45:13Z" level=info msg="appending to multi args gcr.io/priya-wadhwa/test:test"
time="2018-08-31T00:45:13Z" level=info msg="No base image, nothing to extract"
time="2018-08-31T00:45:13Z" level=info msg="Mounted directories: [/kaniko /var/run /proc /dev /dev/pts /sys /sys/fs/cgroup /sys/fs/cgroup/systemd /sys/fs/cgroup/net_cls,net_prio /sys/fs/cgroup/cpu,cpuacct /sys/fs/cgroup/memory /sys/fs/cgroup/perf_event /sys/fs/cgroup/cpuset /sys/fs/cgroup/blkio /sys/fs/cgroup/freezer /sys/fs/cgroup/pids /sys/fs/cgroup/devices /dev/mqueue /workspace /etc/resolv.conf /etc/hostname /etc/hosts /dev/shm /root/.config/gcloud /proc/asound /proc/bus /proc/fs /proc/irq /proc/sys /proc/sysrq-trigger /proc/kcore /proc/keys /proc/timer_list /proc/sched_debug /sys/firmware]"
time="2018-08-31T00:45:13Z" level=info msg="Taking snapshot of full filesystem..."
time="2018-08-31T00:45:14Z" level=info msg="cmd: copy [foo]"
time="2018-08-31T00:45:14Z" level=info msg="dest: foo"
time="2018-08-31T00:45:14Z" level=info msg="Creating directory /foo"
time="2018-08-31T00:45:14Z" level=info msg="Copying file /workspace/foo/.foo to /foo/.foo"
time="2018-08-31T00:45:14Z" level=info msg="Copying file /workspace/foo/bar to /foo/bar"
time="2018-08-31T00:45:14Z" level=info msg="Taking snapshot of files [/foo/ /foo/.foo /foo/bar]..."
2018/08/31 00:45:17 pushed blob sha256:565fa8740b7fbf9b33e25ecb9be5593a3fd2c02d2f0722765d204ae36fcac54e
2018/08/31 00:45:17 pushed blob sha256:b741d4afbd1062c2e5a4153e2c16f95ea7d9b0795286afc759652fe53b1f047a
2018/08/31 00:45:17 gcr.io/priya-wadhwa/test:test: digest: sha256:5add7a75688274b35ee4f9a4491e5c13a2fba986a60f4584bfd0d01c95aeb522 size: 424

Could you provide the Dockerfile you're trying to build and the command you're using to build it?

[Vrtak-CZ]

the problem is not with .file but with .directory/files

[Vrtak-CZ]

I'm trying to debug it more but i'm not able to run ls in kaniko exacutor:debug image.

ls: can't open '.': Value too large for defined data type

[Vrtak-CZ]

a made some debugging and problem is not with file in dot started directory but with symlinks

ilay:kaniko:% ls -lah foo
total 8
drwxr-xr-x  4 vrtak-cz  staff   128B Aug 31 17:34 .
drwxr-xr-x  4 vrtak-cz  staff   128B Aug 31 15:59 ..
drwxr-xr-x  3 vrtak-cz  staff    96B Aug 31 17:36 .bar
-rw-r--r--  1 vrtak-cz  staff     5B Aug 31 15:58 baz

ilay:kaniko:% ls -lah foo/.bar
total 0
drwxr-xr-x  3 vrtak-cz  staff    96B Aug 31 17:36 .
drwxr-xr-x  4 vrtak-cz  staff   128B Aug 31 17:34 ..
lrwxr-xr-x  1 vrtak-cz  staff     6B Aug 31 17:36 baz -> ../baz

ilay:kaniko:% cat foo/.bar/baz
test

ilay:kaniko:% cat Dockerfile
FROM scratch
COPY foo foo

ilay:kaniko:% docker run -v $(pwd):/workspace gcr.io/kaniko-project/executor@sha256:3998f220d46ae8e7eccc17cc13cddd5b82bee2087510910477563a844097d5b1 --no-push
time="2018-08-31T15:36:50Z" level=info msg="No base image, nothing to extract"
time="2018-08-31T15:36:50Z" level=info msg="Mounted directories: [/kaniko /var/run /proc /dev /dev/pts /sys /sys/fs/cgroup /sys/fs/cgroup/cpuset /sys/fs/cgroup/cpu /sys/fs/cgroup/cpuacct /sys/fs/cgroup/blkio /sys/fs/cgroup/memory /sys/fs/cgroup/devices /sys/fs/cgroup/freezer /sys/fs/cgroup/net_cls /sys/fs/cgroup/perf_event /sys/fs/cgroup/net_prio /sys/fs/cgroup/hugetlb /sys/fs/cgroup/pids /sys/fs/cgroup/systemd /dev/mqueue /workspace /etc/resolv.conf /etc/hostname /etc/hosts /dev/shm /proc/bus /proc/fs /proc/irq /proc/sys /proc/sysrq-trigger /proc/acpi /proc/kcore /proc/keys /proc/timer_list /proc/sched_debug /sys/firmware]"
time="2018-08-31T15:36:50Z" level=info msg="Taking snapshot of full filesystem..."
time="2018-08-31T15:36:51Z" level=info msg="cmd: copy [foo]"
time="2018-08-31T15:36:51Z" level=info msg="dest: foo"
time="2018-08-31T15:36:51Z" level=info msg="Creating directory /foo"
time="2018-08-31T15:36:51Z" level=info msg="Creating directory /foo/.bar"
time="2018-08-31T15:36:51Z" level=info msg="Copying file /workspace/foo/baz to /foo/baz"
time="2018-08-31T15:36:51Z" level=info msg="Taking snapshot of files [/foo/ /foo/.bar /foo/.bar/baz /foo/baz]..."
time="2018-08-31T15:36:51Z" level=info msg="Skipping push to container registry due to --no-push flag"

[Jukie]

Looks like this is functioning properly with latest debug image, should this be closed?

$ cat foo/.baz/buzz
hello
$ cat Dockerfile
FROM alpine
COPY foo foo
RUN ls -alh foo/.baz \
    && cat foo/.baz/buzz
$  docker run -v $(pwd):/workspace gcr.io/kaniko-project/executor:debug --no-push
INFO[0000] Resolved base name alpine to alpine
INFO[0000] Resolved base name alpine to alpine
INFO[0000] Downloading base image alpine
2019/05/30 02:33:22 No matching credentials were found, falling back on anonymous
INFO[0000] Error while retrieving image from cache: getting file info: stat /cache/sha256:bf1684a6e3676389ec861c602e97f27b03f14178e5bc3f70dce198f9f160cce9: no such file or directory
INFO[0000] Downloading base image alpine
2019/05/30 02:33:22 No matching credentials were found, falling back on anonymous
INFO[0000] Built cross stage deps: map[]
INFO[0000] Downloading base image alpine
2019/05/30 02:33:22 No matching credentials were found, falling back on anonymous
INFO[0000] Error while retrieving image from cache: getting file info: stat /cache/sha256:bf1684a6e3676389ec861c602e97f27b03f14178e5bc3f70dce198f9f160cce9: no such file or directory
INFO[0000] Downloading base image alpine
2019/05/30 02:33:23 No matching credentials were found, falling back on anonymous
INFO[0001] Unpacking rootfs as cmd RUN ls -alh foo/.baz     && cat foo/.baz/buzz requires it.
INFO[0001] Taking snapshot of full filesystem...
INFO[0001] Using files from context: [/workspace/foo]
INFO[0001] COPY foo foo
INFO[0001] Taking snapshot of files...
INFO[0001] RUN ls -alh foo/.baz     && cat foo/.baz/buzz
INFO[0001] cmd: /bin/sh
INFO[0001] args: [-c ls -alh foo/.baz     && cat foo/.baz/buzz]
total 12
drwxr-xr-x    2 root     root        4.0K May 30 02:33 .
drwxr-xr-x    3 root     root        4.0K May 30 02:33 ..
-rw-r--r--    1 root     root           6 May 30 02:33 buzz
hello
INFO[0001] Taking snapshot of full filesystem...
INFO[0001] No files were changed, appending empty layer to config. No layer added to image.
INFO[0001] Skipping push to container registry due to --no-push flag