Compose: Docker compose 1.10.1 creates invalid tar files during build when permissions do not allow read of any files

[akatrevorjay]

This is rather problematic with volumes for instance. This is a recent regression since 1.9.x.

One of the big gotchas is it thinks the docker daemon just closes the connection, when in reality it's sending it bad data that it never seems to know about.

Kudos.

[shin-]

Hi,

Thank you for the report. I'm a bit confused about what the issue actually is here. Do you mind sharing more details / clarifying? Here's some info that would help:

• OS / version
• Docker Engine version
• What do you mean by "when permissions do not allow read of any files"? Is the user executing docker-compose not able to read the files it's supposed to tar?
• What do you mean by "creates invalid tar files"? Are they corrupted and illegible? Are some files missing?

[akatrevorjay]

Hi.

No problem. It's easy to reproduce, just make a volume owned by root (the default) in a compose project then build it.

[akatrevorjay]

OS/version: Linux of some kind, but that's not a factor here.
Docker engine: Doesn't matter as this is a compose bug, but 1.13.1

Is the user executing docker-compose not able to read the files it's supposed to tar?

Yes.

What do you mean by "creates invalid tar files"? Are they corrupted and illegible? Are some files missing?

It makes invalid tar streams silently and tries to hand them to the daemon as if they're valid, which it rightfully rejects.

[akatrevorjay]

Building web
ERROR: Couldn't connect to Docker daemon at http+docker://localunixsocket - is it running?

If it's at a non-standard location, specify the URL with the DOCKER_HOST environment variable.

Mar 02 19:11:17 nsfdeux dockerd[6193]: time="2017-03-02T19:11:17.660152127-08:00" level=error msg="Handler for POST /v1.22/build returned error: Error processing tar file(archive/tar: invalid tar header): "

[shin-]

I can't seem to reproduce this with Compose 1.11.2:

$ sudo ls -R -l
.:
total 16
drwx------ 3 root    root    4096 Mar  3 12:00 a
drwx------ 3 root    root    4096 Mar  3 12:01 c
-rw-rw-r-- 1 joffrey joffrey   77 Mar  3 12:04 docker-compose.yml
-rw-rw-r-- 1 joffrey joffrey   57 Mar  3 12:04 Dockerfile

./a:
total 4
drwx------ 2 root root 4096 Mar  3 12:00 b

./a/b:
total 4
-rwx------ 1 root root 19 Mar  3 12:00 c.txt

./c:
total 4
drwx------ 2 root root 4096 Mar  3 12:01 d

./c/d:
total 4
-rwx------ 1 root root 41 Mar  3 12:01 xyz.py

$ docker-compose build --no-cache
Building foo
Step 1/3 : FROM alpine:3.4
 ---> 0766572b4bac
Step 2/3 : COPY . /data
 ---> db10f045d62e
Removing intermediate container 1cea5adf291f
Step 3/3 : CMD echo Hello World
 ---> Running in 239216950eba
 ---> 8f41791320f7
Removing intermediate container 239216950eba
Successfully built 8f41791320f7

[akatrevorjay]

What OS? Linux?

On Mar 3, 2017 12:08 PM, "Joffrey F" [e-mail] wrote:

I can't seem to reproduce this with Compose 1.11.2:

$ sudo ls -R -l
.:
total 16
drwx------ 3 root root 4096 Mar 3 12:00 a
drwx------ 3 root root 4096 Mar 3 12:01 c
-rw-rw-r-- 1 joffrey joffrey 77 Mar 3 12:04 docker-compose.yml
-rw-rw-r-- 1 joffrey joffrey 57 Mar 3 12:04 Dockerfile

./a:
total 4
drwx------ 2 root root 4096 Mar 3 12:00 b

./a/b:
total 4
-rwx------ 1 root root 19 Mar 3 12:00 c.txt

./c:
total 4
drwx------ 2 root root 4096 Mar 3 12:01 d

./c/d:
total 4
-rwx------ 1 root root 41 Mar 3 12:01 xyz.py

$ docker-compose build --no-cache
Building foo
Step 1/3 : FROM alpine:3.4
---> 0766572b4bac
Step 2/3 : COPY . /data
---> db10f045d62e
Removing intermediate container 1cea5adf291f
Step 3/3 : CMD echo Hello World
---> Running in 239216950eba
---> 8f41791320f7
Removing intermediate container 239216950eba
Successfully built 8f41791320f7

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/docker/compose/issues/4437#issuecomment-284057461,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABQGImYY2lJztYnn1KfIrkIi5p9uatSOks5riHM9gaJpZM4L5HY1
.

[akatrevorjay]

We can reproduce this reliably, at least on Linux.

[shin-]

Apologies, I lost track of that ticket.

Yes, tested on Linux (Ubuntu 16.04)

[denysvitali]

Experienced the same issue, had to delete my volumes contents to build again. (I think chowning may have worked too)

[akatrevorjay]

AHA! Figured it out. It's due to a socket file. @shin-

[kriestof]

Had same error after upgrading docker(17.05)+docker-compose(1.13).

I had some files with root owner generated by old build on volumes contents. I just had to remove them(this are only some cache and tmp files). I think before I also had to do delete them before build, but the error was more descriptive.

Probably similar to what @denysvitali says.

[HughWarrington]

Can confirm on Ubuntu 16.04, docker-compose 1.16.1, Docker 17.09.0-ce.

I touched a file in the build directory, set it owned by root, and set permissions to 600. Then ran (as a non-root user) docker-compose up --build.

The symptoms were exactly as @akatrevorjay described on 3 March.

[shin-]

Fixed in 1.19.0 (#5496)