Compose: Add support for the --squash flag

[adipascu]

See https://github.com/docker/docker/pull/22641

[henryptung]

Probably relevant: https://github.com/docker/compose/issues/4114

[MichaelPereira]

Update: After looking at the code and notably the Docker engine API documentation for the build endpoint, I understand that the request below is not really feasible since any changes to the Engine API requires a corresponding change in the python SDK client.

I think that the scope of this ticket could be widened to all docker build options as described in this comment of #4114 to make it future-proof by automatically supporting all options of docker-build

[ibaralf]

Is this feature in docker-compose yet? I see that docker build --squash is already available on my docker for mac.

If not yet implemented - is there a workaround? I'm trying to avoid having secrets or access keys in the intermediate images when doing a build.

[dnephin]

Multi-stage builds were releases in 17.06 which provide similar functionality to squash but covers a lot more use cases. I think squash may be removed since it never left experimental.

Do multi-stage builds work for your use case?

[joeyhub]

Multi-stage builds squash intermediate images together don't they?

That's not the same as what --squash is meant to do which is to squash the layers within an image.

Each command you run in build creates a layer. If you:

 COPY huge.file
 RUN script huge_file && rm huge.file

That'll create a layer that adds huge file and another layer that removes it again!

You can see this if you look at the history for an image.

The idea of squash is to have only one layer in your image, that represents the difference between it and the image it is created from.

I'm new to docker (few days) and have traditionally been creating build images from hand using things such as unionfs, so I might have some misconceptions about what docker is doing. However in this case, it's definitely leaking. I temporarily put a file into the container during build, then remove it once it's processed. I still see it in the history and find it in the auf diff folder however.

At the moment the only way around that I know of is to do things such as wget or nc your resource and pipe it into a processor. Alternatively it might be possible to mount a ramdisk.

There are other cases where diffs appear to be around which is plain inexplicable. In some cases it appears as if it stores a copy on removal as well.

I think there is some confusion as there are a multitude of issues with docker concerning space efficiency getting conflated with others.

[dnephin]

Here's the issue for removing squash: https://github.com/moby/moby/issues/34565, I forgot to link it earlier.

Multi-stage builds squash intermediate images together don't they?

No, they don't do any squash, but they do allow for you to build an image that only has the layers you want.

In multi-stage builds the image that gets tagged is just the last "stage", so anything that happens in previous stages will not be part of the layers of the final image. This lets you perform all the intermediate operations in one or more stages, and in the final stage you have something like this:

FROM scratch # or alpine
COPY --from=intermediate file /file

which will give you a 1 or 2 layer image, the same as squash.

[adipascu]

@dnephin multi-stage builds fixed my issue, I no longer need squash.
Thanks !

[johnthagen]

There is still interest in moving --squash out of experimental, as explained here. As squash is still useful to use in combination with multi-stage builds: https://github.com/moby/moby/issues/38657

[mskoenz]

The issue with just multistage build (via COPY --from) is that it does not preserve parent settings like USER, ENTRYPOINT, CMD or WORKDIR. Is there a way to get USER/CMD/ENTRYPOINT/WORKDIR --from parent?