Compose: Multi-stage build Intelligence for rebuilding

[jnovack]

Given a Dockerfile

FROM alpine as keys
RUN apk update && \
     apk add --no-cache openssl && \
     rm -rf /var/cache/apk/*
RUN openssl req -x509 -nodes -newkey rsa:4096 -keyout /key.pem -out /certificate.pem -days 720 -subj "/C=US/ST=State/L=City/O=Department/OU=Company/CN=localhost.localdomain"

FROM nginx
EXPOSE 443
COPY --from=keys /*.pem /etc/nginx/
RUN ls /etc/nginx/*.pem

with appropriate docker-compose.yml:

services:
  nginx:
    build: .

I would like the necessary intelligence within docker-compose to invalidate the (any/all) builder container(s) and rebuild the deployment container with cache ("with cache" is modifying the "deployment container", not "builder container").

I can accomplish this in docker with:

$ docker build --target=keys --no-cache .
$ docker build -t nginx-with-keys .

In the above example, keys (or builder container) fully rebuilds, and the deployment build (tagged as nginx-with-keys) only rebuilds from STEP 3 (COPY --from) as expected. Rebuilding from Step 3 is the important piece here.

I do not want to build the entire deployment build from scratch as the deployment container (or other containers in the Dockerfile) could have time intensive tasks and I would want to take advantage of docker caches where appropriate.

In an compose environment I'd have to hit every scalable container, or at a minimum, attempt to derive the COMPOSE_PROJECT_NAME (don't hate me!) and rebuild each container (builders without cache, deployments with cache) manually.

At a minimum, it would need to be able to successfully identify the correct service (in a multi-service docker-compose.yml file) and correct builder container (in a multi-builder Dockerfile)

[shin-]

Once support for the target property is implemented (https://github.com/docker/compose/pull/5011 - target 1.16.0 in August), you should be able to have something like:

services:
  nginx:
     build: .
  keys:
    build:
      context: .
      target: keys
    command: sh -c 'exit 0'

Then run the following command: docker-compose build keys && docker-compose up --build

Am I missing anything?

[jnovack]

Am I missing anything?

Just my gratitude.. which you have now. <3

[obicons]

I am running docker-compose version 1.16.1, build 6d1ac21, and I am unable to use the target build option. When I run docker-compose build, I see the error message:
... contains unsupported option: 'target'

Also, the documentation @ https://docs.docker.com/compose/compose-file/#build does not indicate that there is a target option.

What's the status of this feature?

[shin-]

Make sure you're using a Compose file version that supports it.

[ashic]

What version supports target with docker-compose 1.16.1?

[shin-]

version 2.3

[RRMoelker]

For future reference, the Compose file version should be >2.3 && <3.0 or >3.4

Versions 3.0 through 3.3 do not seem to support the target option.

https://docs.docker.com/compose/compose-file/compose-versioning/#version-23
https://docs.docker.com/compose/compose-file/compose-versioning/#version-34

[jnovack]

(>2.3 && <3.0) || >3.4

FTFY