Compose: Support for docker build --secret for build-time docker secrets.

[mikhail-khodorovskiy]

Is your feature request related to a problem? Please describe.
https://medium.com/@tonistiigi/build-secrets-and-ssh-forwarding-in-docker-18-09-ae8161d066 described how to provide secret mounts available at build times.

docker build --secret id=mysite.key,src=path/to/mysite.key .

Can be referenced during docker build:

RUN --mount=type=secret,id=mysite.key,required

docker compose build command: https://docs.docker.com/compose/compose-file/#build currently has no support for the secrets section

Describe the solution you'd like
build command should allow referencing the secrets defined in the same docker-compose file

Describe alternatives you've considered
The alternatives is to use secret-containing file contents as build arguments. The solution is cumbersome since the secret has to be splatted so it can become available for Dockerfile commands which may cause it to become part of a docker layer, i.e it may leak the secret value.

Additional context
Add any other context or screenshots about the feature request here.

[ciaranmcnulty]

This would be really useful - right now we're doing a docker build using the new option and matching the image name to the one that compose will generate, as part of a shell wrapper.

Native support would be great.

[ciaranmcnulty]

As a future-proofing measure it might be interesting to have a key that allows arbitrary CLI flags to be passed to build?

services:
  foo:
    build:
      context: .
      dockerfile: Dockerfile
      target: secure
      flags: '--secret id=mysite.key,src=path/to/mysite.key'

[yinzara]

Agree! SUPER useful especially in a setting where you're building artifacts that have to access private repositories of some sort. Nearly impossible to do without this.

While I would support the idea of a 'flags' attribute, I think because of the specifics of this feature, a real language element would be better. Maybe make a separate feature request specifically for this.

services:
  foo:
    build:
      context: .
      dockerfile: Dockerfile
      target: secure
      secrets:
        -  id: mysite.key
           src: path/to/mysite.key

or possibly assuming there will never be other attributes to the command:

services:
  foo:
    build:
      context: .
      dockerfile: Dockerfile
      target: secure
      secrets:
        'mysite.key': path/to/mysite.key

[ciaranmcnulty]

@yinzara flags attribute was to address next time this happens - a specific syntax for secrets would be best

We should also support:

services:
  foo:
    build:
        # ...
        ssh: default

[Legogris]

Just supporting --ssh for docker-compose run would be great.

[chris-crone]

Thanks for the suggestion! This requires that we support BuildKit in the Python SDK, see the issue here.

[mattsrobot]

Has this ticket moved on, is there a way to use ssh secrets in docker compose?

[chris-crone]

@mattsrobot There is now a PR to add BuildKit support to Docker Compose (see: https://github.com/docker/compose/pull/6865).

The approach of the PR is to exec out to the Docker CLI for build so we could pass flags there.

[Teyras]

@chris-crone I'm trying to do that, but I don't see how I could pass flags to the CLI from docker-compose.yml. Am I missing something?

[antiBaconMachine]

👋 I've taken a stab at this. I actually didn't find this issue until it came time to PR so I'm only reading suggestions now. Happily, I think what I've done is broadly in line with this discussion. I actually really like @ciaranmcnulty idea for a more general purpose flags array, particularly as this feature is still marked experimental. If people prefer that can implement. Anyway let me know what you all think. https://github.com/docker/compose/pull/7046

[itamarst]

Until the relevant PR is merged and released, it's possible as a workaround to have a Dockerfile which uses build args to get secrets during Compose-based local development, and BuildKit secrets to build the production image so you don't leak those secrets. Basically you expect the secret to be in an env variable, and in Compose you set it with a build arg, and for production builds you have a script that sets the env variable from the BuildKit-mounted secrets file.

I documented all this here: https://pythonspeed.com/articles/build-secrets-docker-compose/

[Maks3w]

Until the relevant PR is merged and released, it's possible as a workaround to have a Dockerfile which uses build args to get secrets during Compose-based local development, and BuildKit secrets to build the production image so you don't leak those secrets. Basically you expect the secret to be in an env variable, and in Compose you set it with a build arg, and for production builds you have a script that sets the env variable from the BuildKit-mounted secrets file.

I documented all this here: https://pythonspeed.com/articles/build-secrets-docker-compose/

Sorry but build time secrets are working with current version.

services:
  my-application:
    secrets:
      - my-secret

secrets:
  my-secret:
    file: <path to secret file inside of docker context: ./secrets/secret-filename>

DOCKER_BUILDKIT=1 COMPOSE_DOCKER_CLI_BUILD=1 docker-compose build

[itamarst]

That's a runtime secret.

[greg-caffeine]

@Maks3w, thanks, but I am not able to get docker-compose build to be satisfied with the above example configuration. The above would give me "secret my-secret not found" if my Dockerfile contains a "RUN --mount=type=secret,required,id=my-secret,dst=secret.key do-something". From the docker build flag necessary to pass in the secret and make BuildKit happy ('--secret id=my-secret,src=path/to/my-secret') it seems the "id=" part isn't being satisfied by docker-compose.

[greg-caffeine]

FWIW, work on a related workaround is happening on this in #7296.

[tommie]

It was suggested in https://github.com/docker/compose/pull/7296#issuecomment-626817316 that this needs a compose-spec update. I filed https://github.com/compose-spec/compose-spec/issues/81 to continue the compose-spec discussion there.