Buildx: docker buildx bake --print does not output default targets

[Puneeth-n]

I have a simple docker-compose file with two services having build properties.

docker buildx bake builds both the images. However,

docker buildx bake --print prints the equivalent docker-bake.json except the default targets

[tonistiigi]

Sorry, I don't understand this report.

Please follow up with source files, commands you are running, expected results and actual results.

[Puneeth-n]

Sorry for not being clear.

So, I have a docker-compose.yml file:

# vim: set ts=2 sw=2 sts=2 et :

version: '3.7'

services:
  addon:
    build:
      context: .
      dockerfile: ./Dockerfile
      args:
        CT_ECR: foo
        CT_TAG: bar
    image: ct-addon:bar
    environment:
      - NODE_ENV=test
      - AWS_ACCESS_KEY_ID=dummy
      - AWS_SECRET_ACCESS_KEY=dummy
    command: tail -f /dev/null

  aws:
    build:
      context: ../../../tools/mock_servers/fake_aws
      dockerfile: ./Dockerfile
      args:
        CT_ECR: foo
        CT_TAG: bar
    image: ct-fake-aws:bar

I want convert this docker-compose.yml file into a docker-bake.json file. so I ran:

DOCKER_CLI_EXPERIMENTAL=enabled docker buildx bake --print > docker-bake.json

The generated docker-bake.json file was:

{
   "target": {
      "addon": {
         "context": ".",
         "dockerfile": "./Dockerfile",
         "args": {
            "CT_ECR": "foo",
            "CT_TAG": "bar"
         },
         "tags": [
            "ct-addon:bar"
         ]
      },
      "aws": {
         "context": "../../../tools/mock_servers/fake_aws",
         "dockerfile": "./Dockerfile",
         "args": {
            "CT_ECR": "foo",
            "CT_TAG": "bar"
         },
         "tags": [
            "ct-fake-aws:bar"
         ]
      }
   }
}

DOCKER_CLI_EXPERIMENTAL=enabled docker buildx bake -f docker-bake.json

and I got an error failed to find target default

I know I need to specify group attribute in the docker-bake.json for it to work. I would be cool, if the --print command also somehow spat out the default group.

[Puneeth-n]

Now come to think of it, https://github.com/docker/buildx#--print specifies that it should be used to display build targets.

One of the adavantages that docker-bake.json has over docker-compose.yml is that, we can tag a build with multiple tags using the --set command.

[tonistiigi]

Yes, this was not the expected behavior of --print that is used to show the current active configuration, not to generate a build file. To make the JSON file usable you would need to add "group": {"default": ["addon", "aws"]} to it.

This is an interesting use-case though. It might make sense for us to implicitly create a default group containing all targets in none was specified.

[Puneeth-n]

Yes. We have lots of docker-compose.yml files that we use to build and runs tests. However, docker-compose.yml wasn't actually made to builld docker images. Now, we have the bake command which lets us build docker images efficiently. I am just afraid that we end up with tons of docker related files.

Since the bake --print command is already intelligent enough to extract build related info from docker-compose.yml files, may be, just may be with default targets, we do not have to end up maintaining one file.

[FernandoMiguel]

FYI docker-compose will support buildkit from the next release.
Tho I still prefer bake

[FernandoMiguel]

One question? Why convert from compose to json?
Compose files can be used just fine with bake (minus the context path).
In my scenario, I created simpler overrides for compose to change the path
For example
https://github.com/FernandoMiguel/BuildKit/blob/master/docker-compose.stage-2.yml
https://github.com/FernandoMiguel/BuildKit/blob/2e2269a742782bb45fa2574a1c151447d8a47243/.github/workflows/build.yml#L200

[Puneeth-n]

FYI docker-compose will support buildkit from the next release.
Tho I still prefer bake

I am aware of it :) But docker-compose build doesn't allow you to tag your built images with multiple tags. We tag all our docker images with commit_id, branch_name and GitHub release when built on master. Currently we use docker-compose to build all our docker images with commit_id but then outside this process, we tag our docker images additionally.

[FernandoMiguel]

Currently we use docker-compose to build all our docker images with commit_id but then outside this process, we tag our docker images additionally.

i'm familiar with that process :)

so you create an HCL to use with your compose?
https://github.com/FernandoMiguel/BuildKit/blob/master/buildx.hcl

[Puneeth-n]

Currently we use docker-compose to build all our docker images with commit_id but then outside this process, we tag our docker images additionally.

i'm familiar with that process :)

so you create an HCL to use with your compose?
https://github.com/FernandoMiguel/BuildKit/blob/master/buildx.hcl

Interesting :) Thanks for the info :D BTW awesome job with your repo :) https://github.com/FernandoMiguel/BuildKit

[FernandoMiguel]

thanks @Puneeth-n
been documenting most of what i learn, and create examples and samples to link to other people with the same questions

[jer-k]

Another reason this could be useful is that the docker compose specification does not currently support a cache_to field. If you wanted to be able to take advantage of using both cache-to and cache-from, say in a CI platform, it could be advantageous to convert the docker-compose.yml -> docker-bake.json and add in the caching stanzas.

I did however discover that you can do --set target.cache-to=type=registry,ref=<ref> so maybe a possible alternative is to programmatically get the targets and add the caching through --set to the command, rather than the json file.

[AntonDobshikov]

And the working json syntax is:
{ "group": { "default": { "targets": ["addon", "aws"] } }, "target": { "addon": { "context": ".", ...