Jib: Allow pushing multiple tags to registries

[Techtony96]

Description of the feature:
Currently, in order to push multiple tags (of the same image) you must call jib multiple times. This is common with docker where you have the latest tag and version specific tags. I suggest there be a way to specify multiple tags within jib's configuration.

Expected behavior:

jib {
    to {
        image = ["repo/name:tag", "repo/name:tag2", "repo2/name2:tag3"]
    }
}

Jib would handle this by building the container once and then (preferably in parallel) push the container with the given tags.

[chanseokoh]

We should decided if we would allow multiple tags only or pushing to multiple registries. In the latter case, we may need to think about how auth info could be specified in the build files or through the system properties.

[loosebazooka]

@Techtony96, while personally I think deployment configs should use a specific version instead of latest, you can kind of do this by doing multiple builds and using project properties. If you're not doing that already, it's fairly simple, both gradle and jib are cached enough that this should be pretty fast, the second push especially shouldn't take more than a couple seconds.

jib {
  to {
    image = "registry/repo/image:${project.tag}"
  }
}

$./gradlew jib -Ptag=1.0.0 && ./gradlew jib -Ptag=latest

[Techtony96]

@loosebazooka I do something very similar to that, shown here: https://github.com/DiscordBolt/BoltBot/blob/cacbd52103893ad629c00ff1b066bce7c24a25db/build.gradle#L95-L99

I just feel that being able to push multiple tags would be super helpful in certain situations.

I just the latest keyword because I have a program that automatically updates with the latest image, but only with the same tag. That is why I push both latest and a specific version.

[Hronom]

Same here, need this feature for maven to do something like this:

            <plugin>
                <groupId>com.google.cloud.tools</groupId>
                <artifactId>jib-maven-plugin</artifactId>
                <version>0.9.9</version>
                <executions>
                    <execution>
                        <phase>package</phase>
                        <goals>
                            <goal>build</goal>
                        </goals>
                    </execution>
                </executions>
                <configuration>
                    <from>
                        <image>openjdk:10-jre</image>
                    </from>
                    <to>
                        <image>registry.hub.docker.com/hronom/eureka-service:${git.commit.id}</image>
                        <image>registry.hub.docker.com/hronom/eureka-service:${project.version}</image>
                        <image>registry.hub.docker.com/hronom/eureka-service:latest</image>
                    </to>
                    <container>
                        <mainClass>${start-class}</mainClass>
                        <ports>
                            <port>8081</port>
                        </ports>
                    </container>
                </configuration>
            </plugin>

To don't slowing this feature - make it at least for pushing to one registry.

Strange that Jib doesn't has this feature yet, this is a pretty common use case.
For our project this is a "deal breaker" feature.

[coollog]

Hi @Hronom , we opted to allow the user to use Maven constructs to build to multiple tags instead. You can create executions for the plugin with different image configurations:

<plugin>
  <groupId>com.google.cloud.tools</groupId>
  <artifactId>jib-maven-plugin</artifactId>
  ...
  <executions>
    <execution>
      <id>jib1</id>
      <configuration>
        <to>
          <image>myimage:tag1</image>
        </to>
      </configuration>
    </execution>
    <execution>
      <id>jib2</id>
      <configuration>
        <to>
          <image>myimage:tag2</image>
        </to>
      </configuration>
    </execution>
  </executions>
</plugin>

Then, to build both images:

./mvnw compile jib:build@jib1 jib:build@jib2

Let us know if this works for your case.

[Techtony96]

@coollog Is this an option in the Gradle plugin?

[coollog]

@Techtony96 There is the way you are currently doing it, but you can also define separate tasks to build to different image tags:

task jib1(type: com.google.cloud.tools.jib.gradle.BuildImageTask) {
    jibExtension = project.extensions.getByName('jib')
    doFirst {
        jib.to.image = 'myimage:tag1'
    }
}
task jib2(type: com.google.cloud.tools.jib.gradle.BuildImageTask) {
    jibExtension = project.extensions.getByName('jib')
    doFirst {
        jib.to.image = 'myimage:tag2'
    }
}

Then, you can build to both image tags with gradle jib1 jib2.

It is a bit hacky though. Let me know if this works for you.

[coollog]

@Hronom @Techtony96 Do the configuration I outlined work for your use cases? Feel free to reopen if they don't.

[Hronom]

@coollog This workaround works as temporary solution, but it triggers two builds, so it not convenient.

Is it possible to push image marked with two tags or more without build?

Is this work scheduled for future releases? I'm not understand why it's closed, since it not fully resolved.

Marking builded image with two tags is the popular common case. For example when you release new version of service you mark it by version tag v1.0.1 and latest tag.

[coollog]

@Hronom Does the second build take a significant amount of time? Since Jib caches aggressively, the second run of the jib:build should only need to push the new tag and skips all the steps done by the first run.

[Hronom]

@coollog On my slow internet it tooks 24s vs 12s accordingly. So it's looks like it internet dependent.

From the user side it reasonable to read parameters inside to as array, as far as I understand this work should touch core? Or it's only related to the plugin's level?

Here is the proposed SPI from my side:

            <plugin>
                <groupId>com.google.cloud.tools</groupId>
                <artifactId>jib-maven-plugin</artifactId>
                <version>0.9.9</version>
                <executions>
                    <execution>
                        <phase>package</phase>
                        <goals>
                            <goal>build</goal>
                        </goals>
                    </execution>
                </executions>
                <configuration>
                    <from>
                        <image>openjdk:10-jre</image>
                    </from>
                    <to>
                        <image>registry.hub.docker.com/hronom/eureka-service:${git.commit.id}</image>
                        <image>registry.hub.docker.com/hronom/eureka-service:${project.version}</image>
                        <image>registry.hub.docker.com/hronom/eureka-service:latest</image>
                    </to>
                    <container>
                        <mainClass>${start-class}</mainClass>
                        <ports>
                            <port>8081</port>
                        </ports>
                    </container>
                </configuration>
            </plugin>

[anuraaga]

@coollog I'm using your workaround for multiple tags and while something that operated directly on the registry might be more optimal, it works fine. I did notice that setJibExtension is package-private - if this is the official workaround for multiple tags, does it make sense to make it public? Or is this just temporary?

In Java, to make use of the workaround requires either reflection or t.getAsDynamicObject().setProperty("jibExtension", jib); which isn't terrible but is intentionally skipping around the public API.

[coollog]

@Hronom Although I don't believe Maven can read multiple of the same tag within a tag that is not the plural form, we could support something along the lines of (note the comma-separated tags):

<to>
  <image>registry.hub.docker.com/hronom/eureka-service:${git.commit.id},${project.version},latest</image>
</to>

Or a more organized form:

<to>
  <image>
    <reference>registry.hub.docker.com/hronom/eureka-service</reference>
    <tag1>:${git.commit.id}</tag1>
    <tag2>:${project.version}</tag2>
    <tag3>:latest</tag3>
  </image>
</to>

Note that the <reference>, <tag1>, <tag2>, and <tag3> are user-defined and ignored by Maven. The benefit of these two approaches is that neither actually changes the schema of the plugin configuration.

@anuraaga Yes, we will make JibExtension public.

[Hronom]

@coollog is it possible to do something like this:

<to>
  <image>
    <reference>registry.hub.docker.com/hronom/eureka-service</reference>
    <tags>
        <tag>${git.commit.id}</tag>
        <tag>${project.version}</tag>
        <tag>latest</tag>
    </tags>
  </image>
</to>

Maven has similar constructions:

    <dependencyManagement>
        <dependencies>
            <!-- Spring framework cloud -->
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-dependencies</artifactId>
                <version>Finchley.RELEASE</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
            <dependency>
                <groupId>io.springfox</groupId>
                <artifactId>springfox-swagger2</artifactId>
                <version>2.9.2</version>
            </dependency>
            <dependency>
                <groupId>io.springfox</groupId>
                <artifactId>springfox-swagger-ui</artifactId>
                <version>2.9.2</version>
            </dependency>
        </dependencies>
    </dependencyManagement>

[coollog]

@Hronom Yes, that is possible, although that would require changing the schema of <image> to take a <reference> and <tags> field. I'd be hesitant to do that since it makes the usual case of having a single target image reference require <to><image><reference> rather than just <to><image>.

An alternative is to have <tags> be under the <to> tag:

<to>
  <image>registry.hub.docker.com/hronom/eureka-service</image>
  <tags>
    <tag>${git.commit.id}</tag>
    <tag>${project.version}</tag>
    <tag>latest</tag>
  </tags>
</to>

UPDATE: This is currently being implemented.

Tasks:

• [x] Add additional tags to BuildConfiguration. #963
• [x] Push to the additional tags. #968
• [x] Build to Docker daemon with additional tags. #983
• [x] Make build success message also display the additional tags. #982
• [x] Add to.tags configuration for Gradle. #978
• [x] Add the same for Maven. #1026
• [ ] Add additional tags for build to tarball?

Optimize:

• [x] Parallelize pushing to multiple tags. #974 #1028

[Hronom]

@coollog Looks good for me, since it solves problem

[coollog]

Hi everyone, we have released version 0.9.11 with the multi-tag configuration (<to><tags> for Maven, and to.tags for Gradle).

Example:

<configuration>
  ...
  <to>
    ...
    <tags>
      <tag>another</tag>
      <tag>and-yet-another</tag>
    </tags>
  </to>
</configuration>

@Techtony96 @Hronom @anuraaga @mjrsnyder @jeggy @sdavids @asaikali @itzg @davinkevin @nobecutan, @GuustavoPaiva, @sureshg

[Bwvolleyball]

@Techtony96 There is the way you are currently doing it, but you can also define separate tasks to build to different image tags:

task jib1(type: com.google.cloud.tools.jib.gradle.BuildImageTask) {
    jibExtension = project.extensions.getByName('jib')
    doFirst {
        jib.to.image = 'myimage:tag1'
    }
}
task jib2(type: com.google.cloud.tools.jib.gradle.BuildImageTask) {
    jibExtension = project.extensions.getByName('jib')
    doFirst {
        jib.to.image = 'myimage:tag2'
    }
}

Then, you can build to both image tags with gradle jib1 jib2.

It is a bit hacky though. Let me know if this works for you.

As of gradle 5.6.4 with a kotlin build script, this hack is not working to allow us to publish our container to 2 different image names - i.e. ECR and DockerHub, so I've opened #2299 to formally request multi base-image support.

[chanseokoh]

As of gradle 5.6.4 with a kotlin build script, this hack is not working to allow us to publish our container to 2 different image names - i.e. ECR and DockerHub,

Hmm... really? I can't think of a reason that it doesn't work. Can you elaborate?

[Bwvolleyball]

We're doing this workaround from our build plugin, and each time we just get this error for the second jib execution:

FAILURE: Build failed with an exception.

* What went wrong:
A problem was found with the configuration of task ':awsJib'.
> No value has been specified for property 'jib'.

This is how I was configuring it - I ended up trying to make it identical to the hack posted above, and added the extra property setting:

val awsJib = tasks.create<BuildImageTask>("awsJib") {
    @Suppress("UNUSED_VARIABLE") // used to configure jib for this second task
    val jibExtension = project.extensions.getByName("jib")
    group = "Jib"
    description = "Builds a container image to the aws ecr registry."
    // don't re-configure jib until this task is executed
    doFirst {
        jib!!.to.tags = setOf(imageTag, shaTag, buildTag)
        jib!!.to.image = "id.dkr.ecr.us-west-2.amazonaws.com/repo"
        project.setProperty("jib", jib)
    }
}
tasks.findByName("jib")?.finalizedBy(awsJib.path)

[chanseokoh]

Just in case, to re-iterate, for pushing the same image with multiple _tags_, you should use jib.to.tags.

---

@Bwvolleyball the following code will work. However, I don't know Kotlin, so I'm not sure if this code is ideal.

import com.google.cloud.tools.jib.gradle.BuildImageTask
import com.google.cloud.tools.jib.gradle.JibExtension

...

tasks.create<BuildImageTask>("jib1") {
  setJibExtension(project.extensions.getByName("jib") as JibExtension)
  doFirst {
    jib!!.to.image = "francium25/something"
  }
}
tasks.create<BuildImageTask>("jib2") {
  setJibExtension(project.extensions.getByName("jib") as JibExtension)
  doFirst {
    jib!!.to.image = "francium25/another"
  }
}

./gradlew build jib1 jib2 works for me:

$ ./gradlew build jib1 jib2
...
Built and pushed image as francium25/something
...
Built and pushed image as francium25/another
...
BUILD SUCCESSFUL in 10s
2 actionable tasks: 2 executed

However, the only problem with this trick is that you lose the automatic task dependency configuration and some additional auto-configuration (which may matter for WAR containerization or when using containerizingMode = 'packaged'). In many cases, it'd be sufficient to run the build task first. I am not so much familiar with Gradle, so I am not sure if there is an elegant way to overcome this pitfall.

Otherwise, I'd just go for https://github.com/GoogleContainerTools/jib/issues/802#issuecomment-410831916 Running two Gradle commands shouldn't really be slower, as everything is cached.