Skaffold: Multiple or per artifact tagPolicy's

[bryanlarsen]

Use Case

We're developing a service that creates Kubernetes Job's. Both the service and the job are under very active development.

Both the image for the service and the image for the job are listed in skaffold's "artifacts" section. Only the service artifact is referenced in the k8s manifests, the manifest for all jobs are dynamically generated by the service

If tagPolicy is set to gitCommit, then skaffold does a great job of keeping the service deployment up to date, rolling the deployment whenever any source changes.

If tagPolicy is set to envTemplate: "{{ IMAGE_NAME }}:latest", then skaffold does a great job of keeping the job image built. Whenever we ask the service to start a job, it starts the job running an up to date image.

The Ask

I can think of several different ways to solve my use case:

• allow multiple tagPolicy's to be set. If a user has set multiple tagPolicy's, apply multiple tags to the artifact. You'd may need some magic or an explicit flag to determine which tagPolicy to use in the manifests.
• additionally tag all artifacts with {{ IMAGE_NAME }}:latest to make it easier for us and for anybody other sorts of ad-hoc stuff. (perhaps behind a flag).
• allow tagPolicy to be set per artifact as well as per profile
• some sort of mechanism so that the service can determine what tag skaffold has created for the images it builds

Information

• Skaffold version: version 0.18.0
• Operating system: Ubuntu 18.04
• Contents of skaffold.yaml:

apiVersion: skaffold/v1alpha5
kind: Config
build:
  artifacts:
    - image: app
      docker:
        dockerfile: app/Dockerfile
    - image: files
      docker:
        dockerfile: files/Dockerfile
    - image: projects
      docker:
        dockerfile: projects/Dockerfile
    - image: users
      docker:
        dockerfile: users/Dockerfile
    - image: worker
      docker:
        dockerfile: worker/Dockerfile

profiles:
  - name: dev-shared
    build:
      artifacts:
        - image: dev
          docker:
            dockerfile: docker/dev-shared/Dockerfile

    deploy:
      kustomize:
        path: skaffold/dev-shared

  - name: pr
    build:
      tagPolicy:
        envTemplate:
          template: '{{.DOCKER_REGISTRY}}/threekit-com/{{.IMAGE_NAME}}:{{.VERSION}}'
    deploy:
      kustomize:
        path: skaffold/pr

  - name: dev-full
    deploy:
      kustomize:
        path: skaffold/dev-full

[nkubala]

Having multiple tags per artifact I think would be confusing, because Skaffold would have to choose which one to use in the manifests. I don't necessarily hate having a per-artifact tagger though: if no tag policy is set for an artifact, it could fall back to the top-level tag policy in the build stanza, and of course if that one isn't set we'd use the default.

[bryanlarsen]

@nkubala you already appear to generate multiple tags per artifact. For example, my latest container got tagged with both "a593b5ad6a6362a5cc19907a7a79ec14:latest" and "dev:20cea37-dirty-a2b9536". I'm not sure what the former tag is and why you've created that tag, but it's obviously not causing confusion in your codebase.

But as you mentioned, a per-artifact tagger is probably cleaner.

[gottfrois]

I would also be interested in tag policy per artifacts. Our use case is using multiple docker stages, I would like to tag each stages different to pull them later on new builds to benefit from them as cache:

apiVersion: skaffold/v1beta2
kind: Config
build:
  artifacts:
  - image: gottfrois/ruby-app-example
    docker:
      target: bundler
      cacheFrom:
      - gottfrois/ruby-app-example:bundler-{{.BRANCH}}
      - gottfrois/ruby-app-example:bundler
  - image: gottfrois/ruby-app-example
    docker:
      target: runner
      cacheFrom:
      - gottfrois/ruby-app-example:bundler-{{.BRANCH}}
  - image: gottfrois/ruby-app-example
    docker:
      target: release
      buildArgs:
        env: "production"
        revision: "{{.REVISION}}"
        branch: "{{.BRANCH}}"
      cacheFrom:
      - gottfrois/ruby-app-example:bundler-{{.BRANCH}}
      - gottfrois/ruby-app-example:runner-{{.BRANCH}}
      - gottfrois/ruby-app-example:{{.REVISION}}

I would like to create a tag with the branch env variable to be pulled on next build and used as cache to speed up next docker build.
My Dockerfile uses 3 different docker stages bundler, runner and release which is the final stage.
It would be great to be able to tag each stage accordingly and push them under the same repo.

[torstenwalter]

If multiple tags per image would be possible one could have a release provile which adds more specific tags and a "latest" tag at the same time.

That's quite common for images to have multiple tags. For example the tags for debian currently all point to the same image:

• jessie
• jessie-20190122
• 8.11
• 8

I understand that this cause trouble, because you have to decide which tag to use for deployment. If multiple tags are present one could be marked as default. If none has the default flag then the build could fail.

[siwyd]

I would find it very handy to be able to push multiple tags: the git commit, a sha256 tag and a tag with the branch name (I can get that from BRANCH_NAME in Jenkins for instance). That makes it easy to locate an artifact in a registry in a multitude of ways. When it comes to deploying, personally I always want the sha256 to be used so I'm 100% sure of which artifact is actually up and running at any given moment. So to me it makes sense to always use sha256 as the deployment tag.

[balopat]

Thank you for sharing. I think this is reasonable and we could start looking into it in the next coming quarter.