Skaffold: Enhance git tagger to tag with latest git tag or other formats

[r2d4]

The git tagger currently doesnt take any parameters but should take some to configure how we resolve image tags from git history. Some options I'd like to see, not sure if they should be exposed as bool parameters or a string enum.

# tags with the full commit sha256 instead of short
type: longCommit

# uses the most recent tag
type: tag

[r2d4]

Happy to help someone design and implement this, it would be a good first issue to tackle.

[stepharr]

@r2d4 I'm interested in taking this on, but would like some clarification please.

• Is the objective to have git tags created from merged commits?
• Do you know of any examples already in existence as reference?

[r2d4]

Awesome! Let me give you some background.

Skaffold has a few different interfaces: builders, taggers, and deployers. Taggers are responsible for taking the build and artifact metadata, and generating a docker image tag from it. That tag will then be used to tag the image in the repository (or daemon) and as part of the kubernetes deployment.

There are different "taggers" today:

• sha256 tags with the sha256 checksum of the built image
• env tags with the value of an environment variable
• and finally, the git tagger, which this issue is about

All of the relevant code is in this file

https://github.com/GoogleContainerTools/skaffold/blob/master/pkg/skaffold/build/tag/git_commit.go

The current implementation generates a tag with the git commit, shortened to 6 characters. If the repository state is "dirty" (there are uncommited changes), then it will append the checksum the output of git diff to the tag (so that there is a new tag if there is new code).

The proposed additions would be modifying this struct

https://github.com/GoogleContainerTools/skaffold/blob/master/pkg/skaffold/schema/v1alpha2/config.go#L60-L62

To look something more like

type GitTagger struct {
  UseFullCommit bool `yaml:useFullCommit,omitempty`
  UseDescribe bool `yaml:useDescribe,omitempty`
...
}

If UseFullCommit is set, it shouldn't trim the commit sha256 and should use the whole commit.

If UseDescribe is set, it should instead generate a tag with the output of git describe. Unfortunately, the library that we're using in place of the git binary doesn't support this yet. One option is to just shell out to git and call git describe with the util.RunCommand function we've made.

@dlorenc might have some other thoughts on what he would want toggled by this.

The goal of this feature is to enable users to generate more semantic docker image tags from their git repository, without having to resort to figuring out these tags themselves and passing it into skaffold as a custom tag.

[r2d4]

@stepharr Github won't let me assign this issue to you as a non-collaborator, so I'll assign it to myself so nobody else picks it up

[errordeveloper]

We have a simple homegrown script that looks like this:

WORKING_SUFFIX=$(if git status --porcelain | grep -qE '^(?:[^?][^ ]|[^ ][^?])\s'; then echo "-WIP"; else echo ""; fi)
BRANCH_PREFIX=$(git rev-parse --abbrev-ref HEAD)
echo "${BRANCH_PREFIX//\//-}-$(git rev-parse --short HEAD)$WORKING_SUFFIX"

Having branch prefix allows you to:
a) visually tell one image apart from another
b) filter out well-known branches, e.g. master or dev

I'd really like to have this kind of formatting as an option.

[errordeveloper]

Also, another strategy that is quite different, but I think it makes a lot of sense is one that LinuxKit uses. It computes (sub-)tree hash and uses that.

So if you see a LinuxKit image <name>:<hash> you can do git show <hash> in the repo from which that image was built and it will resolve to a tree.

The implementation lives in github.com/linuxkit/linuxkit/src/cmd/linuxkit/pkglib.

cc @rn @ijc for comment

[rn]

I'm not sure about the context, but in LinuxKit we default the image tag to the git tree hashs, something like:

git ls-tree --full-tree HEAD -- $(pwd) | awk '{print $3}'

This is basically the content hash of the directory. It has the advantage over the git commit hash that we only need to retag on every commit, just when the content of the directory changes. The downside is that all the dependencies for a image have to be in one directory.

Since it is hard to get the source code revision from the git tree hash, we add the git commit hash to a image label, ie we pass --label org.opencontainers.image.revision=$(REPO_COMMIT) to the docker build.

We append -dirty to the tag if the git repository is has modified files.

For releases we tag images with the git tree hash and a version tag, such as v0.1.

For development, we allow overwriting the tag as well and there are some short cuts to add :dev as a tag.

HTH

[bhack]

@rn Yes it is a good solution and it is for shure to have.
It will not solve problem about out of dir dependencies but probably you can have a buid file where you will handle versioning of the dependency. So something will always change in the directory.

[bhack]

How we will handle directory based hashes with future features like https://github.com/moby/moby/issues/37129?

[bhack]

Can we use something git rev-list --abbrev-commit -1 HEAD $(pwd)?

[corneliusweig]

@r2d4 I'm going to work on this. Do you think it needs a design proposal?

[corneliusweig]

# tags with the full commit sha256 instead of short
type: longCommit

Just nit-picking: git still uses sha-1