Argo-cd: Git tags pointing to wrong hash

[pbecotte]

Argocd Version: v1.7.2+c342d3f

Describe the bug

We have a number of deployments pinned to a git tag, and updated by moving the tag (the "Tag Tracking" strategy from the docs). Today, a number of our deployments started failing. We have actually been using ARGOCD_APP_REVISION in the manifests- when we build an image, we tag it with the git commit, and then when the tag is moved to that commit, we stick APP_REVISION in the image name.

We updated to 1.7.2 a couple days ago, and this is no longer working. ARGOCD_APP_REVISION now points to the hash of the _tag_ instead of the commit. This means that our images which are tagged with the commit hash (before the tag is even created) are no longer being referenced. To understand that this is an issue in the engine somewhere, the link under "status" in the UI no longer works either (getting a github 404 since its trying to go to the tag hash instead of the commit hash)

To Reproduce

Create an application with target_revision = a tag name instead of a branch.

Expected behavior

The hash that comes up in the build environment ARGOCD_APP_REVISION should be the actual commit hash.

[jannfis]

Hi, to my knowledge, Git tags have no unique revision. They are simply pointers to any revision in the tree, unless they are annotated, in which case they will get their own revision - because the annotated tag is committed into the tree (without being recorded in the Git log, tho).

As an example in a freshly initialized repository with one commit and afterwards creating two tags, v0.0.1 is a lightweight tag, while v0.0.2 is an annotated tag (created using git tag -a):

$ git log
commit a81d319a5b262436eb5cda04740d19420b2ba235 (HEAD -> master, tag: v0.0.2, tag: v0.0.1)
Author: jannfis
Date:   Wed Sep 2 10:26:56 2020 +0200

    Initial commit
$ git rev-parse v0.0.1
a81d319a5b262436eb5cda04740d19420b2ba235
$ git rev-parse v0.0.2
b217fb20161144da7118fb668bee1b2f4adc7780

So I think it might be possible that you have used an annotated tag - which then resolves to a different revision, and ultimately leads to the behaviour you have described.

[pbecotte]

yes, it is an annotated tag. But, it has always been annotated tags- the jenkins script that creates them has not changed, but the argocd behavior has.

[abangser]

Argocd Version: v1.7.2+c342d3f

I am seeing this behaviour on v1.6.1+159674e. @pbecotte which version did you upgrade from? I am curious if we can track down if/where this may have been introduced.

[gaganhegde]

Will take a look into this @jessesuen

[pbecotte]

We were on 1.4.2 before the upgrade

[jaideepr97]

Hi @pbecotte
Do you mind sharing an example for how/where you're using ARGOCD_APP_REVISION in your manifests? Might help trace and reproduce the problem

[abangser]

Thanks for looking into this! In my case we have a helm chart and are overriding a value.

E.g. having an Argo Application with:

spec:
  source:
    helm:
      valueFiles:
        - values.yaml
      parameters:
        - name: commit_sha
          value: $ARGOCD_APP_REVISION

and then a values.yaml with:

commit_sha: SET_BY_ARGO

which then gets templated into a deployment manifest as the image tag like:

apiVersion: app/v1
kind: Deployment
spec:
  template:
    spec:
      containers:
        - image: {{ .Values.imageBase }}:{{ .Values.commit_sha }}

[jaideepr97]

Thanks for providing context @abangser, I think I've been able to reproduce the issue locally
working on a fix now