Argo-cd: Since v1.7.0-rc1 volumeClaimTemplates are out of sync

[niiku]

Describe the bug

Since v1.7.0-rc1 volumeClaimTemplates in StatefulSets are out-of-sync.

To Reproduce

Deploy a StatefulSet using volumeClaimTemplates:

  volumeClaimTemplates:
    - metadata:
        name: data
      spec:
        accessModes:
          - ReadWriteOnce
        resources:
          requests:
            storage: 8Gi

The k8s adds

    - apiVersion: v1
      kind: PersistentVolumeClaim

which is detected as diff.

Expected behavior
Argo CD ignores apiVersion and kind in volumeClaimTemplates

Screenshots

Version
ArgoCD Version: v1.7.0-rc1
Kubernetes Version: v1.18.3
OKD Version: 4.5.0-0.okd-2020-07-14-153706-ga

v1.7.0-rc1+4728412

Logs

time="2020-08-19T21:48:28Z" level=info msg="Refreshing app status (hard refresh requested), level (2)" application=harbor
time="2020-08-19T21:48:28Z" level=info msg="Ignore status for CustomResourceDefinitions"
time="2020-08-19T21:48:28Z" level=info msg="Comparing app state (cluster: https://kubernetes.default.svc, namespace: harbor)" application=harbor

(no other relevant log)

Edit:

It seems like other fields are affected with new diffing issues as well, like when creationTimestamp: null is in Git (seen at a ServiceAccount - ignored in v1.6.2), or when app.conf: \|+ (diff says app.conf: |) is used in a e.g. a ConfigMap

[niiku]

@alexmt There are other diff issues with v1.7.0 which were not present in v1.6.2:

[alexmt]

Thank you, @niiku !

Can you please provide more details about resources that have issues? The first one looks like ConfigMap, right?
What are the group/kind of the second resource?

[niiku]

Sorry for the bad screenshots, @alexmt:
here's the yaml:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: iptables-config
  creationTimestamp: null

apiVersion: v1
kind: ConfigMap
metadata:
  name: harbor-core
  labels:
    app.kubernetes.io/name: harbor
    helm.sh/chart: harbor-6.0.8

    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: core
data:
  app.conf: |+
    appname = Harbor
    runmode = dev
    enablegzip = true

    [dev]
    httpport = 8080

And if you want to have a look yourself:
https://argocd.baloise.dev/applications/iptables-config?node=%2FServiceAccount%2Fiptables-config%2Fiptables-config%2F0
https://argocd.baloise.dev/applications/harbor?node=%2FConfigMap%2Fharbor%2Fharbor-core%2F0

[alexmt]

Thanks for sharing YAMLs and link. That is super useful!

I've found an explanation for the first diff. The current behavior is technically correct: creationTimestamp field is set to null in Git so it is correctly reported as different.

However, this is a breaking change - before 1.7 Argo CD used to exclude creationTimestamp field from diffing. We dropped in because it is no longer needed with new diffing implementation. I think for backward compatibility, we should keep ignoring creationTimestamp. Will work on it.

Could not reproduce ConfigMap issue. Trying on K8S v1.18.3

[alexmt]

Reproduced the ConfigMap issue. Looks like it is related to kubectl version upgrade or to this PR https://github.com/argoproj/gitops-engine/pull/62. Argo v1.6.0 does drop - from |- but Argo v1.7.0 drops.

So the workaround is to sync ConfigMap using Argo CD v1.7.0 once. The new behavior is compatible with kubectl, so probably we need to document is as a behavior change in v1.7 upgrade notes.

[alexmt]

Ticket and fix for creationTimestamp issue:
https://github.com/argoproj/argo-cd/issues/4157
https://github.com/argoproj/gitops-engine/pull/128

[niiku]

Thanks for reacting that quickly! I can confirm that the ConfigMap issue disappears after a sync is triggered.
creationTimestamp: null is given when resources are created using kubectl create so it's probably common to have these in Git as well.