Argo-cd: UI not reflecting state - reduced performance - unable to sync - etc

Created on 12 Aug 2020 · 10Comments · Source: argoproj/argo-cd

If you are trying to resolve an environment-specific issue or have a one-off question about the edge case that does not require a feature then please consider asking a
question in argocd slack channel.

Checklist:

[ x] I've searched in the docs and FAQ for my answer: https://bit.ly/argocd-faq.
[x ] I've included steps to reproduce the bug.
[x ] I've pasted the output of argocd version.

Describe the bug
In the UI it is reporting 'out of sync' but on the cluster all looks well and installed.

To Reproduce

I have had an epic adventure attempting to upgrade cert-manager from 0.15.2 to 0.16.1, while not all of the issues are related to ArgoCD the end point I have reached is this.

I deleted everything cert-manager out of my cluster.
I deleted the app out of ArgoCD.
I deleted/restarted the application controller pod.
I then installed cert-amanger 0.16.1 with the following app:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  annotations:
    argocd.argoproj.io/refresh: normal
    meta.helm.sh/release-name: k8s00-dev-sydney-aws-cluster-addons
    meta.helm.sh/release-namespace: default
  labels:
    app.kubernetes.io/instance: k8s00-dev-sydney-aws-cluster-addons
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: cert-manager
    helm.sh/chart: app-of-apps-0.5.31
  name: k8s00-dev-sydney-aws-cert-manager
  namespace: argocd
spec:
  destination:
    namespace: operations
    server: https://XXXXXXXXX.sk1.ap-southeast-2.eks.amazonaws.com
  project: k8s00-dev-sydney-aws
  source:
    chart: cert-manager
    helm:
      releaseName: cert-manager
      values: |
        global:
          ## Reference to one or more secrets to be used when pulling images
          ## ref: https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/
          ##
          imagePullSecrets: []
          isOpenshift: false
          # - name: "image-pull-secret"

          # Optional priority class to be used for the cert-manager pods
          priorityClassName: ""
          rbac:
            create: true

          podSecurityPolicy:
            enabled: true

          # Set the verbosity of cert-manager. Range of 0 - 6 with 6 being the most verbose.
          logLevel: 1

          leaderElection:
            # Override the namespace used to store the ConfigMap for leader election
            namespace: operations

        installCRDs: true

        replicaCount: 1

        strategy: {}
          # type: RollingUpdate
          # rollingUpdate:
          #   maxSurge: 0
          #   maxUnavailable: 1

        image:
          repository: quay.io/jetstack/cert-manager-controller
          # Override the image tag to deploy by setting this variable.
          # If no value is set, the chart's appVersion will be used.
          # tag: canary
          pullPolicy: IfNotPresent

        # Override the namespace used to store DNS provider credentials etc. for ClusterIssuer
        # resources. By default, the same namespace as cert-manager is deployed within is
        # used. This namespace will not be automatically created by the Helm chart.
        clusterResourceNamespace: ""

        serviceAccount:
          # Specifies whether a service account should be created
          create: true
          # The name of the service account to use.
          # If not set and create is true, a name is generated using the fullname template
          name:
          annotations:
            eks.amazonaws.com/role-arn: arn:aws:iam::XXXXX:role/cert-manager-k8s00-dev-sydney-aws

        # Optional additional arguments
        extraArgs:
          # Use this flag to set a namespace that cert-manager will use to store
          # supporting resources required for each ClusterIssuer (default is kube-system)
          - --cluster-resource-namespace=operations
          # When this flag is enabled, secrets will be automatically removed when the certificate resource is deleted
          # - --enable-certificate-owner-ref=true

        extraEnv: []
        # - name: SOME_VAR
        #   value: 'some value'

        resources: {}
          # requests:
          #   cpu: 10m
          #   memory: 32Mi

        # Pod Security Context
        # ref: https://kubernetes.io/docs/tasks/configure-pod-container/security-context/
        securityContext:
          enabled: true
          fsGroup: 1001
          runAsUser: 1001

        podLabels: {}
        # Optional DNS settings, useful if you have a public and private DNS zone for
        # the same domain on Route 53. What follows is an example of ensuring
        # cert-manager can access an ingress or DNS TXT records at all times.
        # NOTE: This requires Kubernetes 1.10 or `CustomPodDNS` feature gate enabled for
        # the cluster to work.
        # podDnsPolicy: "None"
        # podDnsConfig:
        #   nameservers:
        #     - "1.1.1.1"
        #     - "8.8.8.8"

        nodeSelector: {}

        ingressShim: {}
          # defaultIssuerName: ""
          # defaultIssuerKind: ""
          # defaultIssuerGroup: ""

        prometheus:
          enabled: true
          servicemonitor:
            enabled: false

        # Use these variables to configure the HTTP_PROXY environment variables
        # http_proxy: "http://proxy:8080"
        # http_proxy: "http://proxy:8080"
        # no_proxy: 127.0.0.1,localhost

        # expects input structure as per specification https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.11/#affinity-v1-core
        # for example:
        #   affinity:
        #     nodeAffinity:
        #      requiredDuringSchedulingIgnoredDuringExecution:
        #        nodeSelectorTerms:
        #        - matchExpressions:
        #          - key: foo.bar.com/role
        #            operator: In
        #            values:
        #            - master
        affinity: {}

        # expects input structure as per specification https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.11/#toleration-v1-core
        # for example:
        #   tolerations:
        #   - key: foo.bar.com/role
        #     operator: Equal
        #     value: master
        #     effect: NoSchedule
        tolerations: []

        webhook:
          enabled: true
          replicaCount: 1

          strategy: {}
            # type: RollingUpdate
            # rollingUpdate:
            #   maxSurge: 0
            #   maxUnavailable: 1

          # Optional additional arguments for webhook
          extraArgs: []

          resources: {}
            # requests:
            #   cpu: 10m
            #   memory: 32Mi

          nodeSelector: {}

          affinity: {}

          tolerations: []

          image:
            repository: quay.io/jetstack/cert-manager-webhook
            # Override the image tag to deploy by setting this variable.
            # If no value is set, the chart's appVersion will be used.
            # tag: canary
            pullPolicy: IfNotPresent

          # If true, the apiserver's cabundle will be automatically injected into the
          # webhook's ValidatingWebhookConfiguration resource by the CA injector.
          # in future this will default to false, as the apiserver can use the loopback
          # configuration caBundle to talk to itself in kubernetes 1.11+
          # see https://github.com/kubernetes/kubernetes/pull/62649
          injectAPIServerCA: true

          # The port that the webhook should listen on for requests.
          # In GKE private clusters, by default kubernetes apiservers are allowed to
          # talk to the cluster nodes only on 443 and 10250. so configuring
          # securePort: 10250, will work out of the box without needing to add firewall
          # rules or requiring NET_BIND_SERVICE capabilities to bind port numbers <1000
          securePort: 10250

        cainjector:
          replicaCount: 1

          strategy: {}
            # type: RollingUpdate
            # rollingUpdate:
            #   maxSurge: 0
            #   maxUnavailable: 1

          # Optional additional arguments for cainjector
          extraArgs: []

          resources: {}
            # requests:
            #   cpu: 10m
            #   memory: 32Mi

          nodeSelector: {}

          affinity: {}

          tolerations: []

          image:
            repository: quay.io/jetstack/cert-manager-cainjector
            # Override the image tag to deploy by setting this variable.
            # If no value is set, the chart's appVersion will be used.
            # tag: canary
            pullPolicy: IfNotPresent
    repoURL: https://charts.jetstack.io
    targetRevision: 0.16.1

what I end up is with;

Expected behavior
I expect it to be all green and in sync like my other cluster still on 0.15.2. This was installed with ARgoCD 1.6.1.

Version

argocd-server: v1.6.2+3d1f37b
  BuildDate: 2020-07-31T23:45:07Z
  GitCommit: 3d1f37b0c53f4c75864dc7339e2831c6e6a947e0
  GitTreeState: clean
  GoVersion: go1.14.1
  Compiler: gc
  Platform: linux/amd64
  Ksonnet Version: v0.13.1
  Kustomize Version: {Version:kustomize/v3.6.1 GitCommit:c97fa946d576eb6ed559f17f2ac43b3b5a8d5dbd BuildDate:2020-05-27T20:47:35Z GoOs:linux GoArch:amd64}
  Helm Version: version.BuildInfo{Version:"v3.2.0", GitCommit:"e11b7ce3b12db2941e90399e874513fbd24bcb71", GitTreeState:"clean", GoVersion:"go1.13.10"}
  Kubectl Version: v1.14.0

Logs
here are the logs off the application controller when I added it and then sync'd.

jurgen.weber@Jurgens-MBP ~ $ cat asdf.txt
$ stern argocd-application-con -i "application=k8s00-dev-sydney-aws-cert-manager" -e "application=k8s00-dev-sydney-aws-cert-manager-" -n argocd -s 15m
+ argocd-application-controller-8458ff6b5-25jlr › argocd-application-controller
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:35:12.630868279Z time="2020-08-11T23:35:12Z" level=info msg="Refreshing app status (normal refresh requested), level (2)" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:35:12.636201049Z time="2020-08-11T23:35:12Z" level=info msg="Comparing app state (cluster: https://XXXXXXXXXX.sk1.ap-southeast-2.eks.amazonaws.com, namespace: operations)" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:35:18.478794911Z time="2020-08-11T23:35:18Z" level=info msg="getRepoObjs stats" application=k8s00-dev-sydney-aws-cert-manager build_options_ms=0 helm_ms=0 plugins_ms=1 repo_ms=0 time_ms=5843 unmarshal_ms=388 version_ms=5453
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:35:18.659491443Z time="2020-08-11T23:35:18Z" level=info msg="Updated sync status:  -> OutOfSync" application=k8s00-dev-sydney-aws-cert-manager dest-namespace=operations dest-server="https://XXXXXXXXXX.sk1.ap-southeast-2.eks.amazonaws.com" reason=ResourceUpdated type=Normal
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:35:18.664991409Z time="2020-08-11T23:35:18Z" level=info msg="Updated health status:  -> Missing" application=k8s00-dev-sydney-aws-cert-manager dest-namespace=operations dest-server="https://XXXXXXXXXX.sk1.ap-southeast-2.eks.amazonaws.com" reason=ResourceUpdated type=Normal
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:35:18.685534035Z time="2020-08-11T23:35:18Z" level=info msg="Update successful" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:35:18.685593310Z time="2020-08-11T23:35:18Z" level=info msg="Reconciliation completed" application=k8s00-dev-sydney-aws-cert-manager dedup_ms=0 dest-namespace=operations dest-server="https://XXXXXXXXXX.sk1.ap-southeast-2.eks.amazonaws.com" diff_ms=84 fields.level=2 git_ms=5843 health_ms=0 live_ms=57 settings_ms=2 sync_ms=0 time_ms=6054
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:36:38.041706893Z time="2020-08-11T23:36:38Z" level=info msg="Initialized new operation: {&SyncOperation{Revision:0.16.1,Prune:false,DryRun:false,SyncStrategy:&SyncStrategy{Apply:nil,Hook:&SyncStrategyHook{SyncStrategyApply:SyncStrategyApply{Force:false,},},},Resources:[]SyncOperationResource{},Source:nil,Manifests:[],SyncOptions:[],} {[email protected] false} []}" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:36:38.041762644Z time="2020-08-11T23:36:38Z" level=info msg="Comparing app state (cluster: https://XXXXXXXXXX.sk1.ap-southeast-2.eks.amazonaws.com, namespace: operations)" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:36:38.073328630Z time="2020-08-11T23:36:38Z" level=info msg="getRepoObjs stats" application=k8s00-dev-sydney-aws-cert-manager build_options_ms=0 helm_ms=0 plugins_ms=0 repo_ms=0 time_ms=31 unmarshal_ms=31 version_ms=0
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:36:38.126186159Z time="2020-08-11T23:36:38Z" level=info msg=syncing application=k8s00-dev-sydney-aws-cert-manager skipHooks=false started=false syncId=00001-ItuEG
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:36:45.862138069Z time="2020-08-11T23:36:45Z" level=info msg=tasks application=k8s00-dev-sydney-aws-cert-manager syncId=00001-ItuEG tasks="[Sync/0 resource policy/PodSecurityPolicy:operations/cert-manager nil->obj (,,), Sync/0 resource policy/PodSecurityPolicy:operations/cert-manager-cainjector nil->obj (,,), Sync/0 resource policy/PodSecurityPolicy:operations/cert-manager-webhook nil->obj (,,), Sync/0 resource /ServiceAccount:operations/cert-manager nil->obj (,,), Sync/0 resource /ServiceAccount:operations/cert-manager-cainjector nil->obj (,,), Sync/0 resource /ServiceAccount:operations/cert-manager-webhook nil->obj (,,), Sync/0 resource apiextensions.k8s.io/CustomResourceDefinition:operations/certificaterequests.cert-manager.io nil->obj (,,), Sync/0 resource apiextensions.k8s.io/CustomResourceDefinition:operations/certificates.cert-manager.io nil->obj (,,), Sync/0 resource apiextensions.k8s.io/CustomResourceDefinition:operations/challenges.acme.cert-manager.io nil->obj (,,), Sync/0 resource apiextensions.k8s.io/CustomResourceDefinition:operations/clusterissuers.cert-manager.io nil->obj (,,), Sync/0 resource apiextensions.k8s.io/CustomResourceDefinition:operations/issuers.cert-manager.io nil->obj (,,), Sync/0 resource apiextensions.k8s.io/CustomResourceDefinition:operations/orders.acme.cert-manager.io nil->obj (,,), Sync/0 resource rbac.authorization.k8s.io/ClusterRole:operations/cert-manager-cainjector nil->obj (,,), Sync/0 resource rbac.authorization.k8s.io/ClusterRole:operations/cert-manager-cainjector-psp nil->obj (,,), Sync/0 resource rbac.authorization.k8s.io/ClusterRole:operations/cert-manager-controller-certificates nil->obj (,,), Sync/0 resource rbac.authorization.k8s.io/ClusterRole:operations/cert-manager-controller-challenges nil->obj (,,), Sync/0 resource rbac.authorization.k8s.io/ClusterRole:operations/cert-manager-controller-clusterissuers nil->obj (,,), Sync/0 resource rbac.authorization.k8s.io/ClusterRole:operations/cert-manager-controller-ingress-shim nil->obj (,,), Sync/0 resource rbac.authorization.k8s.io/ClusterRole:operations/cert-manager-controller-issuers nil->obj (,,), Sync/0 resource rbac.authorization.k8s.io/ClusterRole:operations/cert-manager-controller-orders nil->obj (,,), Sync/0 resource rbac.authorization.k8s.io/ClusterRole:operations/cert-manager-edit nil->obj (,,), Sync/0 resource rbac.authorization.k8s.io/ClusterRole:operations/cert-manager-psp nil->obj (,,), Sync/0 resource rbac.authorization.k8s.io/ClusterRole:operations/cert-manager-view nil->obj (,,), Sync/0 resource rbac.authorization.k8s.io/ClusterRole:operations/cert-manager-webhook-psp nil->obj (,,), Sync/0 resource rbac.authorization.k8s.io/ClusterRoleBinding:operations/cert-manager-cainjector nil->obj (,,), Sync/0 resource rbac.authorization.k8s.io/ClusterRoleBinding:operations/cert-manager-cainjector-psp nil->obj (,,), Sync/0 resource rbac.authorization.k8s.io/ClusterRoleBinding:operations/cert-manager-controller-certificates nil->obj (,,), Sync/0 resource rbac.authorization.k8s.io/ClusterRoleBinding:operations/cert-manager-controller-challenges nil->obj (,,), Sync/0 resource rbac.authorization.k8s.io/ClusterRoleBinding:operations/cert-manager-controller-clusterissuers nil->obj (,,), Sync/0 resource rbac.authorization.k8s.io/ClusterRoleBinding:operations/cert-manager-controller-ingress-shim nil->obj (,,), Sync/0 resource rbac.authorization.k8s.io/ClusterRoleBinding:operations/cert-manager-controller-issuers nil->obj (,,), Sync/0 resource rbac.authorization.k8s.io/ClusterRoleBinding:operations/cert-manager-controller-orders nil->obj (,,), Sync/0 resource rbac.authorization.k8s.io/ClusterRoleBinding:operations/cert-manager-psp nil->obj (,,), Sync/0 resource rbac.authorization.k8s.io/ClusterRoleBinding:operations/cert-manager-webhook-psp nil->obj (,,), Sync/0 resource rbac.authorization.k8s.io/Role:operations/cert-manager-cainjector:leaderelection nil->obj (,,), Sync/0 resource rbac.authorization.k8s.io/Role:operations/cert-manager-webhook:dynamic-serving nil->obj (,,), Sync/0 resource rbac.authorization.k8s.io/Role:operations/cert-manager:leaderelection nil->obj (,,), Sync/0 resource rbac.authorization.k8s.io/RoleBinding:operations/cert-manager-cainjector:leaderelection nil->obj (,,), Sync/0 resource rbac.authorization.k8s.io/RoleBinding:operations/cert-manager-webhook:dynamic-serving nil->obj (,,), Sync/0 resource rbac.authorization.k8s.io/RoleBinding:operations/cert-manager:leaderelection nil->obj (,,), Sync/0 resource /Service:operations/cert-manager nil->obj (,,), Sync/0 resource /Service:operations/cert-manager-webhook nil->obj (,,), Sync/0 resource apps/Deployment:operations/cert-manager nil->obj (,,), Sync/0 resource apps/Deployment:operations/cert-manager-cainjector nil->obj (,,), Sync/0 resource apps/Deployment:operations/cert-manager-webhook nil->obj (,,), Sync/0 resource admissionregistration.k8s.io/MutatingWebhookConfiguration:operations/cert-manager-webhook nil->obj (,,), Sync/0 resource admissionregistration.k8s.io/ValidatingWebhookConfiguration:operations/cert-manager-webhook nil->obj (,,)]"
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:36:54.280229380Z time="2020-08-11T23:36:54Z" level=info msg="Updating operation state. phase: Running -> Running, message: '' -> 'one or more tasks are running'" application=k8s00-dev-sydney-aws-cert-manager syncId=00001-ItuEG
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:36:54.876921200Z time="2020-08-11T23:36:54Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'podsecuritypolicy.policy/cert-manager created'" application=k8s00-dev-sydney-aws-cert-manager kind=PodSecurityPolicy name=cert-manager namespace=operations phase=Sync syncId=00001-ItuEG
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:36:54.877081366Z time="2020-08-11T23:36:54Z" level=info msg="Refreshing app status (controller refresh requested), level (1)" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:36:54.877500976Z time="2020-08-11T23:36:54Z" level=info msg="Comparing app state (cluster: https://XXXXXXXXXX.sk1.ap-southeast-2.eks.amazonaws.com, namespace: operations)" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:36:54.880603060Z time="2020-08-11T23:36:54Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'podsecuritypolicy.policy/cert-manager-webhook created'" application=k8s00-dev-sydney-aws-cert-manager kind=PodSecurityPolicy name=cert-manager-webhook namespace=operations phase=Sync syncId=00001-ItuEG
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:36:54.896073981Z time="2020-08-11T23:36:54Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'podsecuritypolicy.policy/cert-manager-cainjector created'" application=k8s00-dev-sydney-aws-cert-manager kind=PodSecurityPolicy name=cert-manager-cainjector namespace=operations phase=Sync syncId=00001-ItuEG
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:36:54.929212372Z time="2020-08-11T23:36:54Z" level=info msg="getRepoObjs stats" application=k8s00-dev-sydney-aws-cert-manager build_options_ms=0 helm_ms=0 plugins_ms=0 repo_ms=0 time_ms=51 unmarshal_ms=51 version_ms=0
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:36:55.033346453Z time="2020-08-11T23:36:55Z" level=info msg="Update successful" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:36:55.033384830Z time="2020-08-11T23:36:55Z" level=info msg="Reconciliation completed" application=k8s00-dev-sydney-aws-cert-manager dedup_ms=0 dest-namespace=operations dest-server="https://XXXXXXXXXX.sk1.ap-southeast-2.eks.amazonaws.com" diff_ms=50 fields.level=1 git_ms=51 health_ms=0 live_ms=0 settings_ms=0 sync_ms=0 time_ms=156
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:36:55.033418854Z time="2020-08-11T23:36:55Z" level=info msg="Refreshing app status (controller refresh requested), level (1)" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:36:55.033723782Z time="2020-08-11T23:36:55Z" level=info msg="Comparing app state (cluster: https://XXXXXXXXXX.sk1.ap-southeast-2.eks.amazonaws.com, namespace: operations)" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:36:55.083675772Z time="2020-08-11T23:36:55Z" level=info msg="getRepoObjs stats" application=k8s00-dev-sydney-aws-cert-manager build_options_ms=0 helm_ms=0 plugins_ms=0 repo_ms=0 time_ms=50 unmarshal_ms=49 version_ms=0
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:36:55.238763299Z time="2020-08-11T23:36:55Z" level=info msg="Update successful" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:36:55.238798350Z time="2020-08-11T23:36:55Z" level=info msg="Reconciliation completed" application=k8s00-dev-sydney-aws-cert-manager dedup_ms=0 dest-namespace=operations dest-server="https://XXXXXXXXXX.sk1.ap-southeast-2.eks.amazonaws.com" diff_ms=85 fields.level=1 git_ms=50 health_ms=0 live_ms=0 settings_ms=0 sync_ms=0 time_ms=205
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:36:55.628699358Z time="2020-08-11T23:36:55Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'serviceaccount/cert-manager-cainjector created'" application=k8s00-dev-sydney-aws-cert-manager kind=ServiceAccount name=cert-manager-cainjector namespace=operations phase=Sync syncId=00001-ItuEG
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:36:55.629252692Z time="2020-08-11T23:36:55Z" level=info msg="Refreshing app status (controller refresh requested), level (1)" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:36:55.629407862Z time="2020-08-11T23:36:55Z" level=info msg="Comparing app state (cluster: https://XXXXXXXXXX.sk1.ap-southeast-2.eks.amazonaws.com, namespace: operations)" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:36:55.643607297Z time="2020-08-11T23:36:55Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'serviceaccount/cert-manager-webhook created'" application=k8s00-dev-sydney-aws-cert-manager kind=ServiceAccount name=cert-manager-webhook namespace=operations phase=Sync syncId=00001-ItuEG
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:36:55.658968344Z time="2020-08-11T23:36:55Z" level=info msg="getRepoObjs stats" application=k8s00-dev-sydney-aws-cert-manager build_options_ms=0 helm_ms=0 plugins_ms=0 repo_ms=0 time_ms=29 unmarshal_ms=29 version_ms=0
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:36:55.694044775Z time="2020-08-11T23:36:55Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'serviceaccount/cert-manager created'" application=k8s00-dev-sydney-aws-cert-manager kind=ServiceAccount name=cert-manager namespace=operations phase=Sync syncId=00001-ItuEG
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:36:55.803795601Z time="2020-08-11T23:36:55Z" level=info msg="Update successful" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:36:55.803882066Z time="2020-08-11T23:36:55Z" level=info msg="Reconciliation completed" application=k8s00-dev-sydney-aws-cert-manager dedup_ms=0 dest-namespace=operations dest-server="https://XXXXXXXXXX.sk1.ap-southeast-2.eks.amazonaws.com" diff_ms=65 fields.level=1 git_ms=29 health_ms=0 live_ms=0 settings_ms=0 sync_ms=0 time_ms=174
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:36:55.803920148Z time="2020-08-11T23:36:55Z" level=info msg="Refreshing app status (controller refresh requested), level (1)" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:36:55.804061954Z time="2020-08-11T23:36:55Z" level=info msg="Comparing app state (cluster: https://XXXXXXXXXX.sk1.ap-southeast-2.eks.amazonaws.com, namespace: operations)" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:36:55.846375028Z time="2020-08-11T23:36:55Z" level=info msg="getRepoObjs stats" application=k8s00-dev-sydney-aws-cert-manager build_options_ms=0 helm_ms=0 plugins_ms=0 repo_ms=0 time_ms=42 unmarshal_ms=41 version_ms=0
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:36:56.037825863Z time="2020-08-11T23:36:56Z" level=info msg="Update successful" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:36:56.037851136Z time="2020-08-11T23:36:56Z" level=info msg="Reconciliation completed" application=k8s00-dev-sydney-aws-cert-manager dedup_ms=0 dest-namespace=operations dest-server="https://XXXXXXXXXX.sk1.ap-southeast-2.eks.amazonaws.com" diff_ms=139 fields.level=1 git_ms=42 health_ms=0 live_ms=0 settings_ms=0 sync_ms=0 time_ms=233
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:36:57.153144762Z time="2020-08-11T23:36:57Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'customresourcedefinition.apiextensions.k8s.io/orders.acme.cert-manager.io created'" application=k8s00-dev-sydney-aws-cert-manager kind=CustomResourceDefinition name=orders.acme.cert-manager.io namespace=operations phase=Sync syncId=00001-ItuEG
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:36:57.157711545Z time="2020-08-11T23:36:57Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'customresourcedefinition.apiextensions.k8s.io/challenges.acme.cert-manager.io created'" application=k8s00-dev-sydney-aws-cert-manager kind=CustomResourceDefinition name=challenges.acme.cert-manager.io namespace=operations phase=Sync syncId=00001-ItuEG
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:36:57.440619026Z time="2020-08-11T23:36:57Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'customresourcedefinition.apiextensions.k8s.io/certificates.cert-manager.io created'" application=k8s00-dev-sydney-aws-cert-manager kind=CustomResourceDefinition name=certificates.cert-manager.io namespace=operations phase=Sync syncId=00001-ItuEG
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:36:57.613192160Z time="2020-08-11T23:36:57Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'customresourcedefinition.apiextensions.k8s.io/certificaterequests.cert-manager.io created'" application=k8s00-dev-sydney-aws-cert-manager kind=CustomResourceDefinition name=certificaterequests.cert-manager.io namespace=operations phase=Sync syncId=00001-ItuEG
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:36:58.229984459Z time="2020-08-11T23:36:58Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'customresourcedefinition.apiextensions.k8s.io/clusterissuers.cert-manager.io created'" application=k8s00-dev-sydney-aws-cert-manager kind=CustomResourceDefinition name=clusterissuers.cert-manager.io namespace=operations phase=Sync syncId=00001-ItuEG
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:36:58.376873481Z time="2020-08-11T23:36:58Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'customresourcedefinition.apiextensions.k8s.io/issuers.cert-manager.io created'" application=k8s00-dev-sydney-aws-cert-manager kind=CustomResourceDefinition name=issuers.cert-manager.io namespace=operations phase=Sync syncId=00001-ItuEG
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:36:59.289622220Z time="2020-08-11T23:36:59Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'clusterrole.rbac.authorization.k8s.io/cert-manager-controller-certificates created'" application=k8s00-dev-sydney-aws-cert-manager kind=ClusterRole name=cert-manager-controller-certificates namespace=operations phase=Sync syncId=00001-ItuEG
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:36:59.300017857Z time="2020-08-11T23:36:59Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'clusterrole.rbac.authorization.k8s.io/cert-manager-controller-challenges created'" application=k8s00-dev-sydney-aws-cert-manager kind=ClusterRole name=cert-manager-controller-challenges namespace=operations phase=Sync syncId=00001-ItuEG
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:36:59.317323714Z time="2020-08-11T23:36:59Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'clusterrole.rbac.authorization.k8s.io/cert-manager-controller-clusterissuers created'" application=k8s00-dev-sydney-aws-cert-manager kind=ClusterRole name=cert-manager-controller-clusterissuers namespace=operations phase=Sync syncId=00001-ItuEG
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:36:59.317668484Z time="2020-08-11T23:36:59Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'clusterrole.rbac.authorization.k8s.io/cert-manager-controller-ingress-shim created'" application=k8s00-dev-sydney-aws-cert-manager kind=ClusterRole name=cert-manager-controller-ingress-shim namespace=operations phase=Sync syncId=00001-ItuEG
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:36:59.317745140Z time="2020-08-11T23:36:59Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'clusterrole.rbac.authorization.k8s.io/cert-manager-controller-issuers created'" application=k8s00-dev-sydney-aws-cert-manager kind=ClusterRole name=cert-manager-controller-issuers namespace=operations phase=Sync syncId=00001-ItuEG
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:36:59.318080929Z time="2020-08-11T23:36:59Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'clusterrole.rbac.authorization.k8s.io/cert-manager-cainjector created'" application=k8s00-dev-sydney-aws-cert-manager kind=ClusterRole name=cert-manager-cainjector namespace=operations phase=Sync syncId=00001-ItuEG
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:36:59.358936227Z time="2020-08-11T23:36:59Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'clusterrole.rbac.authorization.k8s.io/cert-manager-controller-orders created'" application=k8s00-dev-sydney-aws-cert-manager kind=ClusterRole name=cert-manager-controller-orders namespace=operations phase=Sync syncId=00001-ItuEG
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:36:59.683050604Z time="2020-08-11T23:36:59Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'clusterrole.rbac.authorization.k8s.io/cert-manager-cainjector-psp reconciled. reconciliation required create\n\tmissing rules added:\n\t\t{Verbs:[use] APIGroups:[policy] Resources:[podsecuritypolicies] ResourceNames:[cert-manager-cainjector] NonResourceURLs:[]}. clusterrole.rbac.authorization.k8s.io/cert-manager-cainjector-psp configured. Warning:  apply should be used on resource created by either  create --save-config or  apply'" application=k8s00-dev-sydney-aws-cert-manager kind=ClusterRole name=cert-manager-cainjector-psp namespace=operations phase=Sync syncId=00001-ItuEG
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:36:59.683081769Z time="2020-08-11T23:36:59Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'clusterrole.rbac.authorization.k8s.io/cert-manager-edit reconciled. reconciliation required create\n\tmissing rules added:\n\t\t{Verbs:[create delete deletecollection patch update] APIGroups:[cert-manager.io] Resources:[certificates certificaterequests issuers] ResourceNames:[] NonResourceURLs:[]}. clusterrole.rbac.authorization.k8s.io/cert-manager-edit configured. Warning:  apply should be used on resource created by either  create --save-config or  apply'" application=k8s00-dev-sydney-aws-cert-manager kind=ClusterRole name=cert-manager-edit namespace=operations phase=Sync syncId=00001-ItuEG
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:36:59.683507208Z time="2020-08-11T23:36:59Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'clusterrole.rbac.authorization.k8s.io/cert-manager-webhook-psp reconciled. reconciliation required create\n\tmissing rules added:\n\t\t{Verbs:[use] APIGroups:[policy] Resources:[podsecuritypolicies] ResourceNames:[cert-manager-webhook] NonResourceURLs:[]}. clusterrole.rbac.authorization.k8s.io/cert-manager-webhook-psp configured. Warning:  apply should be used on resource created by either  create --save-config or  apply'" application=k8s00-dev-sydney-aws-cert-manager kind=ClusterRole name=cert-manager-webhook-psp namespace=operations phase=Sync syncId=00001-ItuEG
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:36:59.683535302Z time="2020-08-11T23:36:59Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'clusterrole.rbac.authorization.k8s.io/cert-manager-view reconciled. reconciliation required create\n\tmissing rules added:\n\t\t{Verbs:[get list watch] APIGroups:[cert-manager.io] Resources:[certificates certificaterequests issuers] ResourceNames:[] NonResourceURLs:[]}. clusterrole.rbac.authorization.k8s.io/cert-manager-view configured. Warning:  apply should be used on resource created by either  create --save-config or  apply'" application=k8s00-dev-sydney-aws-cert-manager kind=ClusterRole name=cert-manager-view namespace=operations phase=Sync syncId=00001-ItuEG
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:36:59.791947643Z time="2020-08-11T23:36:59Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'clusterrole.rbac.authorization.k8s.io/cert-manager-psp reconciled. reconciliation required create\n\tmissing rules added:\n\t\t{Verbs:[use] APIGroups:[policy] Resources:[podsecuritypolicies] ResourceNames:[cert-manager] NonResourceURLs:[]}. clusterrole.rbac.authorization.k8s.io/cert-manager-psp configured. Warning:  apply should be used on resource created by either  create --save-config or  apply'" application=k8s00-dev-sydney-aws-cert-manager kind=ClusterRole name=cert-manager-psp namespace=operations phase=Sync syncId=00001-ItuEG
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:37:00.514162232Z time="2020-08-11T23:37:00Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-orders created'" application=k8s00-dev-sydney-aws-cert-manager kind=ClusterRoleBinding name=cert-manager-controller-orders namespace=operations phase=Sync syncId=00001-ItuEG
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:37:00.514192299Z time="2020-08-11T23:37:00Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-ingress-shim created'" application=k8s00-dev-sydney-aws-cert-manager kind=ClusterRoleBinding name=cert-manager-controller-ingress-shim namespace=operations phase=Sync syncId=00001-ItuEG
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:37:00.634210802Z time="2020-08-11T23:37:00Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-certificates created'" application=k8s00-dev-sydney-aws-cert-manager kind=ClusterRoleBinding name=cert-manager-controller-certificates namespace=operations phase=Sync syncId=00001-ItuEG
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:37:00.653948860Z time="2020-08-11T23:37:00Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-clusterissuers created'" application=k8s00-dev-sydney-aws-cert-manager kind=ClusterRoleBinding name=cert-manager-controller-clusterissuers namespace=operations phase=Sync syncId=00001-ItuEG
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:37:00.685068521Z time="2020-08-11T23:37:00Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-issuers created'" application=k8s00-dev-sydney-aws-cert-manager kind=ClusterRoleBinding name=cert-manager-controller-issuers namespace=operations phase=Sync syncId=00001-ItuEG
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:37:00.704458291Z time="2020-08-11T23:37:00Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-challenges created'" application=k8s00-dev-sydney-aws-cert-manager kind=ClusterRoleBinding name=cert-manager-controller-challenges namespace=operations phase=Sync syncId=00001-ItuEG
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:37:00.704638903Z time="2020-08-11T23:37:00Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'clusterrolebinding.rbac.authorization.k8s.io/cert-manager-cainjector created'" application=k8s00-dev-sydney-aws-cert-manager kind=ClusterRoleBinding name=cert-manager-cainjector namespace=operations phase=Sync syncId=00001-ItuEG
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:37:00.963866975Z time="2020-08-11T23:37:00Z" level=info msg="Refreshing app status (controller refresh requested), level (1)" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:37:00.964080006Z time="2020-08-11T23:37:00Z" level=info msg="Comparing app state (cluster: https://XXXXXXXXXX.sk1.ap-southeast-2.eks.amazonaws.com, namespace: operations)" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:37:00.993511395Z time="2020-08-11T23:37:00Z" level=info msg="getRepoObjs stats" application=k8s00-dev-sydney-aws-cert-manager build_options_ms=0 helm_ms=0 plugins_ms=0 repo_ms=0 time_ms=29 unmarshal_ms=29 version_ms=0
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:37:01.037535236Z time="2020-08-11T23:37:01Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'clusterrolebinding.rbac.authorization.k8s.io/cert-manager-psp reconciled. reconciliation required create\n\tmissing subjects added:\n\t\t{Kind:ServiceAccount APIGroup: Name:cert-manager Namespace:operations}. clusterrolebinding.rbac.authorization.k8s.io/cert-manager-psp configured. Warning:  apply should be used on resource created by either  create --save-config or  apply'" application=k8s00-dev-sydney-aws-cert-manager kind=ClusterRoleBinding name=cert-manager-psp namespace=operations phase=Sync syncId=00001-ItuEG
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:37:01.046579029Z time="2020-08-11T23:37:01Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'clusterrolebinding.rbac.authorization.k8s.io/cert-manager-cainjector-psp reconciled. reconciliation required create\n\tmissing subjects added:\n\t\t{Kind:ServiceAccount APIGroup: Name:cert-manager-cainjector Namespace:operations}. clusterrolebinding.rbac.authorization.k8s.io/cert-manager-cainjector-psp configured. Warning:  apply should be used on resource created by either  create --save-config or  apply'" application=k8s00-dev-sydney-aws-cert-manager kind=ClusterRoleBinding name=cert-manager-cainjector-psp namespace=operations phase=Sync syncId=00001-ItuEG
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:37:01.127129325Z time="2020-08-11T23:37:01Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'clusterrolebinding.rbac.authorization.k8s.io/cert-manager-webhook-psp reconciled. reconciliation required create\n\tmissing subjects added:\n\t\t{Kind:ServiceAccount APIGroup: Name:cert-manager-webhook Namespace:operations}. clusterrolebinding.rbac.authorization.k8s.io/cert-manager-webhook-psp configured. Warning:  apply should be used on resource created by either  create --save-config or  apply'" application=k8s00-dev-sydney-aws-cert-manager kind=ClusterRoleBinding name=cert-manager-webhook-psp namespace=operations phase=Sync syncId=00001-ItuEG
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:37:02.090950102Z time="2020-08-11T23:37:02Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'role.rbac.authorization.k8s.io/cert-manager-cainjector:leaderelection created'" application=k8s00-dev-sydney-aws-cert-manager kind=Role name="cert-manager-cainjector:leaderelection" namespace=operations phase=Sync syncId=00001-ItuEG
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:37:02.090982256Z time="2020-08-11T23:37:02Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'role.rbac.authorization.k8s.io/cert-manager-webhook:dynamic-serving created'" application=k8s00-dev-sydney-aws-cert-manager kind=Role name="cert-manager-webhook:dynamic-serving" namespace=operations phase=Sync syncId=00001-ItuEG
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:37:02.091081560Z time="2020-08-11T23:37:02Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'role.rbac.authorization.k8s.io/cert-manager:leaderelection created'" application=k8s00-dev-sydney-aws-cert-manager kind=Role name="cert-manager:leaderelection" namespace=operations phase=Sync syncId=00001-ItuEG
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:37:02.875301307Z time="2020-08-11T23:37:02Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'rolebinding.rbac.authorization.k8s.io/cert-manager:leaderelection created'" application=k8s00-dev-sydney-aws-cert-manager kind=RoleBinding name="cert-manager:leaderelection" namespace=operations phase=Sync syncId=00001-ItuEG
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:37:02.882109544Z time="2020-08-11T23:37:02Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'rolebinding.rbac.authorization.k8s.io/cert-manager-cainjector:leaderelection created'" application=k8s00-dev-sydney-aws-cert-manager kind=RoleBinding name="cert-manager-cainjector:leaderelection" namespace=operations phase=Sync syncId=00001-ItuEG
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:37:02.891937034Z time="2020-08-11T23:37:02Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'rolebinding.rbac.authorization.k8s.io/cert-manager-webhook:dynamic-serving created'" application=k8s00-dev-sydney-aws-cert-manager kind=RoleBinding name="cert-manager-webhook:dynamic-serving" namespace=operations phase=Sync syncId=00001-ItuEG
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:37:04.139303131Z time="2020-08-11T23:37:04Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'service/cert-manager created'" application=k8s00-dev-sydney-aws-cert-manager kind=Service name=cert-manager namespace=operations phase=Sync syncId=00001-ItuEG
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:37:04.153626722Z time="2020-08-11T23:37:04Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'service/cert-manager-webhook created'" application=k8s00-dev-sydney-aws-cert-manager kind=Service name=cert-manager-webhook namespace=operations phase=Sync syncId=00001-ItuEG
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:37:04.931683276Z time="2020-08-11T23:37:04Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'deployment.apps/cert-manager-cainjector created'" application=k8s00-dev-sydney-aws-cert-manager kind=Deployment name=cert-manager-cainjector namespace=operations phase=Sync syncId=00001-ItuEG
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:37:04.978405212Z time="2020-08-11T23:37:04Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'deployment.apps/cert-manager created'" application=k8s00-dev-sydney-aws-cert-manager kind=Deployment name=cert-manager namespace=operations phase=Sync syncId=00001-ItuEG
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:37:04.981883850Z time="2020-08-11T23:37:04Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'deployment.apps/cert-manager-webhook created'" application=k8s00-dev-sydney-aws-cert-manager kind=Deployment name=cert-manager-webhook namespace=operations phase=Sync syncId=00001-ItuEG
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:37:05.538732475Z time="2020-08-11T23:37:05Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'mutatingwebhookconfiguration.admissionregistration.k8s.io/cert-manager-webhook created'" application=k8s00-dev-sydney-aws-cert-manager kind=MutatingWebhookConfiguration name=cert-manager-webhook namespace=operations phase=Sync syncId=00001-ItuEG
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:37:06.097276060Z time="2020-08-11T23:37:06Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'validatingwebhookconfiguration.admissionregistration.k8s.io/cert-manager-webhook created'" application=k8s00-dev-sydney-aws-cert-manager kind=ValidatingWebhookConfiguration name=cert-manager-webhook namespace=operations phase=Sync syncId=00001-ItuEG
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:37:06.097837376Z time="2020-08-11T23:37:06Z" level=info msg="Updating operation state. phase: Running -> Succeeded, message: 'one or more tasks are running' -> 'successfully synced (all tasks run)'" application=k8s00-dev-sydney-aws-cert-manager syncId=00001-ItuEG
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:37:06.097853790Z time="2020-08-11T23:37:06Z" level=info msg="sync/terminate complete" application=k8s00-dev-sydney-aws-cert-manager duration=27.971198082s syncId=00001-ItuEG
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-11T23:37:06.134529291Z time="2020-08-11T23:37:06Z" level=info msg="Sync operation to 0.16.1 succeeded" application=k8s00-dev-sydney-aws-cert-manager dest-namespace=operations dest-server="https://XXXXXXXXXX.sk1.ap-southeast-2.eks.amazonaws.com" reason=OperationCompleted type=Normal
jurgen.weber@Jurgens-MBP ~ $

bug

Source

jurgenweber

All 10 comments

just now I tried syncing and I am at a broken state:

+ argocd-application-controller-8458ff6b5-25jlr › argocd-application-controller
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-12T00:37:08.065238919Z time="2020-08-12T00:37:08Z" level=info msg="Initialized new operation: {&SyncOperation{Revision:0.16.1,Prune:false,DryRun:false,SyncStrategy:&SyncStrategy{Apply:nil,Hook:&SyncStrategyHook{SyncStrategyApply:SyncStrategyApply{Force:false,},},},Resources:[]SyncOperationResource{},Source:nil,Manifests:[],SyncOptions:[],} {[email protected] false} []}" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-12T00:37:08.065492258Z time="2020-08-12T00:37:08Z" level=info msg="Comparing app state (cluster: https://XXXXXXXXX.sk1.ap-southeast-2.eks.amazonaws.com, namespace: operations)" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-25jlr argocd-application-controller 2020-08-12T00:37:08.096041989Z time="2020-08-12T00:37:08Z" level=info msg="getRepoObjs stats" application=k8s00-dev-sydney-aws-cert-manager build_options_ms=0 helm_ms=0 plugins_ms=0 repo_ms=0 time_ms=30 unmarshal_ms=30 version_ms=0

and its just hung now, trying to sync and does nothing.

It also destroys argocd performance and entering or doing anything with any app now barely works or is super slow.

jurgenweber on 12 Aug 2020

ok, lastly! I deleted everything cert-manager 0.16.1 from my cluster and went back to 0.15.2 and it worked just fine!

Here are the logs:

jurgen.weber@Jurgens-MBP ~ $ cat asdf.txt

+ argocd-application-controller-8458ff6b5-ktc87 › argocd-application-controller
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:09:43.287625032Z time="2020-08-12T01:09:43Z" level=info msg="Refreshing app status (comparison expired. reconciledAt: never, expiry: 3m0s), level (2)" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:09:43.287916973Z time="2020-08-12T01:09:43Z" level=info msg="Comparing app state (cluster: https://XXXXXXXXXX.sk1.ap-southeast-2.eks.amazonaws.com, namespace: operations)" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:09:45.370150724Z time="2020-08-12T01:09:45Z" level=info msg="getRepoObjs stats" application=k8s00-dev-sydney-aws-cert-manager build_options_ms=0 helm_ms=0 plugins_ms=0 repo_ms=0 time_ms=2082 unmarshal_ms=2082 version_ms=0
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:09:45.430677606Z time="2020-08-12T01:09:45Z" level=info msg="Initiated automated sync to '0.15.2'" application=k8s00-dev-sydney-aws-cert-manager dest-namespace=operations dest-server="https://XXXXXXXXXX.sk1.ap-southeast-2.eks.amazonaws.com" reason=OperationStarted type=Normal
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:09:45.437670290Z time="2020-08-12T01:09:45Z" level=info msg="Initialized new operation: {&SyncOperation{Revision:0.15.2,Prune:false,DryRun:false,SyncStrategy:nil,Resources:[]SyncOperationResource{},Source:nil,Manifests:[],SyncOptions:[],} { true} []}" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:09:45.437677073Z time="2020-08-12T01:09:45Z" level=info msg="Initiated automated sync to '0.15.2'" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:09:45.437681438Z time="2020-08-12T01:09:45Z" level=info msg="Updated sync status:  -> OutOfSync" application=k8s00-dev-sydney-aws-cert-manager dest-namespace=operations dest-server="https://XXXXXXXXXX.sk1.ap-southeast-2.eks.amazonaws.com" reason=ResourceUpdated type=Normal
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:09:45.437686406Z time="2020-08-12T01:09:45Z" level=info msg="Comparing app state (cluster: https://XXXXXXXXXX.sk1.ap-southeast-2.eks.amazonaws.com, namespace: operations)" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:09:45.441433920Z time="2020-08-12T01:09:45Z" level=info msg="Updated health status:  -> Missing" application=k8s00-dev-sydney-aws-cert-manager dest-namespace=operations dest-server="https://XXXXXXXXXX.sk1.ap-southeast-2.eks.amazonaws.com" reason=ResourceUpdated type=Normal
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:09:45.456346721Z time="2020-08-12T01:09:45Z" level=info msg="getRepoObjs stats" application=k8s00-dev-sydney-aws-cert-manager build_options_ms=0 helm_ms=0 plugins_ms=0 repo_ms=0 time_ms=18 unmarshal_ms=18 version_ms=0
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:09:45.460849608Z time="2020-08-12T01:09:45Z" level=info msg="Update successful" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:09:45.460870493Z time="2020-08-12T01:09:45Z" level=info msg="Reconciliation completed" application=k8s00-dev-sydney-aws-cert-manager dedup_ms=0 dest-namespace=operations dest-server="https://XXXXXXXXXX.sk1.ap-southeast-2.eks.amazonaws.com" diff_ms=32 fields.level=2 git_ms=2082 health_ms=0 live_ms=0 settings_ms=0 sync_ms=0 time_ms=2170
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:09:45.460876527Z time="2020-08-12T01:09:45Z" level=info msg="Refreshing app status (comparison expired. reconciledAt: never, expiry: 3m0s), level (2)" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:09:45.460880470Z time="2020-08-12T01:09:45Z" level=info msg="Comparing app state (cluster: https://XXXXXXXXXX.sk1.ap-southeast-2.eks.amazonaws.com, namespace: operations)" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:09:45.480821202Z time="2020-08-12T01:09:45Z" level=info msg="getRepoObjs stats" application=k8s00-dev-sydney-aws-cert-manager build_options_ms=0 helm_ms=0 plugins_ms=0 repo_ms=0 time_ms=21 unmarshal_ms=21 version_ms=0
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:09:45.484259226Z time="2020-08-12T01:09:45Z" level=info msg=syncing application=k8s00-dev-sydney-aws-cert-manager skipHooks=false started=false syncId=00002-FBMhD
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:09:45.520743480Z time="2020-08-12T01:09:45Z" level=info msg="Skipping auto-sync: another operation is in progress" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:09:45.520877761Z time="2020-08-12T01:09:45Z" level=info msg="Updated sync status:  -> OutOfSync" application=k8s00-dev-sydney-aws-cert-manager dest-namespace=operations dest-server="https://XXXXXXXXXX.sk1.ap-southeast-2.eks.amazonaws.com" reason=ResourceUpdated type=Normal
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:09:45.524116483Z time="2020-08-12T01:09:45Z" level=info msg="Updated health status:  -> Missing" application=k8s00-dev-sydney-aws-cert-manager dest-namespace=operations dest-server="https://XXXXXXXXXX.sk1.ap-southeast-2.eks.amazonaws.com" reason=ResourceUpdated type=Normal
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:09:45.541477659Z time="2020-08-12T01:09:45Z" level=info msg="Update successful" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:09:45.541511339Z time="2020-08-12T01:09:45Z" level=info msg="Reconciliation completed" application=k8s00-dev-sydney-aws-cert-manager dedup_ms=0 dest-namespace=operations dest-server="https://XXXXXXXXXX.sk1.ap-southeast-2.eks.amazonaws.com" diff_ms=22 fields.level=2 git_ms=21 health_ms=0 live_ms=0 settings_ms=0 sync_ms=0 time_ms=82
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:09:45.541520306Z time="2020-08-12T01:09:45Z" level=info msg="Refreshing app status (controller refresh requested), level (1)" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:09:45.541758828Z time="2020-08-12T01:09:45Z" level=info msg="Comparing app state (cluster: https://XXXXXXXXXX.sk1.ap-southeast-2.eks.amazonaws.com, namespace: operations)" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:09:45.558398896Z time="2020-08-12T01:09:45Z" level=info msg="getRepoObjs stats" application=k8s00-dev-sydney-aws-cert-manager build_options_ms=0 helm_ms=0 plugins_ms=0 repo_ms=0 time_ms=16 unmarshal_ms=16 version_ms=0
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:09:45.595206598Z time="2020-08-12T01:09:45Z" level=info msg="Skipping auto-sync: another operation is in progress" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:09:45.609780414Z time="2020-08-12T01:09:45Z" level=info msg="Update successful" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:09:45.609811394Z time="2020-08-12T01:09:45Z" level=info msg="Reconciliation completed" application=k8s00-dev-sydney-aws-cert-manager dedup_ms=0 dest-namespace=operations dest-server="https://XXXXXXXXXX.sk1.ap-southeast-2.eks.amazonaws.com" diff_ms=21 fields.level=1 git_ms=16 health_ms=0 live_ms=0 settings_ms=0 sync_ms=0 time_ms=68
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:09:52.422376228Z time="2020-08-12T01:09:52Z" level=info msg=tasks application=k8s00-dev-sydney-aws-cert-manager syncId=00002-FBMhD tasks="[Sync/0 resource policy/PodSecurityPolicy:operations/cert-manager nil->obj (,,), Sync/0 resource policy/PodSecurityPolicy:operations/cert-manager-cainjector nil->obj (,,), Sync/0 resource policy/PodSecurityPolicy:operations/cert-manager-webhook nil->obj (,,), Sync/0 resource /ServiceAccount:operations/cert-manager nil->obj (,,), Sync/0 resource /ServiceAccount:operations/cert-manager-cainjector nil->obj (,,), Sync/0 resource /ServiceAccount:operations/cert-manager-webhook nil->obj (,,), Sync/0 resource apiextensions.k8s.io/CustomResourceDefinition:operations/certificaterequests.cert-manager.io nil->obj (,,), Sync/0 resource apiextensions.k8s.io/CustomResourceDefinition:operations/certificates.cert-manager.io nil->obj (,,), Sync/0 resource apiextensions.k8s.io/CustomResourceDefinition:operations/challenges.acme.cert-manager.io nil->obj (,,), Sync/0 resource apiextensions.k8s.io/CustomResourceDefinition:operations/clusterissuers.cert-manager.io nil->obj (,,), Sync/0 resource apiextensions.k8s.io/CustomResourceDefinition:operations/issuers.cert-manager.io nil->obj (,,), Sync/0 resource apiextensions.k8s.io/CustomResourceDefinition:operations/orders.acme.cert-manager.io nil->obj (,,), Sync/0 resource rbac.authorization.k8s.io/ClusterRole:operations/cert-manager-cainjector nil->obj (,,), Sync/0 resource rbac.authorization.k8s.io/ClusterRole:operations/cert-manager-cainjector-psp nil->obj (,,), Sync/0 resource rbac.authorization.k8s.io/ClusterRole:operations/cert-manager-controller-certificates nil->obj (,,), Sync/0 resource rbac.authorization.k8s.io/ClusterRole:operations/cert-manager-controller-challenges nil->obj (,,), Sync/0 resource rbac.authorization.k8s.io/ClusterRole:operations/cert-manager-controller-clusterissuers nil->obj (,,), Sync/0 resource rbac.authorization.k8s.io/ClusterRole:operations/cert-manager-controller-ingress-shim nil->obj (,,), Sync/0 resource rbac.authorization.k8s.io/ClusterRole:operations/cert-manager-controller-issuers nil->obj (,,), Sync/0 resource rbac.authorization.k8s.io/ClusterRole:operations/cert-manager-controller-orders nil->obj (,,), Sync/0 resource rbac.authorization.k8s.io/ClusterRole:operations/cert-manager-edit nil->obj (,,), Sync/0 resource rbac.authorization.k8s.io/ClusterRole:operations/cert-manager-psp nil->obj (,,), Sync/0 resource rbac.authorization.k8s.io/ClusterRole:operations/cert-manager-view nil->obj (,,), Sync/0 resource rbac.authorization.k8s.io/ClusterRole:operations/cert-manager-webhook-psp nil->obj (,,), Sync/0 resource rbac.authorization.k8s.io/ClusterRoleBinding:operations/cert-manager-cainjector nil->obj (,,), Sync/0 resource rbac.authorization.k8s.io/ClusterRoleBinding:operations/cert-manager-cainjector-psp nil->obj (,,), Sync/0 resource rbac.authorization.k8s.io/ClusterRoleBinding:operations/cert-manager-controller-certificates nil->obj (,,), Sync/0 resource rbac.authorization.k8s.io/ClusterRoleBinding:operations/cert-manager-controller-challenges nil->obj (,,), Sync/0 resource rbac.authorization.k8s.io/ClusterRoleBinding:operations/cert-manager-controller-clusterissuers nil->obj (,,), Sync/0 resource rbac.authorization.k8s.io/ClusterRoleBinding:operations/cert-manager-controller-ingress-shim nil->obj (,,), Sync/0 resource rbac.authorization.k8s.io/ClusterRoleBinding:operations/cert-manager-controller-issuers nil->obj (,,), Sync/0 resource rbac.authorization.k8s.io/ClusterRoleBinding:operations/cert-manager-controller-orders nil->obj (,,), Sync/0 resource rbac.authorization.k8s.io/ClusterRoleBinding:operations/cert-manager-psp nil->obj (,,), Sync/0 resource rbac.authorization.k8s.io/ClusterRoleBinding:operations/cert-manager-webhook-psp nil->obj (,,), Sync/0 resource rbac.authorization.k8s.io/Role:operations/cert-manager-cainjector:leaderelection nil->obj (,,), Sync/0 resource rbac.authorization.k8s.io/Role:operations/cert-manager-webhook:dynamic-serving nil->obj (,,), Sync/0 resource rbac.authorization.k8s.io/Role:operations/cert-manager:leaderelection nil->obj (,,), Sync/0 resource rbac.authorization.k8s.io/RoleBinding:operations/cert-manager-cainjector:leaderelection nil->obj (,,), Sync/0 resource rbac.authorization.k8s.io/RoleBinding:operations/cert-manager-webhook:dynamic-serving nil->obj (,,), Sync/0 resource rbac.authorization.k8s.io/RoleBinding:operations/cert-manager:leaderelection nil->obj (,,), Sync/0 resource /Service:operations/cert-manager nil->obj (,,), Sync/0 resource /Service:operations/cert-manager-webhook nil->obj (,,), Sync/0 resource apps/Deployment:operations/cert-manager nil->obj (,,), Sync/0 resource apps/Deployment:operations/cert-manager-cainjector nil->obj (,,), Sync/0 resource apps/Deployment:operations/cert-manager-webhook nil->obj (,,), Sync/0 resource admissionregistration.k8s.io/ValidatingWebhookConfiguration:operations/cert-manager-webhook nil->obj (,,), Sync/0 resource admissionregistration.k8s.io/MutatingWebhookConfiguration:operations/cert-manager-webhook nil->obj (,,)]"
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:01.628641198Z time="2020-08-12T01:10:01Z" level=info msg="Updating operation state. phase: Running -> Running, message: '' -> 'one or more tasks are running'" application=k8s00-dev-sydney-aws-cert-manager syncId=00002-FBMhD
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:02.280126434Z time="2020-08-12T01:10:02Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'podsecuritypolicy.policy/cert-manager-cainjector created'" application=k8s00-dev-sydney-aws-cert-manager kind=PodSecurityPolicy name=cert-manager-cainjector namespace=operations phase=Sync syncId=00002-FBMhD
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:02.280310819Z time="2020-08-12T01:10:02Z" level=info msg="Refreshing app status (controller refresh requested), level (1)" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:02.280599797Z time="2020-08-12T01:10:02Z" level=info msg="Comparing app state (cluster: https://XXXXXXXXXX.sk1.ap-southeast-2.eks.amazonaws.com, namespace: operations)" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:02.289387631Z time="2020-08-12T01:10:02Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'podsecuritypolicy.policy/cert-manager created'" application=k8s00-dev-sydney-aws-cert-manager kind=PodSecurityPolicy name=cert-manager namespace=operations phase=Sync syncId=00002-FBMhD
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:02.296997714Z time="2020-08-12T01:10:02Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'podsecuritypolicy.policy/cert-manager-webhook created'" application=k8s00-dev-sydney-aws-cert-manager kind=PodSecurityPolicy name=cert-manager-webhook namespace=operations phase=Sync syncId=00002-FBMhD
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:02.302499659Z time="2020-08-12T01:10:02Z" level=info msg="getRepoObjs stats" application=k8s00-dev-sydney-aws-cert-manager build_options_ms=0 helm_ms=0 plugins_ms=0 repo_ms=0 time_ms=21 unmarshal_ms=20 version_ms=0
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:02.500564329Z time="2020-08-12T01:10:02Z" level=info msg="Skipping auto-sync: another operation is in progress" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:02.518874611Z time="2020-08-12T01:10:02Z" level=info msg="Update successful" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:02.518901707Z time="2020-08-12T01:10:02Z" level=info msg="Reconciliation completed" application=k8s00-dev-sydney-aws-cert-manager dedup_ms=0 dest-namespace=operations dest-server="https://XXXXXXXXXX.sk1.ap-southeast-2.eks.amazonaws.com" diff_ms=24 fields.level=1 git_ms=21 health_ms=0 live_ms=0 settings_ms=0 sync_ms=0 time_ms=238
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:02.518958475Z time="2020-08-12T01:10:02Z" level=info msg="Refreshing app status (controller refresh requested), level (1)" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:02.519254827Z time="2020-08-12T01:10:02Z" level=info msg="Comparing app state (cluster: https://XXXXXXXXXX.sk1.ap-southeast-2.eks.amazonaws.com, namespace: operations)" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:02.544099769Z time="2020-08-12T01:10:02Z" level=info msg="getRepoObjs stats" application=k8s00-dev-sydney-aws-cert-manager build_options_ms=0 helm_ms=0 plugins_ms=0 repo_ms=0 time_ms=24 unmarshal_ms=24 version_ms=0
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:02.622131363Z time="2020-08-12T01:10:02Z" level=info msg="Skipping auto-sync: another operation is in progress" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:02.638224829Z time="2020-08-12T01:10:02Z" level=info msg="Update successful" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:02.638256317Z time="2020-08-12T01:10:02Z" level=info msg="Reconciliation completed" application=k8s00-dev-sydney-aws-cert-manager dedup_ms=0 dest-namespace=operations dest-server="https://XXXXXXXXXX.sk1.ap-southeast-2.eks.amazonaws.com" diff_ms=32 fields.level=1 git_ms=24 health_ms=0 live_ms=0 settings_ms=0 sync_ms=0 time_ms=119
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:03.069849409Z time="2020-08-12T01:10:03Z" level=info msg="Refreshing app status (controller refresh requested), level (1)" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:03.070072538Z time="2020-08-12T01:10:03Z" level=info msg="Comparing app state (cluster: https://XXXXXXXXXX.sk1.ap-southeast-2.eks.amazonaws.com, namespace: operations)" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:03.070725204Z time="2020-08-12T01:10:03Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'serviceaccount/cert-manager created'" application=k8s00-dev-sydney-aws-cert-manager kind=ServiceAccount name=cert-manager namespace=operations phase=Sync syncId=00002-FBMhD
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:03.084528135Z time="2020-08-12T01:10:03Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'serviceaccount/cert-manager-webhook created'" application=k8s00-dev-sydney-aws-cert-manager kind=ServiceAccount name=cert-manager-webhook namespace=operations phase=Sync syncId=00002-FBMhD
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:03.086661064Z time="2020-08-12T01:10:03Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'serviceaccount/cert-manager-cainjector created'" application=k8s00-dev-sydney-aws-cert-manager kind=ServiceAccount name=cert-manager-cainjector namespace=operations phase=Sync syncId=00002-FBMhD
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:03.132110601Z time="2020-08-12T01:10:03Z" level=info msg="getRepoObjs stats" application=k8s00-dev-sydney-aws-cert-manager build_options_ms=0 helm_ms=0 plugins_ms=0 repo_ms=0 time_ms=61 unmarshal_ms=61 version_ms=0
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:03.174623749Z time="2020-08-12T01:10:03Z" level=info msg="Skipping auto-sync: another operation is in progress" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:03.189982570Z time="2020-08-12T01:10:03Z" level=info msg="Update successful" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:03.190005753Z time="2020-08-12T01:10:03Z" level=info msg="Reconciliation completed" application=k8s00-dev-sydney-aws-cert-manager dedup_ms=0 dest-namespace=operations dest-server="https://XXXXXXXXXX.sk1.ap-southeast-2.eks.amazonaws.com" diff_ms=25 fields.level=1 git_ms=62 health_ms=0 live_ms=0 settings_ms=0 sync_ms=0 time_ms=120
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:03.190043343Z time="2020-08-12T01:10:03Z" level=info msg="Refreshing app status (controller refresh requested), level (1)" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:03.190245079Z time="2020-08-12T01:10:03Z" level=info msg="Comparing app state (cluster: https://XXXXXXXXXX.sk1.ap-southeast-2.eks.amazonaws.com, namespace: operations)" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:03.226602980Z time="2020-08-12T01:10:03Z" level=info msg="getRepoObjs stats" application=k8s00-dev-sydney-aws-cert-manager build_options_ms=0 helm_ms=0 plugins_ms=0 repo_ms=0 time_ms=36 unmarshal_ms=36 version_ms=0
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:03.345157321Z time="2020-08-12T01:10:03Z" level=info msg="Skipping auto-sync: another operation is in progress" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:03.359553904Z time="2020-08-12T01:10:03Z" level=info msg="Update successful" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:03.359595202Z time="2020-08-12T01:10:03Z" level=info msg="Reconciliation completed" application=k8s00-dev-sydney-aws-cert-manager dedup_ms=0 dest-namespace=operations dest-server="https://XXXXXXXXXX.sk1.ap-southeast-2.eks.amazonaws.com" diff_ms=53 fields.level=1 git_ms=36 health_ms=0 live_ms=0 settings_ms=0 sync_ms=0 time_ms=168
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:04.139447536Z time="2020-08-12T01:10:04Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'customresourcedefinition.apiextensions.k8s.io/certificaterequests.cert-manager.io created'" application=k8s00-dev-sydney-aws-cert-manager kind=CustomResourceDefinition name=certificaterequests.cert-manager.io namespace=operations phase=Sync syncId=00002-FBMhD
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:04.139486943Z time="2020-08-12T01:10:04Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'customresourcedefinition.apiextensions.k8s.io/orders.acme.cert-manager.io created'" application=k8s00-dev-sydney-aws-cert-manager kind=CustomResourceDefinition name=orders.acme.cert-manager.io namespace=operations phase=Sync syncId=00002-FBMhD
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:04.324999242Z time="2020-08-12T01:10:04Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'customresourcedefinition.apiextensions.k8s.io/certificates.cert-manager.io created'" application=k8s00-dev-sydney-aws-cert-manager kind=CustomResourceDefinition name=certificates.cert-manager.io namespace=operations phase=Sync syncId=00002-FBMhD
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:04.620544810Z time="2020-08-12T01:10:04Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'customresourcedefinition.apiextensions.k8s.io/issuers.cert-manager.io created'" application=k8s00-dev-sydney-aws-cert-manager kind=CustomResourceDefinition name=issuers.cert-manager.io namespace=operations phase=Sync syncId=00002-FBMhD
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:04.769166235Z time="2020-08-12T01:10:04Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'customresourcedefinition.apiextensions.k8s.io/clusterissuers.cert-manager.io created'" application=k8s00-dev-sydney-aws-cert-manager kind=CustomResourceDefinition name=clusterissuers.cert-manager.io namespace=operations phase=Sync syncId=00002-FBMhD
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:05.095132745Z time="2020-08-12T01:10:05Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'customresourcedefinition.apiextensions.k8s.io/challenges.acme.cert-manager.io created'" application=k8s00-dev-sydney-aws-cert-manager kind=CustomResourceDefinition name=challenges.acme.cert-manager.io namespace=operations phase=Sync syncId=00002-FBMhD
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:05.871306432Z time="2020-08-12T01:10:05Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'clusterrole.rbac.authorization.k8s.io/cert-manager-controller-ingress-shim created'" application=k8s00-dev-sydney-aws-cert-manager kind=ClusterRole name=cert-manager-controller-ingress-shim namespace=operations phase=Sync syncId=00002-FBMhD
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:05.897050464Z time="2020-08-12T01:10:05Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'clusterrole.rbac.authorization.k8s.io/cert-manager-controller-certificates created'" application=k8s00-dev-sydney-aws-cert-manager kind=ClusterRole name=cert-manager-controller-certificates namespace=operations phase=Sync syncId=00002-FBMhD
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:05.972984135Z time="2020-08-12T01:10:05Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'clusterrole.rbac.authorization.k8s.io/cert-manager-controller-clusterissuers created'" application=k8s00-dev-sydney-aws-cert-manager kind=ClusterRole name=cert-manager-controller-clusterissuers namespace=operations phase=Sync syncId=00002-FBMhD
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:06.002892040Z time="2020-08-12T01:10:06Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'clusterrole.rbac.authorization.k8s.io/cert-manager-controller-issuers created'" application=k8s00-dev-sydney-aws-cert-manager kind=ClusterRole name=cert-manager-controller-issuers namespace=operations phase=Sync syncId=00002-FBMhD
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:06.002932800Z time="2020-08-12T01:10:06Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'clusterrole.rbac.authorization.k8s.io/cert-manager-cainjector created'" application=k8s00-dev-sydney-aws-cert-manager kind=ClusterRole name=cert-manager-cainjector namespace=operations phase=Sync syncId=00002-FBMhD
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:06.053310893Z time="2020-08-12T01:10:06Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'clusterrole.rbac.authorization.k8s.io/cert-manager-controller-orders created'" application=k8s00-dev-sydney-aws-cert-manager kind=ClusterRole name=cert-manager-controller-orders namespace=operations phase=Sync syncId=00002-FBMhD
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:06.053342892Z time="2020-08-12T01:10:06Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'clusterrole.rbac.authorization.k8s.io/cert-manager-controller-challenges created'" application=k8s00-dev-sydney-aws-cert-manager kind=ClusterRole name=cert-manager-controller-challenges namespace=operations phase=Sync syncId=00002-FBMhD
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:06.224919955Z time="2020-08-12T01:10:06Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'clusterrole.rbac.authorization.k8s.io/cert-manager-webhook-psp reconciled. reconciliation required create\n\tmissing rules added:\n\t\t{Verbs:[use] APIGroups:[policy] Resources:[podsecuritypolicies] ResourceNames:[cert-manager-webhook] NonResourceURLs:[]}. clusterrole.rbac.authorization.k8s.io/cert-manager-webhook-psp configured. Warning:  apply should be used on resource created by either  create --save-config or  apply'" application=k8s00-dev-sydney-aws-cert-manager kind=ClusterRole name=cert-manager-webhook-psp namespace=operations phase=Sync syncId=00002-FBMhD
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:06.295413662Z time="2020-08-12T01:10:06Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'clusterrole.rbac.authorization.k8s.io/cert-manager-cainjector-psp reconciled. reconciliation required create\n\tmissing rules added:\n\t\t{Verbs:[use] APIGroups:[policy] Resources:[podsecuritypolicies] ResourceNames:[cert-manager-cainjector] NonResourceURLs:[]}. clusterrole.rbac.authorization.k8s.io/cert-manager-cainjector-psp configured. Warning:  apply should be used on resource created by either  create --save-config or  apply'" application=k8s00-dev-sydney-aws-cert-manager kind=ClusterRole name=cert-manager-cainjector-psp namespace=operations phase=Sync syncId=00002-FBMhD
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:06.295443622Z time="2020-08-12T01:10:06Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'clusterrole.rbac.authorization.k8s.io/cert-manager-edit reconciled. reconciliation required create\n\tmissing rules added:\n\t\t{Verbs:[create delete deletecollection patch update] APIGroups:[cert-manager.io] Resources:[certificates certificaterequests issuers] ResourceNames:[] NonResourceURLs:[]}. clusterrole.rbac.authorization.k8s.io/cert-manager-edit configured. Warning:  apply should be used on resource created by either  create --save-config or  apply'" application=k8s00-dev-sydney-aws-cert-manager kind=ClusterRole name=cert-manager-edit namespace=operations phase=Sync syncId=00002-FBMhD
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:06.395583793Z time="2020-08-12T01:10:06Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'clusterrole.rbac.authorization.k8s.io/cert-manager-psp reconciled. reconciliation required create\n\tmissing rules added:\n\t\t{Verbs:[use] APIGroups:[policy] Resources:[podsecuritypolicies] ResourceNames:[cert-manager] NonResourceURLs:[]}. clusterrole.rbac.authorization.k8s.io/cert-manager-psp configured. Warning:  apply should be used on resource created by either  create --save-config or  apply'" application=k8s00-dev-sydney-aws-cert-manager kind=ClusterRole name=cert-manager-psp namespace=operations phase=Sync syncId=00002-FBMhD
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:06.395628530Z time="2020-08-12T01:10:06Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'clusterrole.rbac.authorization.k8s.io/cert-manager-view reconciled. reconciliation required create\n\tmissing rules added:\n\t\t{Verbs:[get list watch] APIGroups:[cert-manager.io] Resources:[certificates certificaterequests issuers] ResourceNames:[] NonResourceURLs:[]}. clusterrole.rbac.authorization.k8s.io/cert-manager-view configured. Warning:  apply should be used on resource created by either  create --save-config or  apply'" application=k8s00-dev-sydney-aws-cert-manager kind=ClusterRole name=cert-manager-view namespace=operations phase=Sync syncId=00002-FBMhD
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:07.037439406Z time="2020-08-12T01:10:07Z" level=info msg="Refreshing app status (controller refresh requested), level (1)" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:07.037443167Z time="2020-08-12T01:10:07Z" level=info msg="Comparing app state (cluster: https://XXXXXXXXXX.sk1.ap-southeast-2.eks.amazonaws.com, namespace: operations)" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:07.064811866Z time="2020-08-12T01:10:07Z" level=info msg="getRepoObjs stats" application=k8s00-dev-sydney-aws-cert-manager build_options_ms=0 helm_ms=0 plugins_ms=0 repo_ms=0 time_ms=28 unmarshal_ms=28 version_ms=0
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:07.184030647Z time="2020-08-12T01:10:07Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'clusterrolebinding.rbac.authorization.k8s.io/cert-manager-cainjector created'" application=k8s00-dev-sydney-aws-cert-manager kind=ClusterRoleBinding name=cert-manager-cainjector namespace=operations phase=Sync syncId=00002-FBMhD
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:07.211573383Z time="2020-08-12T01:10:07Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-clusterissuers created'" application=k8s00-dev-sydney-aws-cert-manager kind=ClusterRoleBinding name=cert-manager-controller-clusterissuers namespace=operations phase=Sync syncId=00002-FBMhD
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:07.211597071Z time="2020-08-12T01:10:07Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-certificates created'" application=k8s00-dev-sydney-aws-cert-manager kind=ClusterRoleBinding name=cert-manager-controller-certificates namespace=operations phase=Sync syncId=00002-FBMhD
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:07.232210705Z time="2020-08-12T01:10:07Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-orders created'" application=k8s00-dev-sydney-aws-cert-manager kind=ClusterRoleBinding name=cert-manager-controller-orders namespace=operations phase=Sync syncId=00002-FBMhD
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:07.232273695Z time="2020-08-12T01:10:07Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-ingress-shim created'" application=k8s00-dev-sydney-aws-cert-manager kind=ClusterRoleBinding name=cert-manager-controller-ingress-shim namespace=operations phase=Sync syncId=00002-FBMhD
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:07.232286802Z time="2020-08-12T01:10:07Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-issuers created'" application=k8s00-dev-sydney-aws-cert-manager kind=ClusterRoleBinding name=cert-manager-controller-issuers namespace=operations phase=Sync syncId=00002-FBMhD
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:07.249150300Z time="2020-08-12T01:10:07Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-challenges created'" application=k8s00-dev-sydney-aws-cert-manager kind=ClusterRoleBinding name=cert-manager-controller-challenges namespace=operations phase=Sync syncId=00002-FBMhD
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:07.729140450Z time="2020-08-12T01:10:07Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'clusterrolebinding.rbac.authorization.k8s.io/cert-manager-psp reconciled. reconciliation required create\n\tmissing subjects added:\n\t\t{Kind:ServiceAccount APIGroup: Name:cert-manager Namespace:operations}. clusterrolebinding.rbac.authorization.k8s.io/cert-manager-psp configured. Warning:  apply should be used on resource created by either  create --save-config or  apply'" application=k8s00-dev-sydney-aws-cert-manager kind=ClusterRoleBinding name=cert-manager-psp namespace=operations phase=Sync syncId=00002-FBMhD
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:07.749394286Z time="2020-08-12T01:10:07Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'clusterrolebinding.rbac.authorization.k8s.io/cert-manager-webhook-psp reconciled. reconciliation required create\n\tmissing subjects added:\n\t\t{Kind:ServiceAccount APIGroup: Name:cert-manager-webhook Namespace:operations}. clusterrolebinding.rbac.authorization.k8s.io/cert-manager-webhook-psp configured. Warning:  apply should be used on resource created by either  create --save-config or  apply'" application=k8s00-dev-sydney-aws-cert-manager kind=ClusterRoleBinding name=cert-manager-webhook-psp namespace=operations phase=Sync syncId=00002-FBMhD
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:07.749431359Z time="2020-08-12T01:10:07Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'clusterrolebinding.rbac.authorization.k8s.io/cert-manager-cainjector-psp reconciled. reconciliation required create\n\tmissing subjects added:\n\t\t{Kind:ServiceAccount APIGroup: Name:cert-manager-cainjector Namespace:operations}. clusterrolebinding.rbac.authorization.k8s.io/cert-manager-cainjector-psp configured. Warning:  apply should be used on resource created by either  create --save-config or  apply'" application=k8s00-dev-sydney-aws-cert-manager kind=ClusterRoleBinding name=cert-manager-cainjector-psp namespace=operations phase=Sync syncId=00002-FBMhD
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:08.156889755Z time="2020-08-12T01:10:08Z" level=info msg="Normalized app spec: {\"status\":{\"conditions\":[{\"lastTransitionTime\":\"2020-08-12T01:10:07Z\",\"message\":\"CustomResourceDefinition/certificaterequests.cert-manager.io is part of a different application: cert-manager\",\"type\":\"SharedResourceWarning\"},{\"lastTransitionTime\":\"2020-08-12T01:10:07Z\",\"message\":\"CustomResourceDefinition/orders.acme.cert-manager.io is part of a different application: cert-manager\",\"type\":\"SharedResourceWarning\"}]}}" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:08.752352561Z time="2020-08-12T01:10:08Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'role.rbac.authorization.k8s.io/cert-manager-cainjector:leaderelection created'" application=k8s00-dev-sydney-aws-cert-manager kind=Role name="cert-manager-cainjector:leaderelection" namespace=operations phase=Sync syncId=00002-FBMhD
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:08.752441731Z time="2020-08-12T01:10:08Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'role.rbac.authorization.k8s.io/cert-manager-webhook:dynamic-serving created'" application=k8s00-dev-sydney-aws-cert-manager kind=Role name="cert-manager-webhook:dynamic-serving" namespace=operations phase=Sync syncId=00002-FBMhD
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:08.752754669Z time="2020-08-12T01:10:08Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'role.rbac.authorization.k8s.io/cert-manager:leaderelection created'" application=k8s00-dev-sydney-aws-cert-manager kind=Role name="cert-manager:leaderelection" namespace=operations phase=Sync syncId=00002-FBMhD
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:09.580985740Z time="2020-08-12T01:10:09Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'rolebinding.rbac.authorization.k8s.io/cert-manager-webhook:dynamic-serving created'" application=k8s00-dev-sydney-aws-cert-manager kind=RoleBinding name="cert-manager-webhook:dynamic-serving" namespace=operations phase=Sync syncId=00002-FBMhD
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:09.582433999Z time="2020-08-12T01:10:09Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'rolebinding.rbac.authorization.k8s.io/cert-manager:leaderelection created'" application=k8s00-dev-sydney-aws-cert-manager kind=RoleBinding name="cert-manager:leaderelection" namespace=operations phase=Sync syncId=00002-FBMhD
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:09.582455894Z time="2020-08-12T01:10:09Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'rolebinding.rbac.authorization.k8s.io/cert-manager-cainjector:leaderelection created'" application=k8s00-dev-sydney-aws-cert-manager kind=RoleBinding name="cert-manager-cainjector:leaderelection" namespace=operations phase=Sync syncId=00002-FBMhD
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:10.498502698Z time="2020-08-12T01:10:10Z" level=info msg="Skipping auto-sync: another operation is in progress" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:10.513697387Z time="2020-08-12T01:10:10Z" level=info msg="Update successful" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:10.513719700Z time="2020-08-12T01:10:10Z" level=info msg="Reconciliation completed" application=k8s00-dev-sydney-aws-cert-manager dedup_ms=0 dest-namespace=operations dest-server="https://XXXXXXXXXX.sk1.ap-southeast-2.eks.amazonaws.com" diff_ms=123 fields.level=1 git_ms=28 health_ms=0 live_ms=952 settings_ms=0 sync_ms=0 time_ms=3477
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:10.513726253Z time="2020-08-12T01:10:10Z" level=info msg="Refreshing app status (controller refresh requested), level (1)" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:10.513968501Z time="2020-08-12T01:10:10Z" level=info msg="Comparing app state (cluster: https://XXXXXXXXXX.sk1.ap-southeast-2.eks.amazonaws.com, namespace: operations)" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:10.544101639Z time="2020-08-12T01:10:10Z" level=info msg="getRepoObjs stats" application=k8s00-dev-sydney-aws-cert-manager build_options_ms=0 helm_ms=0 plugins_ms=0 repo_ms=0 time_ms=30 unmarshal_ms=29 version_ms=0
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:10.808708298Z time="2020-08-12T01:10:10Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'service/cert-manager-webhook created'" application=k8s00-dev-sydney-aws-cert-manager kind=Service name=cert-manager-webhook namespace=operations phase=Sync syncId=00002-FBMhD
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:10.819168556Z time="2020-08-12T01:10:10Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'service/cert-manager created'" application=k8s00-dev-sydney-aws-cert-manager kind=Service name=cert-manager namespace=operations phase=Sync syncId=00002-FBMhD
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:11.417315400Z time="2020-08-12T01:10:11Z" level=info msg="Normalized app spec: {\"status\":{\"conditions\":[{\"lastTransitionTime\":\"2020-08-12T01:10:07Z\",\"message\":\"CustomResourceDefinition/certificaterequests.cert-manager.io is part of a different application: cert-manager\",\"type\":\"SharedResourceWarning\"},{\"lastTransitionTime\":\"2020-08-12T01:10:10Z\",\"message\":\"CustomResourceDefinition/certificates.cert-manager.io is part of a different application: cert-manager\",\"type\":\"SharedResourceWarning\"},{\"lastTransitionTime\":\"2020-08-12T01:10:10Z\",\"message\":\"CustomResourceDefinition/orders.acme.cert-manager.io is part of a different application: cert-manager\",\"type\":\"SharedResourceWarning\"}]}}" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:11.586930210Z time="2020-08-12T01:10:11Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'deployment.apps/cert-manager created'" application=k8s00-dev-sydney-aws-cert-manager kind=Deployment name=cert-manager namespace=operations phase=Sync syncId=00002-FBMhD
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:11.595322579Z time="2020-08-12T01:10:11Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'deployment.apps/cert-manager-webhook created'" application=k8s00-dev-sydney-aws-cert-manager kind=Deployment name=cert-manager-webhook namespace=operations phase=Sync syncId=00002-FBMhD
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:11.604810587Z time="2020-08-12T01:10:11Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'deployment.apps/cert-manager-cainjector created'" application=k8s00-dev-sydney-aws-cert-manager kind=Deployment name=cert-manager-cainjector namespace=operations phase=Sync syncId=00002-FBMhD
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:12.170591112Z time="2020-08-12T01:10:12Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'validatingwebhookconfiguration.admissionregistration.k8s.io/cert-manager-webhook created'" application=k8s00-dev-sydney-aws-cert-manager kind=ValidatingWebhookConfiguration name=cert-manager-webhook namespace=operations phase=Sync syncId=00002-FBMhD
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:12.831121599Z time="2020-08-12T01:10:12Z" level=info msg="adding resource result, status: 'Synced', phase: 'Running', message: 'mutatingwebhookconfiguration.admissionregistration.k8s.io/cert-manager-webhook created'" application=k8s00-dev-sydney-aws-cert-manager kind=MutatingWebhookConfiguration name=cert-manager-webhook namespace=operations phase=Sync syncId=00002-FBMhD
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:12.831159774Z time="2020-08-12T01:10:12Z" level=info msg="Updating operation state. phase: Running -> Succeeded, message: 'one or more tasks are running' -> 'successfully synced (all tasks run)'" application=k8s00-dev-sydney-aws-cert-manager syncId=00002-FBMhD
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:12.831168207Z time="2020-08-12T01:10:12Z" level=info msg="sync/terminate complete" application=k8s00-dev-sydney-aws-cert-manager duration=27.346905804s syncId=00002-FBMhD
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:13.525346283Z time="2020-08-12T01:10:13Z" level=info msg="Sync operation to 0.15.2 succeeded" application=k8s00-dev-sydney-aws-cert-manager dest-namespace=operations dest-server="https://XXXXXXXXXX.sk1.ap-southeast-2.eks.amazonaws.com" reason=OperationCompleted type=Normal
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:15.264399490Z time="2020-08-12T01:10:15Z" level=info msg="Skipping auto-sync: another operation is in progress" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:15.286624104Z time="2020-08-12T01:10:15Z" level=info msg="Update successful" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:15.286648385Z time="2020-08-12T01:10:15Z" level=info msg="Reconciliation completed" application=k8s00-dev-sydney-aws-cert-manager dedup_ms=0 dest-namespace=operations dest-server="https://XXXXXXXXXX.sk1.ap-southeast-2.eks.amazonaws.com" diff_ms=102 fields.level=1 git_ms=30 health_ms=0 live_ms=742 settings_ms=0 sync_ms=0 time_ms=4772
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:15.286696892Z time="2020-08-12T01:10:15Z" level=info msg="Refreshing app status (controller refresh requested), level (2)" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:15.286959833Z time="2020-08-12T01:10:15Z" level=info msg="Comparing app state (cluster: https://XXXXXXXXXX.sk1.ap-southeast-2.eks.amazonaws.com, namespace: operations)" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:15.314804875Z time="2020-08-12T01:10:15Z" level=info msg="getRepoObjs stats" application=k8s00-dev-sydney-aws-cert-manager build_options_ms=0 helm_ms=0 plugins_ms=0 repo_ms=0 time_ms=26 unmarshal_ms=25 version_ms=0
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:15.811460354Z time="2020-08-12T01:10:15Z" level=info msg="Normalized app spec: {\"status\":{\"conditions\":[{\"lastTransitionTime\":\"2020-08-12T01:10:07Z\",\"message\":\"CustomResourceDefinition/certificaterequests.cert-manager.io is part of a different application: cert-manager\",\"type\":\"SharedResourceWarning\"},{\"lastTransitionTime\":\"2020-08-12T01:10:15Z\",\"message\":\"CustomResourceDefinition/certificates.cert-manager.io is part of a different application: cert-manager\",\"type\":\"SharedResourceWarning\"},{\"lastTransitionTime\":\"2020-08-12T01:10:15Z\",\"message\":\"CustomResourceDefinition/challenges.acme.cert-manager.io is part of a different application: cert-manager\",\"type\":\"SharedResourceWarning\"},{\"lastTransitionTime\":\"2020-08-12T01:10:15Z\",\"message\":\"CustomResourceDefinition/clusterissuers.cert-manager.io is part of a different application: cert-manager\",\"type\":\"SharedResourceWarning\"},{\"lastTransitionTime\":\"2020-08-12T01:10:15Z\",\"message\":\"CustomResourceDefinition/issuers.cert-manager.io is part of a different application: cert-manager\",\"type\":\"SharedResourceWarning\"},{\"lastTransitionTime\":\"2020-08-12T01:10:15Z\",\"message\":\"CustomResourceDefinition/orders.acme.cert-manager.io is part of a different application: cert-manager\",\"type\":\"SharedResourceWarning\"}]}}" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:15.854494015Z time="2020-08-12T01:10:15Z" level=info msg="Skipping auto-sync: application status is Synced" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:15.854919329Z time="2020-08-12T01:10:15Z" level=info msg="Updated sync status: OutOfSync -> Synced" application=k8s00-dev-sydney-aws-cert-manager dest-namespace=operations dest-server="https://XXXXXXXXXX.sk1.ap-southeast-2.eks.amazonaws.com" reason=ResourceUpdated type=Normal
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:15.858201229Z time="2020-08-12T01:10:15Z" level=info msg="Updated health status: Missing -> Progressing" application=k8s00-dev-sydney-aws-cert-manager dest-namespace=operations dest-server="https://XXXXXXXXXX.sk1.ap-southeast-2.eks.amazonaws.com" reason=ResourceUpdated type=Normal
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:15.884224984Z time="2020-08-12T01:10:15Z" level=info msg="Update successful" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:15.884254260Z time="2020-08-12T01:10:15Z" level=info msg="Reconciliation completed" application=k8s00-dev-sydney-aws-cert-manager dedup_ms=0 dest-namespace=operations dest-server="https://XXXXXXXXXX.sk1.ap-southeast-2.eks.amazonaws.com" diff_ms=168 fields.level=2 git_ms=26 health_ms=2 live_ms=304 settings_ms=0 sync_ms=0 time_ms=597
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:17.403489498Z time="2020-08-12T01:10:17Z" level=info msg="Refreshing app status (controller refresh requested), level (1)" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:17.403763331Z time="2020-08-12T01:10:17Z" level=info msg="Comparing app state (cluster: https://XXXXXXXXXX.sk1.ap-southeast-2.eks.amazonaws.com, namespace: operations)" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:17.426356054Z time="2020-08-12T01:10:17Z" level=info msg="getRepoObjs stats" application=k8s00-dev-sydney-aws-cert-manager build_options_ms=0 helm_ms=0 plugins_ms=0 repo_ms=0 time_ms=22 unmarshal_ms=22 version_ms=0
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:18.564341261Z time="2020-08-12T01:10:18Z" level=info msg="Normalized app spec: {\"status\":{\"conditions\":[{\"lastTransitionTime\":\"2020-08-12T01:10:07Z\",\"message\":\"CustomResourceDefinition/certificaterequests.cert-manager.io is part of a different application: cert-manager\",\"type\":\"SharedResourceWarning\"},{\"lastTransitionTime\":\"2020-08-12T01:10:17Z\",\"message\":\"CustomResourceDefinition/certificates.cert-manager.io is part of a different application: cert-manager\",\"type\":\"SharedResourceWarning\"},{\"lastTransitionTime\":\"2020-08-12T01:10:17Z\",\"message\":\"CustomResourceDefinition/challenges.acme.cert-manager.io is part of a different application: cert-manager\",\"type\":\"SharedResourceWarning\"},{\"lastTransitionTime\":\"2020-08-12T01:10:17Z\",\"message\":\"CustomResourceDefinition/clusterissuers.cert-manager.io is part of a different application: cert-manager\",\"type\":\"SharedResourceWarning\"},{\"lastTransitionTime\":\"2020-08-12T01:10:17Z\",\"message\":\"CustomResourceDefinition/issuers.cert-manager.io is part of a different application: cert-manager\",\"type\":\"SharedResourceWarning\"},{\"lastTransitionTime\":\"2020-08-12T01:10:17Z\",\"message\":\"CustomResourceDefinition/orders.acme.cert-manager.io is part of a different application: cert-manager\",\"type\":\"SharedResourceWarning\"}]}}" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:20.615699028Z time="2020-08-12T01:10:20Z" level=info msg="Skipping auto-sync: application status is Synced" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:20.639493079Z time="2020-08-12T01:10:20Z" level=info msg="Update successful" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:20.639521759Z time="2020-08-12T01:10:20Z" level=info msg="Reconciliation completed" application=k8s00-dev-sydney-aws-cert-manager dedup_ms=0 dest-namespace=operations dest-server="https://XXXXXXXXXX.sk1.ap-southeast-2.eks.amazonaws.com" diff_ms=154 fields.level=1 git_ms=22 health_ms=2 live_ms=959 settings_ms=0 sync_ms=0 time_ms=3235
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:20.639527388Z time="2020-08-12T01:10:20Z" level=info msg="Refreshing app status (controller refresh requested), level (1)" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:20.639786868Z time="2020-08-12T01:10:20Z" level=info msg="Comparing app state (cluster: https://XXXXXXXXXX.sk1.ap-southeast-2.eks.amazonaws.com, namespace: operations)" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:20.670166067Z time="2020-08-12T01:10:20Z" level=info msg="getRepoObjs stats" application=k8s00-dev-sydney-aws-cert-manager build_options_ms=0 helm_ms=0 plugins_ms=0 repo_ms=0 time_ms=30 unmarshal_ms=30 version_ms=0
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:21.158099190Z time="2020-08-12T01:10:21Z" level=info msg="Normalized app spec: {\"status\":{\"conditions\":[{\"lastTransitionTime\":\"2020-08-12T01:10:07Z\",\"message\":\"CustomResourceDefinition/certificaterequests.cert-manager.io is part of a different application: cert-manager\",\"type\":\"SharedResourceWarning\"},{\"lastTransitionTime\":\"2020-08-12T01:10:20Z\",\"message\":\"CustomResourceDefinition/certificates.cert-manager.io is part of a different application: cert-manager\",\"type\":\"SharedResourceWarning\"},{\"lastTransitionTime\":\"2020-08-12T01:10:20Z\",\"message\":\"CustomResourceDefinition/challenges.acme.cert-manager.io is part of a different application: cert-manager\",\"type\":\"SharedResourceWarning\"},{\"lastTransitionTime\":\"2020-08-12T01:10:20Z\",\"message\":\"CustomResourceDefinition/clusterissuers.cert-manager.io is part of a different application: cert-manager\",\"type\":\"SharedResourceWarning\"},{\"lastTransitionTime\":\"2020-08-12T01:10:20Z\",\"message\":\"CustomResourceDefinition/issuers.cert-manager.io is part of a different application: cert-manager\",\"type\":\"SharedResourceWarning\"},{\"lastTransitionTime\":\"2020-08-12T01:10:20Z\",\"message\":\"CustomResourceDefinition/orders.acme.cert-manager.io is part of a different application: cert-manager\",\"type\":\"SharedResourceWarning\"}]}}" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:22.055215531Z time="2020-08-12T01:10:22Z" level=info msg="Skipping auto-sync: application status is Synced" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:22.076414198Z time="2020-08-12T01:10:22Z" level=info msg="Update successful" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:22.076439300Z time="2020-08-12T01:10:22Z" level=info msg="Reconciliation completed" application=k8s00-dev-sydney-aws-cert-manager dedup_ms=0 dest-namespace=operations dest-server="https://XXXXXXXXXX.sk1.ap-southeast-2.eks.amazonaws.com" diff_ms=148 fields.level=1 git_ms=30 health_ms=2 live_ms=313 settings_ms=0 sync_ms=0 time_ms=1436
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:22.849103413Z time="2020-08-12T01:10:22Z" level=info msg="Refreshing app status (controller refresh requested), level (0)" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:22.919020404Z time="2020-08-12T01:10:22Z" level=info msg="Update successful" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:22.919051092Z time="2020-08-12T01:10:22Z" level=info msg="Reconciliation completed" application=k8s00-dev-sydney-aws-cert-manager dest-namespace=operations dest-server="https://XXXXXXXXXX.sk1.ap-southeast-2.eks.amazonaws.com" fields.level=0 time_ms=82
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:22.919071457Z time="2020-08-12T01:10:22Z" level=info msg="Refreshing app status (controller refresh requested), level (1)" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:22.919321745Z time="2020-08-12T01:10:22Z" level=info msg="Comparing app state (cluster: https://XXXXXXXXXX.sk1.ap-southeast-2.eks.amazonaws.com, namespace: operations)" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:22.945114410Z time="2020-08-12T01:10:22Z" level=info msg="getRepoObjs stats" application=k8s00-dev-sydney-aws-cert-manager build_options_ms=0 helm_ms=0 plugins_ms=0 repo_ms=0 time_ms=23 unmarshal_ms=23 version_ms=0
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:23.469870446Z time="2020-08-12T01:10:23Z" level=info msg="Normalized app spec: {\"status\":{\"conditions\":[{\"lastTransitionTime\":\"2020-08-12T01:10:07Z\",\"message\":\"CustomResourceDefinition/certificaterequests.cert-manager.io is part of a different application: cert-manager\",\"type\":\"SharedResourceWarning\"},{\"lastTransitionTime\":\"2020-08-12T01:10:22Z\",\"message\":\"CustomResourceDefinition/certificates.cert-manager.io is part of a different application: cert-manager\",\"type\":\"SharedResourceWarning\"},{\"lastTransitionTime\":\"2020-08-12T01:10:22Z\",\"message\":\"CustomResourceDefinition/challenges.acme.cert-manager.io is part of a different application: cert-manager\",\"type\":\"SharedResourceWarning\"},{\"lastTransitionTime\":\"2020-08-12T01:10:22Z\",\"message\":\"CustomResourceDefinition/clusterissuers.cert-manager.io is part of a different application: cert-manager\",\"type\":\"SharedResourceWarning\"},{\"lastTransitionTime\":\"2020-08-12T01:10:22Z\",\"message\":\"CustomResourceDefinition/issuers.cert-manager.io is part of a different application: cert-manager\",\"type\":\"SharedResourceWarning\"},{\"lastTransitionTime\":\"2020-08-12T01:10:22Z\",\"message\":\"CustomResourceDefinition/orders.acme.cert-manager.io is part of a different application: cert-manager\",\"type\":\"SharedResourceWarning\"}]}}" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:23.526750133Z time="2020-08-12T01:10:23Z" level=info msg="Skipping auto-sync: application status is Synced" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:23.526782193Z time="2020-08-12T01:10:23Z" level=info msg="Updated health status: Progressing -> Healthy" application=k8s00-dev-sydney-aws-cert-manager dest-namespace=operations dest-server="https://XXXXXXXXXX.sk1.ap-southeast-2.eks.amazonaws.com" reason=ResourceUpdated type=Normal
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:23.552023568Z time="2020-08-12T01:10:23Z" level=info msg="Update successful" application=k8s00-dev-sydney-aws-cert-manager
argocd-application-controller-8458ff6b5-ktc87 argocd-application-controller 2020-08-12T01:10:23.552056379Z time="2020-08-12T01:10:23Z" level=info msg="Reconciliation completed" application=k8s00-dev-sydney-aws-cert-manager dedup_ms=0 dest-namespace=operations dest-server="https://XXXXXXXXXX.sk1.ap-southeast-2.eks.amazonaws.com" diff_ms=197 fields.level=1 git_ms=23 health_ms=2 live_ms=302 settings_ms=0 sync_ms=0 time_ms=632

so something in the 0.16.1 chart is breaking all the things.

jurgenweber on 12 Aug 2020

I noticed the exact same thing when using cert-manager 0.16.1. 😞

cookandy on 21 Aug 2020

👍1

I noticed the exact same thing when using cert-manager 0.16.1. 😞

good to hear I was not going mad! :)

jurgenweber on 21 Aug 2020

we believe this was fixed in #4149. @jurgenweber can you confirm that this is the case?

rbreeze on 26 Aug 2020

sure, is this in 1.7.1? I can look into it today/tomorrow.

jurgenweber on 26 Aug 2020

@jurgenweber yeah, it should be in 1.7.1. Thanks!

rbreeze on 26 Aug 2020

👍1

I am using 1.7.2, and I don't see any obvious issues with cert-manager 0.16.1. I do get those warnings, but I guess they are non-critical, and are reporting inconsistencies in cert-manager manifests.

# other such logs about cert-manager CRDs omitted
CustomResourceDefinition/clusterissuers.cert-manager.io is part of a different application: cert-manager

My application is synced, and UI reports everything green.

charandas on 1 Sep 2020

Hello @charandas , I think you are seeing https://argoproj.github.io/argo-cd/faq/#why-is-my-app-out-of-sync-even-after-syncing . Argo CD uses label app.kubernetes.io/instance to mark application resources and cert-manager CRDs have that label by default. We are planning to switch to annotation, meanwhile workaround is to switch Argo CD to another label using application.instanceLabelKey setting.