Argo-cd: Manage Argo CD Using Argo CD Reconciliation loop

[KarstenSiemer]

Describe the bug
Apparently i've somehow build a reconciliation loop by managing argocd using argocd.

argocd-application-controller-65d85974d-t9kwx argocd-application-controller time="2019-04-26T10:53:29Z" level=info msg="Comparing app state (cluster: https://kubernetes.default.svc, namespace: system-cicd)" application=argo-cd
argocd-application-controller-65d85974d-t9kwx argocd-application-controller time="2019-04-26T10:53:29Z" level=info msg="Update successful" application=argo-cd
argocd-application-controller-65d85974d-t9kwx argocd-application-controller time="2019-04-26T10:53:29Z" level=info msg="Reconciliation completed" application=argo-cd full=true time_ms=189.357799
argocd-application-controller-65d85974d-t9kwx argocd-application-controller time="2019-04-26T10:53:29Z" level=info msg="Refreshing app status (controller refresh requested)" application=argo-cd
argocd-server-7dc6666d45-mk8lp argocd-server time="2019-04-26T10:53:29Z" level=info msg="received unary call /application.ApplicationService/ResourceTree" grpc.method=ResourceTree grpc.request.claims="{\"acr\":\"0\",\"aud\":\"cd.svc.xxx.de\",\"auth_time\":1556267652,\"azp\":\"cd.svc.xxx.de\",\"email\":\"[email protected]\",\"exp\":1556276350,\"groups\":[\"/teams/black-ops\",\"/orgs/engineering\",\"/teams/operations\",\"/k8s/cluster/admins\"],\"iat\":1556275450,\"iss\":\"https://id.domain.net/auth/realms/xxx\",\"jti\":\"xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx\",\"nbf\":0,\"nonce\":\"xxxxxxxxx\",\"session_state\":\"xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx\",\"sub\":\"xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx\",\"typ\":\"ID\"}" grpc.request.content="applicationName:\"argo-cd\" " grpc.service=application.ApplicationService grpc.start_time="2019-04-26T10:53:29Z" span.kind=server system=grpc
argocd-server-7dc6666d45-mk8lp argocd-server time="2019-04-26T10:53:29Z" level=info msg="finished unary call with code OK" grpc.code=OK grpc.method=ResourceTree grpc.service=application.ApplicationService grpc.start_time="2019-04-26T10:53:29Z" grpc.time_ms=6.115 span.kind=server system=grpc
argocd-application-controller-65d85974d-t9kwx argocd-application-controller W0426 10:53:29.569909       1 listers.go:63] can not retrieve list of objects using index : Index with name namespace does not exist
argocd-application-controller-65d85974d-t9kwx argocd-application-controller time="2019-04-26T10:53:29Z" level=info msg="Comparing app state (cluster: https://kubernetes.default.svc, namespace: system-cicd)" application=argo-cd
argocd-application-controller-65d85974d-t9kwx argocd-application-controller time="2019-04-26T10:53:29Z" level=info msg="Update successful" application=argo-cd
argocd-application-controller-65d85974d-t9kwx argocd-application-controller time="2019-04-26T10:53:29Z" level=info msg="Reconciliation completed" application=argo-cd full=true time_ms=201.867236
argocd-application-controller-65d85974d-t9kwx argocd-application-controller time="2019-04-26T10:53:29Z" level=info msg="Refreshing app status (controller refresh requested)" application=argo-cd
argocd-server-7dc6666d45-mk8lp argocd-server time="2019-04-26T10:53:29Z" level=info msg="received unary call /application.ApplicationService/ResourceTree" grpc.method=ResourceTree grpc.request.claims="{\"acr\":\"0\",\"aud\":\"cd.svc.xxx.de\",\"auth_time\":1556267652,\"azp\":\"cd.svc.xxx.de\",\"email\":\"[email protected]\",\"exp\":1556276350,\"groups\":[\"/teams/black-ops\",\"/orgs/engineering\",\"/teams/operations\",\"/k8s/cluster/admins\"],\"iat\":1556275450,\"iss\":\"https://id.domain.net/auth/realms/xxx\",\"jti\":\"xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx\",\"nbf\":0,\"nonce\":\"xxxxxxxxx\",\"session_state\":\"xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx\",\"sub\":\"xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx\",\"typ\":\"ID\"}" grpc.request.content="applicationName:\"argo-cd\" " grpc.service=application.ApplicationService grpc.start_time="2019-04-26T10:53:29Z" span.kind=server system=grpc
argocd-server-7dc6666d45-mk8lp argocd-server time="2019-04-26T10:53:29Z" level=info msg="finished unary call with code OK" grpc.code=OK grpc.method=ResourceTree grpc.service=application.ApplicationService grpc.start_time="2019-04-26T10:53:29Z" grpc.time_ms=6.487 span.kind=server system=grpc
argocd-application-controller-65d85974d-t9kwx argocd-application-controller W0426 10:53:29.772895       1 listers.go:63] can not retrieve list of objects using index : Index with name namespace does not exist
argocd-application-controller-65d85974d-t9kwx argocd-application-controller time="2019-04-26T10:53:29Z" level=info msg="Comparing app state (cluster: https://kubernetes.default.svc, namespace: system-cicd)" application=argo-cd
argocd-application-controller-65d85974d-t9kwx argocd-application-controller time="2019-04-26T10:53:29Z" level=info msg="Update successful" application=argo-cd
argocd-application-controller-65d85974d-t9kwx argocd-application-controller time="2019-04-26T10:53:29Z" level=info msg="Reconciliation completed" application=argo-cd full=true time_ms=187.965437
argocd-application-controller-65d85974d-t9kwx argocd-application-controller time="2019-04-26T10:53:29Z" level=info msg="Refreshing app status (controller refresh requested)" application=argo-cd
argocd-server-7dc6666d45-mk8lp argocd-server time="2019-04-26T10:53:29Z" level=info msg="received unary call /application.ApplicationService/ResourceTree" grpc.method=ResourceTree grpc.request.claims="{\"acr\":\"0\",\"aud\":\"cd.svc.xxx.de\",\"auth_time\":1556267652,\"azp\":\"cd.svc.xxx.de\",\"email\":\"[email protected]\",\"exp\":1556276350,\"groups\":[\"/teams/black-ops\",\"/orgs/engineering\",\"/teams/operations\",\"/k8s/cluster/admins\"],\"iat\":1556275450,\"iss\":\"https://id.domain.net/auth/realms/xxx\",\"jti\":\"xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx\",\"nbf\":0,\"nonce\":\"xxxxxxxxx\",\"session_state\":\"xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx\",\"sub\":\"xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx\",\"typ\":\"ID\"}" grpc.request.content="applicationName:\"argo-cd\" " grpc.service=application.ApplicationService grpc.start_time="2019-04-26T10:53:29Z" span.kind=server system=grpc
argocd-server-7dc6666d45-mk8lp argocd-server time="2019-04-26T10:53:29Z" level=info msg="finished unary call with code OK" grpc.code=OK grpc.method=ResourceTree grpc.service=application.ApplicationService grpc.start_time="2019-04-26T10:53:29Z" grpc.time_ms=5.521 span.kind=server system=grpc
argocd-application-controller-65d85974d-t9kwx argocd-application-controller W0426 10:53:29.960502       1 listers.go:63] can not retrieve list of objects using index : Index with name namespace does not exist
argocd-application-controller-65d85974d-t9kwx argocd-application-controller time="2019-04-26T10:53:29Z" level=info msg="Comparing app state (cluster: https://kubernetes.default.svc, namespace: system-cicd)" application=argo-cd
argocd-application-controller-65d85974d-t9kwx argocd-application-controller time="2019-04-26T10:53:30Z" level=info msg="Update successful" application=argo-cd
argocd-application-controller-65d85974d-t9kwx argocd-application-controller time="2019-04-26T10:53:30Z" level=info msg="Reconciliation completed" application=argo-cd full=true time_ms=183.817583
argocd-application-controller-65d85974d-t9kwx argocd-application-controller time="2019-04-26T10:53:30Z" level=info msg="Refreshing app status (controller refresh requested)" application=argo-cd
argocd-server-7dc6666d45-mk8lp argocd-server time="2019-04-26T10:53:30Z" level=info msg="received unary call /application.ApplicationService/ResourceTree" grpc.method=ResourceTree grpc.request.claims="{\"acr\":\"0\",\"aud\":\"cd.svc.xxx.de\",\"auth_time\":1556267652,\"azp\":\"cd.svc.xxx.de\",\"email\":\"[email protected]\",\"exp\":1556276350,\"groups\":[\"/teams/black-ops\",\"/orgs/engineering\",\"/teams/operations\",\"/k8s/cluster/admins\"],\"iat\":1556275450,\"iss\":\"https://id.domain.net/auth/realms/xxx\",\"jti\":\"xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx\",\"nbf\":0,\"nonce\":\"xxxxxxxxx\",\"session_state\":\"xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx\",\"sub\":\"xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx\",\"typ\":\"ID\"}" grpc.request.content="applicationName:\"argo-cd\" " grpc.service=application.ApplicationService grpc.start_time="2019-04-26T10:53:30Z" span.kind=server system=grpc
argocd-server-7dc6666d45-mk8lp argocd-server time="2019-04-26T10:53:30Z" level=info msg="finished unary call with code OK" grpc.code=OK grpc.method=ResourceTree grpc.service=application.ApplicationService grpc.start_time="2019-04-26T10:53:30Z" grpc.time_ms=26.681 span.kind=server system=grpc
argocd-application-controller-65d85974d-t9kwx argocd-application-controller W0426 10:53:30.153956       1 listers.go:63] can not retrieve list of objects using index : Index with name namespace does not exist
argocd-application-controller-65d85974d-t9kwx argocd-application-controller time="2019-04-26T10:53:30Z" level=info msg="Comparing app state (cluster: https://kubernetes.default.svc, namespace: system-cicd)" application=argo-cd
argocd-application-controller-65d85974d-t9kwx argocd-application-controller time="2019-04-26T10:53:30Z" level=info msg="Update successful" application=argo-cd
argocd-application-controller-65d85974d-t9kwx argocd-application-controller time="2019-04-26T10:53:30Z" level=info msg="Reconciliation completed" application=argo-cd full=true time_ms=190.51399899999998
argocd-application-controller-65d85974d-t9kwx argocd-application-controller time="2019-04-26T10:53:30Z" level=info msg="Refreshing app status (controller refresh requested)" application=argo-cd
argocd-application-controller-65d85974d-t9kwx argocd-application-controller W0426 10:53:30.338137       1 listers.go:63] can not retrieve list of objects using index : Index with name namespace does not exist
argocd-application-controller-65d85974d-t9kwx argocd-application-controller time="2019-04-26T10:53:30Z" level=info msg="Comparing app state (cluster: https://kubernetes.default.svc, namespace: system-cicd)" application=argo-cd
argocd-server-7dc6666d45-mk8lp argocd-server time="2019-04-26T10:53:30Z" level=info msg="received unary call /application.ApplicationService/ResourceTree" grpc.method=ResourceTree grpc.request.claims="{\"acr\":\"0\",\"aud\":\"cd.svc.xxx.de\",\"auth_time\":1556267652,\"azp\":\"cd.svc.xxx.de\",\"email\":\"[email protected]\",\"exp\":1556276350,\"groups\":[\"/teams/black-ops\",\"/orgs/engineering\",\"/teams/operations\",\"/k8s/cluster/admins\"],\"iat\":1556275450,\"iss\":\"https://id.domain.net/auth/realms/xxx\",\"jti\":\"xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx\",\"nbf\":0,\"nonce\":\"xxxxxxxxx\",\"session_state\":\"xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx\",\"sub\":\"xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx\",\"typ\":\"ID\"}" grpc.request.content="applicationName:\"argo-cd\" " grpc.service=application.ApplicationService grpc.start_time="2019-04-26T10:53:30Z" span.kind=server system=grpc
argocd-server-7dc6666d45-mk8lp argocd-server time="2019-04-26T10:53:30Z" level=info msg="finished unary call with code OK" grpc.code=OK grpc.method=ResourceTree grpc.service=application.ApplicationService grpc.start_time="2019-04-26T10:53:30Z" grpc.time_ms=6.126 span.kind=server system=grpc
argocd-application-controller-65d85974d-t9kwx argocd-application-controller time="2019-04-26T10:53:30Z" level=info msg="Update successful" application=argo-cd
argocd-application-controller-65d85974d-t9kwx argocd-application-controller time="2019-04-26T10:53:30Z" level=info msg="Reconciliation completed" application=argo-cd full=true time_ms=207.910877
argocd-application-controller-65d85974d-t9kwx argocd-application-controller time="2019-04-26T10:53:30Z" level=info msg="Refreshing app status (controller refresh requested)" application=argo-cd
argocd-server-7dc6666d45-mk8lp argocd-server time="2019-04-26T10:53:30Z" level=info msg="received unary call /application.ApplicationService/ResourceTree" grpc.method=ResourceTree grpc.request.claims="{\"acr\":\"0\",\"aud\":\"cd.svc.xxx.de\",\"auth_time\":1556267652,\"azp\":\"cd.svc.xxx.de\",\"email\":\"[email protected]\",\"exp\":1556276350,\"groups\":[\"/teams/black-ops\",\"/orgs/engineering\",\"/teams/operations\",\"/k8s/cluster/admins\"],\"iat\":1556275450,\"iss\":\"https://id.domain.net/auth/realms/xxx\",\"jti\":\"xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx\",\"nbf\":0,\"nonce\":\"xxxxxxxxx\",\"session_state\":\"xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx\",\"sub\":\"xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx\",\"typ\":\"ID\"}" grpc.request.content="applicationName:\"argo-cd\" " grpc.service=application.ApplicationService grpc.start_time="2019-04-26T10:53:30Z" span.kind=server system=grpc
argocd-server-7dc6666d45-mk8lp argocd-server time="2019-04-26T10:53:30Z" level=info msg="finished unary call with code OK" grpc.code=OK grpc.method=ResourceTree grpc.service=application.ApplicationService grpc.start_time="2019-04-26T10:53:30Z" grpc.time_ms=5.991 span.kind=server system=grpc
argocd-application-controller-65d85974d-t9kwx argocd-application-controller W0426 10:53:30.547050       1 listers.go:63] can not retrieve list of objects using index : Index with name namespace does not exist
argocd-application-controller-65d85974d-t9kwx argocd-application-controller time="2019-04-26T10:53:30Z" level=info msg="Comparing app state (cluster: https://kubernetes.default.svc, namespace: system-cicd)" application=argo-cd
argocd-application-controller-65d85974d-t9kwx argocd-application-controller time="2019-04-26T10:53:30Z" level=info msg="Update successful" application=argo-cd
argocd-application-controller-65d85974d-t9kwx argocd-application-controller time="2019-04-26T10:53:30Z" level=info msg="Reconciliation completed" application=argo-cd full=true time_ms=186.86920700000002
argocd-application-controller-65d85974d-t9kwx argocd-application-controller time="2019-04-26T10:53:30Z" level=info msg="Refreshing app status (controller refresh requested)" application=argo-cd
argocd-application-controller-65d85974d-t9kwx argocd-application-controller W0426 10:53:30.738720       1 listers.go:63] can not retrieve list of objects using index : Index with name namespace does not exist
argocd-application-controller-65d85974d-t9kwx argocd-application-controller time="2019-04-26T10:53:30Z" level=info msg="Comparing app state (cluster: https://kubernetes.default.svc, namespace: system-cicd)" application=argo-cd
argocd-server-7dc6666d45-mk8lp argocd-server time="2019-04-26T10:53:30Z" level=info msg="received unary call /application.ApplicationService/ResourceTree" grpc.method=ResourceTree grpc.request.claims="{\"acr\":\"0\",\"aud\":\"cd.svc.xxx.de\",\"auth_time\":1556267652,\"azp\":\"cd.svc.xxx.de\",\"email\":\"[email protected]\",\"exp\":1556276350,\"groups\":[\"/teams/black-ops\",\"/orgs/engineering\",\"/teams/operations\",\"/k8s/cluster/admins\"],\"iat\":1556275450,\"iss\":\"https://id.domain.net/auth/realms/xxx\",\"jti\":\"xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx\",\"nbf\":0,\"nonce\":\"xxxxxxxxx\",\"session_state\":\"xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx\",\"sub\":\"xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx\",\"typ\":\"ID\"}" grpc.request.content="applicationName:\"argo-cd\" " grpc.service=application.ApplicationService grpc.start_time="2019-04-26T10:53:30Z" span.kind=server system=grpc
argocd-server-7dc6666d45-mk8lp argocd-server time="2019-04-26T10:53:30Z" level=info msg="finished unary call with code OK" grpc.code=OK grpc.method=ResourceTree grpc.service=application.ApplicationService grpc.start_time="2019-04-26T10:53:30Z" grpc.time_ms=9.8 span.kind=server system=grpc
argocd-application-controller-65d85974d-t9kwx argocd-application-controller time="2019-04-26T10:53:30Z" level=info msg="Update successful" application=argo-cd
argocd-application-controller-65d85974d-t9kwx argocd-application-controller time="2019-04-26T10:53:30Z" level=info msg="Reconciliation completed" application=argo-cd full=true time_ms=190.88876299999998
argocd-application-controller-65d85974d-t9kwx argocd-application-controller time="2019-04-26T10:53:30Z" level=info msg="Refreshing app status (controller refresh requested)" application=argo-cd
argocd-application-controller-65d85974d-t9kwx argocd-application-controller W0426 10:53:30.927467       1 listers.go:63] can not retrieve list of objects using index : Index with name namespace does not exist
argocd-application-controller-65d85974d-t9kwx argocd-application-controller time="2019-04-26T10:53:30Z" level=info msg="Comparing app state (cluster: https://kubernetes.default.svc, namespace: system-cicd)" application=argo-cd
argocd-server-7dc6666d45-mk8lp argocd-server time="2019-04-26T10:53:30Z" level=info msg="received unary call /application.ApplicationService/ResourceTree" grpc.method=ResourceTree grpc.request.claims="{\"acr\":\"0\",\"aud\":\"cd.svc.xxx.de\",\"auth_time\":1556267652,\"azp\":\"cd.svc.xxx.de\",\"email\":\"[email protected]\",\"exp\":1556276350,\"groups\":[\"/teams/black-ops\",\"/orgs/engineering\",\"/teams/operations\",\"/k8s/cluster/admins\"],\"iat\":1556275450,\"iss\":\"https://id.domain.net/auth/realms/xxx\",\"jti\":\"xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx\",\"nbf\":0,\"nonce\":\"xxxxxxxxx\",\"session_state\":\"xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx\",\"sub\":\"xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx\",\"typ\":\"ID\"}" grpc.request.content="applicationName:\"argo-cd\" " grpc.service=application.ApplicationService grpc.start_time="2019-04-26T10:53:30Z" span.kind=server system=grpc
argocd-server-7dc6666d45-mk8lp argocd-server time="2019-04-26T10:53:30Z" level=info msg="finished unary call with code OK" grpc.code=OK grpc.method=ResourceTree grpc.service=application.ApplicationService grpc.start_time="2019-04-26T10:53:30Z" grpc.time_ms=7.142 span.kind=server system=grpc

As it can be seen here, the controller constantly requests a refresh of the argo-cd application.
Even though it is already healthy and in sync. The auto sync feature isn't even on.

Screenshots

The argo-cd application is owned by the system-cicd appproject.
The argocd-apps application is owned by argocd-objects appproject and used to create applications that are lying inside git.
The argocd-projects application is also owned by argocd-objects and used to create appprojects that are lying inside git.

I do not see any reason why it would be in a loop.
Here you can see, that it is trying to reconcile very often:

argocd-application-controller-65d85974d-t9kwx argocd-application-controller time="2019-04-26T10:45:51Z" level=info msg="Reconciliation completed" application=argo-cd full=true time_ms=186.208233
argocd-application-controller-65d85974d-t9kwx argocd-application-controller time="2019-04-26T10:45:51Z" level=info msg="Reconciliation completed" application=argo-cd full=true time_ms=196.083342
argocd-application-controller-65d85974d-t9kwx argocd-application-controller time="2019-04-26T10:45:52Z" level=info msg="Reconciliation completed" application=argo-cd full=true time_ms=192.908786
argocd-application-controller-65d85974d-t9kwx argocd-application-controller time="2019-04-26T10:45:52Z" level=info msg="Reconciliation completed" application=argo-cd full=true time_ms=192.06886
argocd-application-controller-65d85974d-t9kwx argocd-application-controller time="2019-04-26T10:45:52Z" level=info msg="Reconciliation completed" application=argo-cd full=true time_ms=205.887309
argocd-application-controller-65d85974d-t9kwx argocd-application-controller time="2019-04-26T10:45:52Z" level=info msg="Reconciliation completed" application=argo-cd full=true time_ms=196.757855

I am also unsure what can not retrieve list of objects using index : Index with name namespace does not exist means.
Since argo-cd works wonderfully on all applications that i've passed to it.

Versions:
argo-cd: v1.0.0rc0
kubernetes: v1.14.1

I am having this problem for a few releases now. I am not sure when it started, but in version v0.10.3 this was working just fine

[KarstenSiemer]

I can't find why it is happening. And thus cannot use that feature, which is quite sad, since it is actually the most important feature and why i use argocd

[Oded-B]

We had the same issue when trying to have an ArgoCD application manage all application including the aforementioned application.
We had the same looped arrow in the UI and the same reconciliation loop.
Once we removed the recursive configuration, so we manage all other application declaratively, and the argocd configuration application manually, the reconciliation loop stopped:

[alexec]

After some discussion, we're not sure if this is a feature of a bug. Certainly, people have Argo CD self-manage, but due to how the plumbing now works inside it, this will result in a reconciliation loop.

There's no simple fix to this AFAIK.

This workaround is to manage Argo CD outside of itself, be that manually or with another Argo CD instance.

[alexmt]

There's a fix. It is just not trivial :)

[alexmt]

Trying to find simple way to improve it.

[alexmt]

Looks like it is broken again. See https://github.com/argoproj/argo-cd/pull/1547#issuecomment-498996076

[alexmt]

Hello @KarstenSiemer , finally got back to this issue, but unfortunately cannot reproduce it. Implemented e2e test which creates self-referenced app and makes sure there is not reconciliation loop: https://github.com/argoproj/argo-cd/pull/1724

PR deploys app which is pointing to itself. Can you please check if I'm missing something?

[alexmt]

@KarstenSiemer can you please let us know if you are still able to reproduce it?

[KarstenSiemer]

@alexmt is #1684 included in the new rc image, i see the commit is in the history for the 1.1.0-rc5
but was the image definitely built with it included?
We are currently running a custom build image with the commit included

If it is inside, i will gladly retry :-)

[KarstenSiemer]

okay i've just updated everything is looking great. I think this issue is now resolved.

[KarstenSiemer]

Nope, i have to reopen.
This was gone when i first created the app for self reconciliation. But the problem is back after some time:

here the red line is actually two lines, one is the argo-cd self reconciling app the other one is the seeder application which is pulling applications from git and also pulls the argo-cd app

hence the green line being twice the amount of the red one

it feels less bad then before but definitely destroys argo-cd in a way of having the two apps extremely busy so that nothing gets rolled out any more

[KarstenSiemer]

We had the same issue when trying to have an ArgoCD application manage _all_ application including the aforementioned application.
We had the same looped arrow in the UI and the same reconciliation loop.
Once we removed the recursive configuration, so we manage all other application declaratively, and the argocd configuration application manually, the reconciliation loop stopped:

He is correct, if the self reconciling app isn't being pulled by the seeding application, then there is no reconciliation loop.
Last time i created it by hand and merged it to git in seeding directory at about the same time, which is why the problem wasn't there at first but reappeared.

The last thing that is a problem with seeding applications is the heredity of application state and that argocd doesn't do another reconciliation as long as an app is in progressing state.

When a seeded application becomes progressing, the seeder will be in progressing state, too.
Our seeder now governs around 80 applications, there is one in progressing state almost always.
That is hindering updates for all applications.
Can this be prevented?
The selfhealing starting from 1.2 would definitely help with this i think. But they'd still remain in progressing

[v-gu]

some app use configmap map as leader-election helper, like cert-manager, nginx-ingress, this will trigger ArgoCD reconciliations, those configmap usually are un-managed resources.

[yann-soubeyrand]

Hello,

We're hit by this issue too. We have app of apps (with the root of them managing itself), an app which manages Argo CD, and apps with leader election config maps or lock config maps. That is, we collection all the pattern which seem to lead to high reconciliation rate and CPU usage (Argo CD uses a full CPU)!