Argo-cd: Using ArgoCD with Rancher makes Rancher endpoints slow

[daanemanz]

Describe the bug
We have a setup where we set up, manage, and proxy our Kubernetes clusters through Rancher. Two of our clusters run on GKE and are currently deployed to by ArgoCD, and as soon as we sync those clusters with ArgoCD, any kubectl command run against the Rancher endpoint for that cluster is very slow or just times out. ArgoCD syncs will time out as well.

When we use the GKE endpoints instead, kubectl will be very snappy and ArgoCD will sync properly, all without any timeouts.

To Reproduce
Steps to reproduce the behavior:

1. Set up a cluster (GKE) in Rancher
2. Set up ArgoCD
3. Configure an app in ArgoCD that syncs to the Rancher endpoint for the cluster you just set up
4. Try running kubectl get nodes against the Rancher endpoint

Then:

1. Fetch the GKE endpoint and credentials from GKE
2. Configure the app in ArgoCD to sync to this endpoint instead
3. Run kubectl get nodes against the GKE endpoint; it should return very quickly

Expected behavior
When using the Rancher provided endpoint, app syncs don't time out, and kubectl returns within seconds (or at least not 2 minutes or timeout).

Screenshots
N/A

Additional context
N/A

[daanemanz]

As a side note, the last three steps in the To Reproduce section can probably be tried with any kind of k8s cluster.

[daanemanz]

@alexmt - does ArgoCD track resources that are _not_ explicitly defined in my kustomize repo?

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[Frintrop]

Is argocd on rancher still slow ?

[daanemanz]

@Frintrop we stopped pointing ArgoCD at the Rancher endpoints, and put an ArgoCD deployment in every cluster we manage instead. That seems to work a lot better for us, so we haven't really looked back.

[Linutux]

Does anyone have new information?