Argo-cd: Make a SyncFailure hook available

[arnarg]

It would be nice to have a hook available on sync failure (when it reaches degraded state).
According to the documentation PostSync only runs on successful sync.

Executes after all Sync hooks completed and were successful, a succcessful apply, and all resources in a Healthy state.

This could be useful to notify you that a deployment failed when automatic sync is turned on.

[jessesuen]

I agree this would be useful, this was considered but never implemented (mostly due to time, and waiting on demand). Even have a comment about this in the code:

https://github.com/argoproj/argo-cd/blob/a70cab7ac19dda57e5fadab2110f540f68323483/pkg/apis/application/v1alpha1/types.go#L246-L249

Now that there's interest, I think we can plan for it.

[viggeh]

@jessesuen After a quick glance at the code, is this a case of:

1. Adding a new HookType of type SyncFail
2. Adding a sync on hooks with with Type SyncFail if the Health of the application is HealthStatusDegraded in https://github.com/argoproj/argo-cd/blob/cf486a480e3f915c4527c6f0f5c046a1dfbcd9ff/controller/sync_hooks.go#L55
   ?

I'm trying to get an idea of the scope of work needed.

[viggeh]

On a related note, there might be a case for having a _SyncTimeout_ hook where, if the application is in Progressing for some (configurable) time, a hook is triggered.

[alexmt]

The health status changes might be handled externally. Instead of forcing to implement handler manually the functionality might be encapsulated in CRD: https://github.com/argoproj/argo-cd/issues/1431 @viggeh , what do you think?

[alexec]

@viggeh did you make any progress on this please?

[alexec]

Hmmm. Should HookTypeSyncFailed run before HookTypePostSync or after?

[alexec]

Run on dry-run?

[alexec]

Key point here, in Argo CD terminology Sync Failed is not the same as app becomes degraded. A sync can be successful, but the app still becomes degraded.

Key question is therefore, what do we actually want? What are the uses cases?

[arnarg]

I started to implement this and I voiced some of these questions in your slack channel. We were always thinking of this feature for alerting us when something goes wrong with an auto-syncing application. However, we later realized that having Prometheus scrape ArgoCD and alert on all kinds of health and sync states would be a lot better as that tool is designed for this purpose.

As far as we're concerned (me and @viggeh ) this can be closed.

[simster7]

@alexec @jessesuen @alexmt If there is still interest/intention in implementing this feature I'd be glad to work on it. Let me know

[alexec]

This is a good issue for community contribution.

[simster7]

I'll get started on this!

[alexec]

Fixed for v1.2