Kiali: Allow custom health configuration to be deployed as resources/yaml or annotations

[erSitzt]

Is your feature request related to a problem? Please describe.

Currently custom health configuration has to be done in the kiali operator CRD or via Helm --set or in the configmap, but custom error rates are much more dependent on the behavior of the services that get deployed and i think it would make more sense to be able to define them there

Describe the solution you'd like
I would like to be able to deploy those configs like any other resource or at least be able to use annotations to set these values directly on a service/virtualservice.

Our services display very different behavoir and error-rates so a general config for a whole namespace would not suffice and i think our devs know best how their services are working and what their "error" baseline is.
This way they could specify thresholds that are used when the services are deployed.
The format of the config looks good to me, only the way to apply those config seems counter intuitive for a feature related to the resources that kiali is displaing/graphing

Describe alternatives you've considered
My only other alternative would be to do namespace wide custom values and explicit ones for services/workloads that are showing more extreme behavior

Additional context
I did not yet implement something like this, so i dont know how much work/complexity it adds to read/validate configs from CRs
And i think it might be a lot to put in annotations... but i have seen good use of annotation for other tools, so this was the first think that came to mind :)

I would be happy if this could be considered for an already great tool :)

[lucasponce]

Thanks for the feedback @erSitzt

When we started the Health Config feature [1] [2] we know that probably the way config is defined may evolve.

I like the proposal to use annotations rather than custom CRDs for this.

Perhaps CRDs would be more canonical, but adding new CRDs to maintain increases complexity in other places, so having the possibility to use annotation (in Services and/or Workloads) may be a good point to explore.

I'll also cc @aljesusg which is one of the original authors of this work as well.

[1] https://kiali.io/documentation/latest/features/#_health_configuration
[2] https://kiali.io/documentation/latest/configuration/health/

[jotak]

+1 for both the feature request (decentralized health config), and @lucasponce 's comment that CRD adds complexity in the installation / update processes.
So going through annotations worth the try.
Let's just try to not make it too complex, and make it easy to reason about (ideally I would expect regex in config only when it's necessary).

[jmazzitelli]

+1 on CRDs make things more complicated and +1 on trying annotations first.

My contribution to this thread would be to suggest we keep the current way we do it (with the config) but allow annotations to OVERRIDE the config (hopefully that would be possible).

Supporting that would allow people to still be able to provide a single-point of configuration if they want but allow them to spread the config out to the individual resources (via annotations to override the config) if they would prefer to do it that way. You could use both if you want (one main config for most of your apps, but if some app behaves differently than all the rest, you could add annotations to that app's resources).