Cloud-on-k8s: Configure labels on child resources

[spencergilbert]

Proposal

Allow users to set labels on the "top level" resources created from the CRDs

Use case. Why is this important?
Today users can label the pods created by the ECK operator, but the statefulset for elasticsearch doesn't have any means of being labelled. Presumably the kibana would have the same issue - we use labels in monitoring, logs, generic discoverability, etc.

It'd be very helpful to have specifications in the CRD to apply labels to the deployments/statefulsets created from them.

[spencergilbert]

I suppose also labelling the created services and all would be nice too

[sebgl]

Would it make sense to have all StatefulSets/Deployment inherit the labels applied to the Elasticsearch/Kibana resource?

Example:

apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
  name: quickstart
  labels:
    foo: bar
spec:
  version: 7.6.0
  nodeSets:
  - name: default
    count: 1

would label the underlying StatefulSet with foo: bar. Same for Kibana Deployment. And all other resources: services, secrets, etc.

It feels like an easy way of labeling everything at once. Would that cover your use case @spencergilbert? Or would you expect to be able to specify per-NodeSet/StatefulSet labels?

[spencergilbert]

That would definitely solve my use case :+1:

[charith-elastic]

After prototyping how this would work, I think that it would be better to let users decide which labels or annotations they want to propagate to child objects. This is because some labels/annotations are probably meant for the CR only and having them propagated to children could result in confusion and/or unexpected behaviour (e.g. deleting by label selector).

I am proposing that we introduce two annotations that can be applied to CRs:

eck.k8s.elastic.co/propagate-annotations: "comma,separated,list,of,annotations,to,propagate"
eck.k8s.elastic.co/propagate-labels: "comma,separated,list,of,labels,to,propagate"

Setting the annotation value to * would propagate all annotations/labels in the CR.

[sebgl]

Similar issue for setting headless services labels: https://github.com/elastic/cloud-on-k8s/issues/3990.
As Charith suggested above, I think it makes sense to label all child resources, and not just StatefulSets.

[tschroeder-zendesk]

I am currently unable to use ECK due to this issue as my company requires me to label all resources with my team. It's been open for a long time. Any possibility of doing this soon?