Kube-prometheus: how to override nodeSelector?

[tewing-riffyn]

When compiling the manifests using jb, I would like to set a NodeSelector within the prometheus-operator, prometheus, grafana and alertmanager pods. Is there a documented way to do this?

Apologies if this is something I missed in the docs. I'm a jsonnet newb.

[metalmatze]

Here's an example that overwrites the nodeSelector for Prometheus, node-exporter and kube-state-metrics:

diff --git example.jsonnet example.jsonnet
index 7002eff..bd4bd6f 100644
--- example.jsonnet
+++ example.jsonnet
@@ -1,22 +1,47 @@
+local k = import 'ksonnet/ksonnet.beta.4/k.libsonnet';
+
 local kp =
   (import 'kube-prometheus/kube-prometheus.libsonnet') +
   // Uncomment the following imports to enable its patches
   // (import 'kube-prometheus/kube-prometheus-anti-affinity.libsonnet') +
   // (import 'kube-prometheus/kube-prometheus-managed-cluster.libsonnet') +
   // (import 'kube-prometheus/kube-prometheus-node-ports.libsonnet') +
   // (import 'kube-prometheus/kube-prometheus-static-etcd.libsonnet') +
   // (import 'kube-prometheus/kube-prometheus-thanos-sidecar.libsonnet') +
   {
     _config+:: {
       namespace: 'monitoring',
     },
+
+    // As Prometheus is a Custom Resource, we need to use that object's way of overwriting the nodeSelector
+    prometheus+:: {
+      prometheus+: {
+        spec+: {
+          nodeSelector: { 'kubernetes.io/os': 'my-other-operatingsystem' },
+        },
+      },
+    },
+
+    nodeExporter+:: {
+      local daemonset = k.apps.v1.daemonSet,
+
+      daemonset+:
+        daemonset.mixin.spec.template.spec.withNodeSelector({ 'kubernetes.io/os': 'my-other-operatingsystem' }),
+    },
+
+    kubeStateMetrics+:: {
+      local deployment = k.apps.v1.deployment,
+
+      deployment+:
+        deployment.mixin.spec.template.spec.withNodeSelector({ 'kubernetes.io/os': 'my-other-operatingsystem' }),
+    },
   };

 { ['00namespace-' + name]: kp.kubePrometheus[name] for name in std.objectFields(kp.kubePrometheus) } +
 { ['0prometheus-operator-' + name]: kp.prometheusOperator[name] for name in std.objectFields(kp.prometheusOperator) } +
 { ['node-exporter-' + name]: kp.nodeExporter[name] for name in std.objectFields(kp.nodeExporter) } +
 { ['kube-state-metrics-' + name]: kp.kubeStateMetrics[name] for name in std.objectFields(kp.kubeStateMetrics) } +
 { ['alertmanager-' + name]: kp.alertmanager[name] for name in std.objectFields(kp.alertmanager) } +
 { ['prometheus-' + name]: kp.prometheus[name] for name in std.objectFields(kp.prometheus) } +
 { ['prometheus-adapter-' + name]: kp.prometheusAdapter[name] for name in std.objectFields(kp.prometheusAdapter) } +
 { ['grafana-' + name]: kp.grafana[name] for name in std.objectFields(kp.grafana) }

[ryangilbert]

@metalmatze do you happen to have an example for how to do this for alertmanager? I stumbled around but couldn't get it to work.

[kouk]

@ryangilbert this worked for me:

    alertmanager+:: {
      alertmanager+: {
        spec+: {
          nodeSelector: { 'beta.kubernetes.io/os': 'linux' },
        },
      },
    },

[paulfantom]

Closing due to staleness. Please reopen if you think this is still relevant.