Spark-on-k8s-operator: Exposing driver's 8080 port for datadog configuration

[leandro-rouberte]

Hi, everyone.

I'm trying to integrate the spark applications deployed by the operator with datadog monitoring solutions.
Looking into datadog's documentation, it seems that the driver's port 8080 needs to be exposed so datadog's k8s agent can access it and collect metrics from there. https://docs.datadoghq.com/integrations/spark/?tab=containerized

Just want to know if this is possible to do through the operator, otherwise I will fallback to spark 3.0 prometheus integration or JMX exporter. Thanks.

[fbalicchia]

Hi @leandro-rouberte .
Did you try to configure https://github.com/GoogleCloudPlatform/spark-on-k8s-operator/blob/master/charts/spark-operator-chart/templates/deployment.yaml#L26 ?

[leandro-rouberte]

From what I understood this would expose prometheus metrics about spark operator. I would like to expose metrics about drivers, executors, tasks, workers and leader pods, k8s underlying nodes, and so on. By enabling prometheus for Spark 3.x and adding the prometheus annotations to the driver spec, I was able to access some metrics and visualize it through Grafana. But, in that case, I would need to build dashboards with Grafana, while Datadog already has those.

[mcd01]

Did you find a solution for this? If so, do you mind sharing your insights? Thanks in advance!

[luizm]

@mcd01 and @leandro-rouberte there is an PR linked here, that was merged already.. It's gonna help you guys.