Thanos: Thanos deployment with helm

[arthur-c]

I forked the stable/prometheus helm chart and added options to deploy thanos sidecar along prometheus and peers service as well.

https://github.com/arthur-c/charts/tree/prometheus-thanos/stable/prometheus

It's not battle-tested and I'm not a thanos expert but it works for me at the moment.

What should I do with that ? It could be useful to someone else.

• propose a patch to stable/prometheus ? I could understand why they would refuse, thanos is not part of Prometheus project.
• propose a prometheus-thanos chart ? (a maintained fork of stable/prometheus)
• propose to store the chart in improbable-eng/thanos (a maintained fork of stable/prometheus)
• a dedicated github repo (a maintained fork of stable/prometheus)

I plan to create charts for store, query, compactor, rule... as well.

tests/reviews are welcomed, especially about default options.

[bwplotka]

Thanks for this. I am not super familiar with helm that much to tell where usually charts are stored. Maybe @joshpmcghee can help here? But as far as I understand this, why not proposing to store it here: https://github.com/helm/charts/tree/master/stable?

And does this chart really need to be a fork of stable/prometheus? Why not proposing separate thanos chart here: https://github.com/helm/charts/tree/master/stable Unless by fork you mean something else than just github fork.

[arthur-c]

I contacted the stable/prometheus owners and they are ok for a PR on the main chart. So that's a good news. I'll wait a bit as I prefer to get other testers before patching the main chart.

I included the option to deploy thanos peers service. My idea: if the chart is deployed multiple times on a single cluster, only one prometheus replica would enable the peers service.

[arthur-c]

And an even better option from @gianrubio (prom chart maintainer):

Another way to push this PR is to create a new chart prometheus-thanos and use prometheus as a requirement

[bwplotka]

This is exactly what I was thinking.. to not do fork (: Nice one!

[bwplotka]

I included the option to deploy thanos peers service. My idea: if the chart is deployed multiple times on a single cluster, only one prometheus replica would enable the peers service.

what if you have 2 Prometheus replicas? Each will start peer service?

[arthur-c]

the service is a declarative option, you only have to enable it one time on one deployment.

But I can create a specific chart for the peer service as well.

[nrshrivatsan]

@arthur-c did you get a chance to create a PR for this issue?

It would be fantastic to have a read-to-use helm spec for Thanos, given Kubernetes's adoption & Prometheus graduating in CNCF.

[arthur-c]

hi @nrshrivatsan, I have a working chart in my fork (https://github.com/arthur-c/charts/tree/prometheus-thanos) at the moment.

I did not give more time to try to upstream a thanos chart as I am a bit stuck to find an elegant solution. Any help from a more experienced helm packager is welcomed.

[mvisonneau]

FYI, I believe that the chart is about to be replaced with the coreos one :
https://github.com/coreos/prometheus-operator/issues/592

[tarokkk]

We've been working on our Thanos solution (for customers as well). So we decided to maintain a non-official Thanos chart: https://github.com/banzaicloud/banzai-charts/tree/master/thanos This solution works with Prometheus operator.

[kapilgarg83]

I forked the stable/prometheus helm chart and added options to deploy thanos sidecar along prometheus and peers service as well.

https://github.com/arthur-c/charts/tree/prometheus-thanos/stable/prometheus

It's not battle-tested and I'm not a thanos expert but it works for me at the moment.

What should I do with that ? It could be useful to someone else.

• propose a patch to stable/prometheus ? I could understand why they would refuse, thanos is not part of Prometheus project.
• propose a prometheus-thanos chart ? (a maintained fork of stable/prometheus)
• propose to store the chart in improbable-eng/thanos (a maintained fork of stable/prometheus)
• a dedicated github repo (a maintained fork of stable/prometheus)

I plan to create charts for store, query, compactor, rule... as well.

tests/reviews are welcomed, especially about default options.

where to add s3 sccess and secret key..??

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.