Jaeger: Kubernetes Operator and changes to the Helm chart

[jpkrohling]

Requirement - what kind of business use case are you trying to solve?

Historically, it's been hard to provide a Helm chart for Jaeger with support for the all-in-one image. Using an Operator, the deployment is done via code (Go, usually), making it possible to have complex deployment scenarios described in a simple "Custom Resource". For instance, these could be examples of such resources:

apiVersion: jaegertracing.io/v1alpha1
kind: Jaeger
deployment: all-in-one
storage: memory

```yaml
apiVersion: jaegertracing.io/v1alpha1
kind: Jaeger
deployment: all-in-one
storage: cassandra

```yaml
apiVersion: jaegertracing.io/v1alpha1
kind: Jaeger
deployment: production
storage: elasticsearch

A Helm chart would still be provided, to install the operator (custom resource definition) and to load the YAMLs like the ones above (custom resource).

This issue is about:

• [x] Creating the operator under jaegertracing/jaeger-operator
• [ ] Changing the current Helm chart to use the new operator

cc @dvonthenen, @mikelorant, @pavelnikolov

[jpkrohling]

@dvonthenen, we talked briefly about this during the last bi-weekly meeting. IIRC, you agreed that this would be a good idea, but would be nice if you could share your thoughts here as well.

@mikelorant, @pavelnikolov: you are also listed as maintainers on the helm chart, so, I'd also like to hear from you on this.

[pavelnikolov]

This is a very interesting idea. I'm currently working on the all-in-one default for the helm chart and plan to send a PR soon. I'm sorry for the delay.
I'm not 100% sure that the operator + CRDs are worth the effort. Is there any scenario that the existing helm chart (plust the all-in-one as default option) will not be able to deliver?

[jpkrohling]

If the all-in-one scenario can be covered by the current helm chart, then I think there's nothing a basic operator can do that a helm chart can't.

For the future, it would be nice to use the monitoring features of the Operator, so that we can take actions based on that, but it's not something we should aim for at first.

[pavelnikolov]

I'll get back to you ASAP with updates on the all-in-one chart update. It looks like a low-hanging fruit. I'm very interested in the monitoring features that you have in mind : )

[jpkrohling]

monitoring features that you have in mind

I mostly have the lifecycle manager in mind (https://github.com/operator-framework/operator-lifecycle-manager), but we don't have much use for that right now.

A couple of things we could do with operators easier than with charts are:

• Cassandra schema updates
• Automatic migration whenever we introduce a breaking change (like we just did with the Docker images for 1.6)

[dvonthenen]

@jpkrohling Yea, that's exactly why operators are so powerful. It really isn't the deployment aspects of your application (although that is obviously helpful) because simple deployment mostly something that can be covered using charts today.

The real benefit of operators are the "day 2" activities that applications create/need/etc. Things like monitoring, remediation from issues, expanding capacity for backend storage, or etc are great examples of these types of "day 2" activities that operators can automate.

I have previous experience in the Mesos and DC/OS space with Mesos Frameworks which are functionally equivalent to Kubernetes Operators. I had previously written a Framework to deploy and manage a storage platform for Mesos.

[jpkrohling]

Thanks for your insights, @dvonthenen! Do we have a volunteer to work on this? If not, I can get an initial version running "soon" (couple of weeks, as my queue isn't exactly empty).

Edit: @pavelnikolov, are you on board with this? Do you agree we should use the Operator for the major tasks, having a Helm chart mainly as a "wrapper" around this?

[dvonthenen]

@jpkrohling I can definitely help out and contribute some time to work on this, but I currently have other assignments in my queue that have some urgency behind it.

I would also agree that having the chart deploy the operator would be a great strategy going forward.

[jpkrohling]

I started working on this and the code is located here: https://github.com/jaegertracing/jaeger-operator

It currently supports only the all-in-one, but I'd appreciate any feedback on the main design.

[jpkrohling]

We are very close to an initial release of the operator. I'd love feedback from you, @pavelnikolov and @dvonthenen. Of course, I'd also welcome feedback from anyone else interested in trying it out :)

[pavelnikolov]

I’ll review it tomorrow.