Docker-jitsi-meet: [Helm] New chart

[taktakpeops]

Hi,

I had been working lately on building a Helm chart for running Jitsi.

So far, it looks like working. I am able to have the web component deployed on its own, multiple JVB instances per Prosody/Jicofo couples and OAuth using oauth2_proxy.

My sources are available here: https://github.com/taktakpeops/jitsi-meet-helm

I am running my example in AWS EKS using a HAProxy ingress controller with URI load-balancing and all looks good so far.

I was wondering if the Jitsi team was already planning on pushing the Helm chart to the official Helm repo. If not, I would like to contribute on that. I saw that there are at least 2 PRs open for adding the chart here and one branch (pretty much inactive by now) for Helm in this repo.

What would be the best way for contributing to the project regarding Helm?

[sapkra]

If I understand @saghul correctly he only want to officially support docker-compose, all the other implementations shall be managed by the community. But I'm also for more official support - I think many jitsi users would donate for a full time employee to manage such stuff for the community. But there is currently no possibility to donate.

@saghul Please correct me if I'm wrong.

[taktakpeops]

My idea is not to add the chart to this repository but to push it here: https://github.com/helm/charts

However, I don’t know if someone within the community of Jitsi is already working on it.

The final goal being to distribute the Helm chart through the official channels.

[ChrisTomAlx]

@taktakpeops Is there a way to add HA jibri as well?

[taktakpeops]

I am planning on adding the extra components gradually. Jibri is on my list and it's being integrated currently.

[ChrisTomAlx]

@taktakpeops Awesome.. Will test the current HA chart on GKE in the mean time and report back.
Let me know if you need any help with jibri.. I could also take a look at certain things if required.. For jibri my understanding is we require node level changes. and every jibri container supports only one recording or live streaming at a time

[taktakpeops]

My current approach works with multiple pods running Jicofo + prosody + Jibri (when enabled). The service in front of the pods do a load balancing with sticky session using session affinity. For JVB, each pod running Prosody + Jicofo + Jibri gets its own JVB deployment with 2 to 3 JVB instances. Sync between JVB pods and XMPP server is made using MUC.

So in this case, I would be able to have as many Jibri as I have prosody servers.

Hope it answers your question. However, all of that is yet to be tested.

[ChrisTomAlx]

Scaling jibri with prosody does not seem to be cost efficient since jibri takes a lot of resources and concurrent recordings will need to have that many jibri pods.

So say you have 3 conferences running and all 3 start recording. These 3 would run fine with single instances of prosody jicofo jvb and web but it would require 3 instances of jibri. And if all 3 conferences also start live streaming that would be 6 instances of jibri..

[taktakpeops]

Because GCP + Jibri aren't specific to this issue, let's move the discussion here: https://github.com/taktakpeops/jitsi-meet-helm/issues/1

[saghul]

If I understand @saghul correctly he only want to officially support docker-compose, all the other implementations shall be managed by the community. But I'm also for more official support - I think many jitsi users would donate for a full time employee to manage such stuff for the community. But there is currently no possibility to donate.

@saghul Please correct me if I'm wrong.

Mostly correct :-) I'm ok with putting things in examples/ but as the readme states, they are self-serving / community maintained.

[taktakpeops]

Ok understood. I was planning on keeping the chart isolated in its own repository.

Now, as I mentioned @saghul, my idea is to make the Chart available from here: https://github.com/helm/charts

The goal being that anyone playing with helm can use a chart available in their official artifactory.

If it's up to the community to decide, what would be the best place for triggering the discussion?

I wonder if someone has already started to work on that as there are already 2 or 3 other Helm Charts that I know of.

[taktakpeops]

@ChrisTomAlx - regarding the architecture I am following, it's described here: https://github.com/taktakpeops/jitsi-meet-helm/issues/1#issuecomment-627257639

Let's move the discussion over there regarding Jibri and other components

[ChrisTomAlx]

@ChrisTomAlx - regarding the architecture I am following, it's described here: taktakpeops/jitsi-meet-helm#1 (comment)

Sure @taktakpeops . Let's continue there.

---

Now, as I mentioned @saghul, my idea is to make the Chart available from here: https://github.com/helm/charts

If Jitsi wishes to officially support kubernetes at one point then I think the helm charts can be placed here. Elastic search did something similar where they started releasing the official helm charts and the charts at helm/stable became obsolete. Thats just my two cents.

@saghul Is there any plans of officially supporting kubernetes / helm eventually?

[saghul]

I don't know if we have plans for that, sorry.

[sapkra]

@ChrisTomAlx IMO it would make more sense to build a kubernetes operator for jitsi which will also handle the scaling of jvb instances and updates.

@ktenzer from RedHat already started building an operator as part of his podium project (jitsi is part of it). https://github.com/sa-mw-dach/podium/tree/master/podium-operator

[ChrisTomAlx]

@sapkra I have very little experience with kubernetes operators. Let me look at it. Thanks for sharing RedHat's podium operator, could be a good starting point to understand this concept properly.

@saghul no worries. If we get the Kubernetes HA installation to a decent level, maybe jitsi can pick it up later on.

[taktakpeops]

@sapkra : indeed, we could make an operator. The chart as it is currently can be transformed.

As suggested by @ChrisTomAlx, please find here a diagram describing how my chart works:

I was thinking on running the pod containing Jicofo + Prosody with an optional container running Jibri. What do you think?

In this case, I would annotate the Jicofo + Prosody + Jibri pods for running on their own nodes customised for the needs of the 3 containers (so a lot of CPU and Memory).

Regarding load balancing, everything uses sticky session. The Haproxy for the web component receive the traffic from a NLB. The Haproxy ingress controller uses the URI load balance algorythm to ensure that the traffic for one room goes always to the same replicas. Between the web pods and the Prosody instances, I have a service applying session affinity based on the client IP (web pod in this case). Each Prosody pod gets its own JVB deployment with 2 to 3 replicas. Each deployment set has a service of type Node Port with its own port open for receiving UDP traffic. Each service uses also session affinity based on the client IP.

Let me know if you need more info.

[krakazyabra]

Hi @taktakpeops
Any news? Do you need some help?

[taktakpeops]

Hello,

I haven't worked much on that lately. But ideally, building a K8S operator instead of the Helm chart seems to be what's suggested by the maintainers.

Feel free to start with that, I can help if needed.

[krakazyabra]

Hi! I deployed one more chart, but it is scalable without operator (hpa, or manually by setting number of shard). Look, may be you'll find something interesting
https://github.com/krakazyabra/jitsi-helm

[MatthiasHertel]

@krakazyabra
realy nice chart .. thank u!
i have the following problem , i use traefik as a my default ingress ... how can i use this chart with traefik in front of that ?
it is possible in general?
or do I have to replace ha proxy with traefik ?