Jitsi-meet: Error message after 30 minutes

[DanielBatteryStapler]

Anywhere from an hour to 30 minutes jitsi meet will stop working and will give the message "Jitsi-videobridge is currently unavailable, please try again later". Audio and video will not work after this.

This can only be fixed by restarting jicofo, not jitsi-videobridge. Restarting jitsi-videobridge does not fix the problem.

This makes jitsi-meet completely unusable for me because I have to keep restarting it just to use it. Even if nobody is using it on my website, it will still give this message to anyone that joins after an hour of it starting.

[becoadmin]

I have the same issue but its not consistent on when it does it. It could be a few days or a week before I get the "Jitsi-videobridge is currently unavailable, please try again later" message.

Brian

[anatomism]

I'll second this. Still happening even with the current (4th July) nightly build. Didn't used to happen with build from about 2 months ago, seems to be related to Jicofo

[anatomism]

Still an issue that's not gained any traction by the looks. Restarting Jicofo fixes it, but its unpredictable as to when it errors

[paweldomas]

Can you show us Jicofo logs from the time when it happens ?

[anatomism]

See link for Jicofo Log of a time when I accessed Jitsi Meet and it errored

http://pastebin.com/6piLNqW2

[paweldomas]

Can you include some of the logs before that happens ? Maybe even up to few minutes. The important thing is what happens with the JVB state and why Jicofo thinks there is no bridge available.

[anatomism]

This was from the restart of Jicofo. I then made a conference with myself (two tabs same browser) for a few seconds to veryify it worked. Then returned at 16:41 to try again and discovered the error

http://pastebin.com/mQH7R3Vj

[paweldomas]

Thanks ! So this is a problem with the JVB. It has failed the health check with an error: "Too many open files". What version do you use there ?

[anatomism]

I use the latest version from the Debian Unstable Repo (751-1?) [deb http://download.jitsi.org/deb unstable/] - the only thing upgraded from this is Nginx to mainline and Prosody to 0.9.10

I have however used the nightly a few weeks ago and that was the same problem (I did full purge before installing the non-nightly version)

[anatomism]

@paweldomas Do you have any suggestion as what I could be looking for in my config in attept to fix please, or do you think its a bug in the code somewhere?

[bgrozev]

@anatomism this doesn't look right. Is it reproducible? Do you have any conferences during these 30 minutes? Can you please attach the full logs from the bridge?

[conorsch]

Shot in the dark, but "Too many open files" suggests this may be a problem with max open files. Use ulimit -n on your server to check what that's set to. A rather conservative value of 1024 serves me well for small meetings (2-6 participants).

[paweldomas]

@conorsch correct, but it may also mean that jvb is not releasing resources after health checks which has to be investigated and fixed eventually

[anatomism]

Thanks @conorsch but I'm at 65536 currently for all process users, so I doubt that raising it will be sufficient

[anatomism]

@bgrozev. It's very reproducible, it's every time I restart the services have a conference and then come back for another conference a day or so later. Logs showing its actually about 3-5 mins after the first conference is closed that the Health Check fails. I'm just collecting log files at the moment

[bgrozev]

After you hit the error, can you leave the bridge running and attach the output of lsof -p <PID>, so we can see what kind of fds are being used?

[anatomism]

Here is the jicofo log that's been left running when it errored: http://pastebin.com/z5Bav1H8

Here is the output of "lsof -p 22352" http://pastebin.com/JaeQJZij

Thanks

[bgrozev]

This looks exactly like jitsi/jitsi-videobridge#245. We still don't understand why it happens, or even under what circumstances (we tried to repro multiple times without success).

Closing this one, since it is not a problem in jitsi-meet itself.

[anatomism]

@DanielBatteryStapler @becoadmin

The guys were correct, it's not a jitsi-meet error. I suspect it's a Java mis configuration. I completely wiped my vitual machine and created a new Linux machine, installed as per quick install instructions and it's been working fine for 3 days now. Before it wouldn't last an hour

[anatomism]

Initially nothing I could do would work including new install. Its been running for over a week once i did the following:

You can disable health checks from jicofo by adding this to /etc/jitsi/jicofo/sip-communicator.properties
org.jitsi.jicofo.HEALTH_CHECK_INTERVAL=-1