Janus-gateway: Random Service Restarts

[DaGLiMiOuX]

Hi there.

I'm reporting this to github issues to let @lminiero find if there is any bug, as stated in this discussion in google groups: https://groups.google.com/forum/#!topic/meetecho-janus/ozYlTa9NSTA

I've posted all pastes from the group discussion:

This paste is from v0.4.1 - https://pastebin.com/E7P5aZTJ

This paste is from v0.4.1 - https://pastebin.com/HCG730hL

This paste is from v0.4.2 - https://pastebin.com/FBnyvDiR

I still have random restarts on the janus service when I call for a videoroom destroy. @mirkobrankovic stated in the google group post post that he actually do not destroy videorooms, just detach users from it when creating dynamic rooms. I destroy them because I do not know if it has any performance issues to keep videorooms open or security issues. By the way, the service should not restart "randomly". Most of the times when the videoroom is closed.

My service is configured as stated here: https://janus.conf.meetecho.com/docs/service.html#systemd
I've also included the -e option for event handlers.

I rolledback to v0.2.6, which worked perfectly before updating to v0.3.x and v0.4.x to see if it throws any errors and/or check if the service is restarted.

I believe I have installed all the required libraries and I have all of them up to date.

D.

EDIT: After a while (2 hours or so), the service did crashed when closed the videoroom, even with v0.2.6 which was working perfectly. I'm starting to think that is something related with library dependency.

EDIT2: Still not seeing any problems in my libraries. Any advice?

EDIT3: New different crash when a participant tried to join the videoroom: https://pastebin.com/aJ8HN4Zd

[lminiero]

The traces you're providing are incomplete: please do a bt on gdb for a complete one. Besides, try building with libasan support as well, as it may provide more info.

[lminiero]

That said, a couple of your dumps referred to line 4282:

janus_videoroom_publisher *owner = g_hash_table_lookup(publisher->room->private_ids, GUINT_TO_POINTER(subscriber->pvt_id));

and I guess the problem here is publisher->room that may be NULL. I added a check for that in the code, but we'll have to check if that can cause issues: in fact, this means that if publisher->room, we're never removing the subscriber pointer from a list that is used for the kick request. I guess it won't be a problem, as you're destroying the room now so there's nothing to kick anymore, but still, something I'll have to remember.

[DaGLiMiOuX]

This gives me much more information. Could it be crashing because clients are trying to call hangup function when the videoroom is already destroyed?

They call the hangup and when janus try to kick that publisher from the room, the app is crashing because no room was found in the publisher as the publisher should be NULL. If the publisher is NULL, then you cannot access to the room property. Am I right?

EDIT: It also happens when the room is closed from within the app and no users are in it. I mean, my server app does a request to janus to close the videoroom.

EDIT2: I recently moved all my app and services (including janus) into a new CentOS 7 installation to be sure that is not a bug in any of my systems. Still crashing: https://pastebin.com/YUH4nQFE
I forgot to use bt command when getting the older logs, sorry.

[lminiero]

Calling hangup should be absolutely ok when destroying a room (if you don't, the core does it for you when you destroy the handle). Please let me now if the fix I made fixes it, as the pastebin you provided in the response clearly shows it's not on master (the line I changed is still the same).

[DaGLiMiOuX]

I mean that a call on hangup function was placed when listening the videoroom destroyed event in the client (JS) and I think that was causing the issue. Btw, while I do some tests with the new fix, I got another crash in a different line when a user joins the videoroom.

Here is the paste: https://pastebin.com/TqHiSbi5

I will test that too with the new master version.

[lminiero]

I made a commit that should help with that: https://github.com/meetecho/janus-gateway/commit/2a647644c53a0366f4dd477920c92d1673f99a33
But it sounds like something weird in your environment: the same property (subscriber->room) is successfully used a few lines above, which makes me think you're having people join when destroying a room.

Please only provide new logs for master, and not for versions before the fixes, or you'll make my job harder :wink:

[DaGLiMiOuX]

Sorry, I updated my janus instance before that commit XD I will keep you updated with new master version.

[DaGLiMiOuX]

After a day and a half doing setting up videorooms and destroying them, the service still not crashing.

[lminiero]

Good to know! Feel free to close the issue when you feel confident this has been fixed.