Docker-jitsi-meet: Container restarting permanently if SELinux is set to enforcing - current codebase

[martinbehrens]

I've cleaned all images and containers from previous jitsi meet docker installs and freshly cloned the project (commit d44230e2b121e72542a60b6dfbce9b07e84ed706) again. After performing respective steps as outlined https://github.com/jitsi/docker-jitsi-meet#quick-start followed by a docker-compose -f docker-compose.yml -f jibri.yml up -d containers keep on restarting:

[jitsi@meet docker-jitsi-meet]$ docker-compose ps
          Name              Command     State                      Ports
-----------------------------------------------------------------------------------------
dockerjitsimeet_jicofo_1    /init     Restarting
dockerjitsimeet_jvb_1       /init     Restarting
dockerjitsimeet_prosody_1   /init     Up           5222/tcp, 5269/tcp, 5280/tcp, 5347/tcp
dockerjitsimeet_web_1       /init     Restarting

Therefore the only thing I could try to fix was the following I found in /var/log/messages:

Apr 17 16:34:27 meet journal: **************************
Apr 17 16:34:27 meet journal: Prosody was unable to find the configuration file.
Apr 17 16:34:27 meet journal: We looked for: /etc/prosody//config/prosody.cfg.lua
Apr 17 16:34:27 meet journal: A sample config file is included in the Prosody download called prosody.cfg.lua.dist
Apr 17 16:34:27 meet journal: Copy or rename it to prosody.cfg.lua and edit as necessary.
Apr 17 16:34:27 meet journal: More help on configuring Prosody can be found at https://prosody.im/doc/configure
Apr 17 16:34:27 meet journal: Good luck!
Apr 17 16:34:27 meet journal: **************************

But, there was no /etc/prosody directory and /config/ is not accessible:

root@36123f894017:/# ls -al /config/
ls: cannot open directory '/config/': Permission denied
root@36123f894017:/# chown root /config/
chown: changing ownership of '/config/': Permission denied

Please try to replicate and fix. If there is any advice, please share. Thanks!

[martinbehrens]

Those were my changes to the .env file, nothing fancy (and yes, the sound module is present):

[...]
# System time zone
TZ=Etc/Utc

# Public URL for the web service
PUBLIC_URL=https://FQDN
[...]
# Enable authentication
ENABLE_AUTH=0

# Enable guest access
ENABLE_GUESTS=1

# Select authentication type: internal, jwt or ldap
AUTH_TYPE=internal
[...]
# Enable recording
ENABLE_RECORDING=1
[...]

[Franky1]

Are you sure that your config paths are right?
This double-slash // in this line looks suspicious:
We looked for: /etc/prosody//config/prosody.cfg.lua

[martinbehrens]

Are you sure that your config paths are right?
This double-slash // in this line looks suspicious:
We looked for: /etc/prosody//config/prosody.cfg.lua

Agreed but that I did not touch and as mentioned: there isn't even a /etc/prosody directory in the prosody container.

[Franky1]

I can only guess...

• i still think your config path is weird
• where is your .env file on the host?
• did you also delete the config directories?
• did you change the user between some of the steps?
• What is your line beginning with CONFIG= in .env ?
• Dangling docker volumes? Check with docker volume ls

[Franky1]

But, there was no /etc/prosody directory and /config/ is not accessible:

Where did you run the command ls -al /config/ ? On the host or inside the container?
I assume, you're looking in the wrong place.
Or you placed the config folders in the wrong path or as another user.

[martinbehrens]

I can only guess...

• i still think your config path is weird

It certainly is.

• where is your .env file on the host?

/home/jitsi/docker-jitsi-meet/.env

• did you also delete the config directories?

Yes, I removed /home/jitsi/.jitsi-meet-cfg/ completely prior deployment.

• did you change the user between some of the steps?

Nop, it was jitsi all the way.

• What is your line beginning with CONFIG= in .env ?

CONFIG=~/.jitsi-meet-cfg

• Dangling docker volumes? Check with docker volume ls

I removed all volumes and containers and everything, so currently the output is:

[jitsi@meet docker-jitsi-meet]$ docker volume ls
DRIVER              VOLUME NAME
local               0062be9030cede6c90bc677db4a1916fd66766a238729f7cb26833edd333617e
local               40990675a81ee4561480574682152609f6538d51550caf137d51592a55d88110
local               97f4677415f71004cd79255595c5f43640cf274d3d6a8d586a26a68779056e57

But, there was no /etc/prosody directory and /config/ is not accessible:

Where did you run the command ls -al /config/ ? On the host or inside the container?

Yes, inside the container.

I assume, you're looking in the wrong place.
Or you placed the config folders in the wrong path or as another user.

I didn't place/move anything manually, all I did was as described above; executing the commands as mentioned in https://github.com/jitsi/docker-jitsi-meet#quick-start

Thus, steps to replicate are simple. Check out the project, follow https://github.com/jitsi/docker-jitsi-meet#quick-start and try to start with docker-compose -f docker-compose.yml -f jibri.yml up -d since I wanted to bring up jibri as well.

[Franky1]

I am running out of ideas.. and i am not a linux/docker expert...

• does the user jitsi have the proper (group) rights?
• did you use sudo with one of your commands?

Maybe open a thread in the community forum, i assume, you will get more attention and help there?

[martinbehrens]

I am running out of ideas.. and i am not a linux/docker expert...

• does the user jitsi have the proper (group) rights?

It's member of jitsi and dockerroot, the latter is required for docker operations to be permitted in the first place.

• did you use sudo with one of your commands?

No, there was no need.

Maybe open a thread in the community forum, i assume, you will get more attention and help there?

Maybe. Maybe not breaking things on update intentionally - https://github.com/jitsi/docker-jitsi-meet/issues/404#issuecomment-613312100 - would help as well, that's why I had to start over from scratch.

[saghul]

@martinbehrens Maybe you can suggest a better way to do things, instead of throwing out sndie remaks next time.

Make sure you are running the latest image versions, try this: docker-compose pull also if a container is misbehaving, it's best to check the logs with docker-compoe logs

[martinbehrens]

@martinbehrens Maybe you can suggest a better way to do things, instead of throwing out sndie remaks next time.

The suggestion would be pretty simple with proper QA incorporated in a CI/CD pipeline which includes protecting the master branch, working with feature branches, a CI/CD with dev/staging/prod tiers and respective jenkins jobs that actually test the stuff before it gets merged into master and all that. Of course all that requires some effort and, surely, funding.

Make sure you are running the latest image versions, try this: docker-compose pull also if a container is misbehaving, it's best to check the logs with docker-compoe logs

Done that, but all docker-compose logs spits out is what was also being written into syslog ...

prosody_1  | **************************
prosody_1  | Prosody was unable to find the configuration file.
prosody_1  | We looked for: /etc/prosody//config/prosody.cfg.lua
prosody_1  | A sample config file is included in the Prosody download called prosody.cfg.lua.dist
prosody_1  | Copy or rename it to prosody.cfg.lua and edit as necessary.
prosody_1  | More help on configuring Prosody can be found at https://prosody.im/doc/configure
prosody_1  | Good luck!
prosody_1  | **************************

However, as last resort I tried to set SELinux to permissive and it worked. So apparently some SELinux restrictions prevent those containers from successfully starting up in a Centos 7.7.1908 env, not sure what exactly though. Technically this issue is not resolved yet.

[lab-at-nohl]

However, as last resort I tried to set SELinux to permissive and it worked. So apparently some SELinux restrictions prevent those containers from successfully starting up in a Centos 7.7.1908 env, not sure what exactly though. Technically this issue is not resolved yet.

Thank you so much, I just wanted to open an issue for that. Any idea how to fix this?

[saghul]

The suggestion would be pretty simple with proper QA incorporated in a CI/CD pipeline which includes protecting the master branch, working with feature branches, a CI/CD with dev/staging/prod tiers and respective jenkins jobs that actually test the stuff before it gets merged into master and all that. Of course all that requires some effort and, surely, funding.

Yeah, everybody wants to save the world, but nobody wants to do the dishes.

So apparently some SELinux restrictions prevent those containers from successfully starting up in a Centos 7.7.1908 env, not sure what exactly though. Technically this issue is not resolved yet.

To me that sounds more like a generic Docker problem than specific to this setup.

[Franky1]

Maybe this gives some hints, but i have no clue, if this applies to the problem:

https://docs.docker.com/storage/bind-mounts/#configure-the-selinux-label
https://jaosorior.dev/2018/selinux-and-docker-notes/
https://prefetch.net/blog/2017/09/30/using-docker-volumes-on-selinux-enabled-servers/

[lab-at-nohl]

Yes, SELINUX is the culprit here. You have to tell docker that the config directories must be accessible if selinux is enforcing.

This is done by adding :Z to the VOLUME path (config is made accessible inside the containers via bind mount). You can do so by adding it to each line in the volumes: section in docker-compose.yml.