rabbitmq-server won't be able to read a RABBITMQ_ENABLED_PLUGINS_FILE it created if umask is strict

[selivan]

Some environments require strict umask, for example PCI DSS complaint hosts. Or just if somebody is concerned with security and uses strict defaults.

Command rabbitmq-plugins should be run from root. If root umask=0027, it creates file /etc/rabbitmq/enabled_plugins with permissions root:root 0640. rabbitmq-server running under rabbitmq user can not read it.

It should either always use pre-defined umask=0022 or make rabbitmq user owner or group owner for /etc/rabbitmq/enabled_plugins, so it can read it.

Example of error:

root@work:~# umask
0027
root@work:~# rabbitmq-plugins enable rabbitmq_management 
The following plugins have been enabled:
  mochiweb
  webmachine
  rabbitmq_web_dispatch
  amqp_client
  rabbitmq_management_agent
  rabbitmq_management

Applying plugin configuration to rabbit@work... failed.
Error: {cannot_read_enabled_plugins_file,"/etc/rabbitmq/enabled_plugins",
           eacces}
root@work:~# ls -l /etc/rabbitmq/enabled_plugins
-rw-r----- 1 root root 23 Feb 27 18:12 /etc/rabbitmq/enabled_plugins

[michaelklishin]

Thanks for the detailed report. I'm not sure what user should be the owner of said file because rabbitmq-plugins actually is NOT required to be executed by root and besides package-based installation methods there is never a user named rabbitmq on the host.

So it's a good idea but not at all clear what user should be the owner. Perhaps this should only be addressed in package scripts (which could pre-create this file and others and set up the permissions).

[michaelklishin]

Consider the following example: every RabbitMQ team member or contributor runs RabbitMQ nodes primarily as a non-privileged user, often from source and in foreground, every single day. Every Homebrew and standalone MacOS build user runs RabbitMQ as a non-privileged user only.

So how would whatever piece of code has to create enabled_plugins know what user should be the owner? It's known in package post-install scripts only.

[selivan]

@michaelklishin

pre-create this file and others and set up the permissions

Would not work, because file is not overwritten, but deleted and created again. I can see it because permissions and inode number are changed each time I run rabbitmq-plugins.

So how would whatever piece of code has to create enabled_plugins know what user should be the owner?

Ideal solution is to overwrite enabled_plugins file instead of delete and create. Sufficient workaround is to set umask 0022 in rabbitmq-plugins shell script.

[dumbbell]

Instead of trying to determine the user:group who runs RabbitMQ, I believe it would be simpler and easier to reason about if rabbitmq-plugins(1) made sure the file is readable by everyone, or at least readable by the group. In the latter case, the admin would be responsible to make sure the RabbitMQ user belongs to that group.

[michaelklishin]

enabled_plugins does not contain anything sensitive (just a list of enabled plugins), so maybe making it world-readable wouldn't be too bad. It will probably cause some security scanners and operators to scream "bloody murder!" for no reason, though.

@dumbbell readable by what group, rabbitmq or similar? that is, a well-known group as opposed to something dynamically computed?

[gerhard]

If this environment is expected to be secure, I'm pretty sure running commands as root go against this directive. Furthermore, enabling RabbitMQ plugins from the command-line mutate the deployment, which also doesn't sound like a good idea in a security strict environment.

Would it be acceptable to enable plugins via configuration rather than the command line? If so, the infrastructure automation tool of your choice could simply create RABBITMQ_ENABLED_PLUGINS_FILE specific to your platform with the correct permissions which would contain e.g. [rabbitmq_management,rabbitmq_top].

[michaelklishin]

Unfortunately RabbitMQ Puppet module and such use rabbitmq-plugins instead of pre-creating a file.

[dumbbell]

@michaelklishin: I would not try to change the group from rabbitmq-plugins(1) either, just the permissions. Ownership could/should be handled outside of the script. That said, the fact the file is deleted/created instead of updated makes changing ownership difficult.

[michaelklishin]

So doing a couple of things:

• Switching to rewriting the file instead of creating and deleting it
• Pre-creating a file in package post-installation scripts

would yield some improvement.

[selivan]

@gerhard

We just follow the admin guide. Commands are run as root only when we deploy new server.

Several workarounds can be used to fix this: setting umask before running rabbitmq-plugins, creating RABBITMQ_ENABLED_PLUGINS_FILE by other tool, etc. But I think having it working out-of-the-box is better option :)

So I continue to insist that using umask 0022 in rabbitmq-plugins is one suitable solution, and making it overwrite file in place instead of delete/create is even better.

[selivan]

@michaelklishin

Switching to rewriting the file instead of creating and deleting it
Pre-creating a file in package post-installation scripts

This will solve the issue completely.

[michaelklishin]

Unfortunately "out of the box" here means a really large number (and growing) of possible deployment scenarios. This is only solvable as long as we known what user or group RabbitMQ's effective process uses.

[michaelklishin]

There's little appetite for messing with umask in our scripts. As explained above, we won't even try to solve this for the general case, only packages.

[selivan]

@michaelklishin Overwriting files instead of delete and create will solve the problem for some of the users, me included. It's easy to create files with proper permissions in initial deployment process, both for pre-built distributive packages and automation tools. And it does not break anything. So I think it's a good thing to implement it.

[hairyhum]

rabbit_file:write_file creates a "backup" file and renames it. https://github.com/rabbitmq/rabbitmq-server/blob/stable/src/rabbit_file.erl#L169

This was introduced to avoid an issue when erlang process (presumably) dies after file truncation but before write. bugzilla#25617

[selivan]

@hairyhum Then it would help if backup file owner and permissions were copied from original, not created from scratch

[roeintense]

I have to agree. This is causing issues on my rabbitmq_management enable as well.

rabbitmq-plugins enable rabbitmq_management
Plugin configuration unchanged.

Applying plugin configuration to rabbit@devbox... failed.
Error: {cannot_read_enabled_plugins_file,"/etc/rabbitmq/enabled_plugins",
           eacces}

[michaelklishin]

This is mostly for @Gsantomaggio :)

Currently there are two options that seem to have support on our team:

• Making the enabled_plugins file world-readable in package post-install scripts
• Avoiding re-creating the file (this can end up being much more involved than it sounds)

[Gsantomaggio]

(After internal conversation )

Making the enabled_plugins file world-readable is the easier ( and safest) way.

[michaelklishin]

@Gsantomaggio group readable may be sufficient. @selivan's idea in https://github.com/rabbitmq/rabbitmq-server/issues/1129#issuecomment-292957764 sounds like a safe and straightforward option as well, so we should investigate it.

[Gsantomaggio]

Let's recap the problem:

The file enabled_plugins is created by rabbitmq-plugins inside the dir /etc/rabbitmq

Now the directory has the following permissions:

root@jessie:/home/vagrant# ls -la /etc/rabbitmq/
drwxr-xr-x  2 root root 4096 May 25 10:44 .
drwxr-xr-x 91 root root 4096 Jun 20 17:23 ..

So the rabbitmq group can't access on this file:

root@jessie:/home/vagrant# rabbitmq-plugins enable rabbitmq_management
...
Applying plugin configuration to rabbit@jessie... failed.
Error: {cannot_read_enabled_plugins_file,"/etc/rabbitmq/enabled_plugins",
           eacces}

(Note: the error we see here is from RabbitMQ server and not from rabbitmq-plugins )

We have two solutions:

1 - Modify rabbitmq-plugins adding umask 0022 as suggested here.

2 - Modify the setup and give the access to /etc/rabbitmq also to the rabbitmq group:

chgrp rabbitmq /etc/rabbitmq ; chmod g+s /etc/rabbitmq

In this way the RabbitMQ service can access without problems.

We ( Me and @dumbbell) prefer the second solution, we'd like to avoid to modify the umask inside rabbitmq-plugins

We exclude this solution, because the the owner of enabled_plugin file is root ( in this case ) so copy the rights won't solve the problem.

Important note:

inside /etc/rabbitmq there is also the rabbitmq.config file, it can be created manually or by tools, so the problem is not only rabbitmq-plugins.

The problem can be solved using the solution 2:

root@jessie:/# ls -la /etc/rabbitmq/
drwxr-sr-x  2 root rabbitmq  4096 Jun 20 19:48 .
drwxr-xr-x 91 root root      4096 Jun 20 19:36 ..
-rw-r-----  1 root rabbitmq    23 Jun 20 19:48 enabled_plugins
-rw-r-----  1 root rabbitmq 22321 Jun 20 19:47 rabbitmq.config

[michaelklishin]

@Gsantomaggio I like the 2nd option better as well. Covering packages (Debian, RPM) and eventually Chef cookbook, Puppet module, etc. should be good enough as we don't have control over how generic UNIX binary builds are provisioned.

[michaelklishin]

@selivan would you be interested in testing a one-off build of https://github.com/rabbitmq/rabbitmq-server-release/pull/30? If so, would you like a Debian or RPM package?

[Gsantomaggio]

@selivan @michaelklishin :
Packages uploaded here: https://github.com/rabbitmq/rabbitmq-server-release/pull/30#issuecomment-310658554

[selivan]

@michaelklishin @Gsantomaggio Installed deb package from https://github.com/rabbitmq/rabbitmq-server-release/pull/30#issuecomment-310658554.

SGID bit on /etc/rabbitmq works, rabbitmq-plugins now works fine:

# umask
0027
# rabbitmq-plugins disable rabbitmq_top
The following plugins have been disabled:
  rabbitmq_top

Applying plugin configuration to rabbit@work... stopped 1 plugin.

So this fixes my use case. But systems with stricter umask like 0077 will still have problems. Complete solution will be to preserve file permissions either by explicit saving them or by rewriting old file. If that is not possible, this one IMHO is acceptable.

[Gsantomaggio]

@selivan Thank you for your test, and glad to hear that it fixes your case.

At this point I'd merge the PR https://github.com/rabbitmq/rabbitmq-server-release/pull/30, then we can investigate how to handle stricter umask like 0077.

[dumbbell]

Hi!

I merged @Gsantomaggio's pull request as it improves the situation. I would like to keep this issue open so we remember to completely fix it.

[Gsantomaggio]

Hi,
I tried different solutions as

• set the permissions on the rabbitmq-plugins code.
  Change/play with permissions inside the code can be dangerous, we have to considerer other platforms and file-system types.

• avoid to recreate the file
  Even if this could be a solution, it requires a manual configuration to set the right permissions to the file, so I'd exclude it.

In my option working with umask as 077 can be solved in different ways, for example:

• run the command with another umask

#umask
0077
#  (umask 007; rabbitmq-plugins enable rabbitmq_federation)
The following plugins have been enabled:
  amqp_client
  rabbitmq_federation

Applying plugin configuration to rabbit@jessie... started 2 plugins.

• using the ACL:setfacl -d -m group:rabbitmq:r /etc/rabbitmq/

• and other permissions configurations

In conclusion, I'd avoid to change the code/setup for this specific situation, we can document it by showing the problem and the possible solutions

[michaelklishin]

I tend to agree with @Gsantomaggio. The solution we have in place is safe, easy to reason about and covers a lot of cases. The last few % can be solved in a way appropriate for a given system using deployment tools such as Chef, Puppet, BOSH and so on.

@dumbbell WDYT?

[dumbbell]

Yes, I think it's fine as well.

[michaelklishin]

See the PR above. Some are not happy about the changes that help this use case.

I strongly recommend people who want to run RabbitMQ in a particularly strict environment to use their deployment tools to set the permissions and umask to their needs instead of expecting our packages to cater to every possible case: sometimes their needs are on the opposite ends of the usability/security spectrum.

[Gsantomaggio]

I agree with it

[piyushjalan02]

Switch to rabbitmq user and then enable plugin:

$su - rabbitmq
$rabbitmq-plugins enable rabbitmq_management

[Muhammedshafeerali]

If this environment is expected to be secure, I'm pretty sure running commands as root go against this directive. Furthermore, enabling RabbitMQ plugins from the command-line mutate the deployment, which also doesn't sound like a good idea in a security strict environment.

Would it be acceptable to enable plugins via configuration rather than the command line? If so, the infrastructure automation tool of your choice could simply create RABBITMQ_ENABLED_PLUGINS_FILE specific to your platform with the correct permissions which would contain e.g. [rabbitmq_management,rabbitmq_top].

Thank you its worked for me