Rabbitmq: [RFC] use augeas to write RabbitMQ config file instead of hardcoded bash

[jippi]

Augeas is pretty amazing, it allows for easy modification of the configuration file using "sniper" accuracy

example:

augtool --autosave set /files/etc/rabbitmq/rabbitmq.config/rabbit/vm_memory_high_watermark "0.4"

This will inject just that one change into the file, leaving everything as it initially was.

I'm currently hitting a problem where I want to modify the configuration file in my derivative Dockerfile, but all those changes are overwritten by the current docker endpoint

Usage:

• apt-get install augeas-tools augeas-lenses
• wget https://raw.githubusercontent.com/hercules-team/augeas/master/lenses/erlang.aug -O /usr/share/augeas/lenses/dist/erlang.aug
• wget https://raw.githubusercontent.com/hercules-team/augeas/master/lenses/rabbitmq.aug -O /usr/share/augeas/lenses/dist/rabbitmq.aug
• augtool --autosave set /files/etc/rabbitmq/rabbitmq.config/rabbit/vm_memory_high_watermark "0.4"

done! :)

we could even have the docker entrypoint run all augeas files in /etc/rabbitmq/conf.d/*.aug to allow easier extension of this amazing Dockerfile :)

My current case is that I want to modify the vm_memory_high_watermark value, but can't find a way to do it without modifying the image

[yosifkit]

That sounds pretty amazing and would make our entrypoint and possibly #53 much simpler, but I am curious what users think about it and also if it is recommended by upstream. @michaelklishin or @dumbbell, do you have any opinions about adding and using Augeas to generate/modify the config file in the image?

[dumbbell]

Hi!

I have too little experience with Docker and none with Augeas to have an educated opinion. I guess whatever makes your life and the end users' life easier is perfect :-)

[tianon]

This looks like a much cleaner way to munge the configuration indeed, but the size hit for installing it appears to be ~40.06MB (which is ~22% of our current image size of ~178.7MB) :cold_sweat:

[jippi]

do the image size truly matter in those small increments compared to the flexibility and features it would provide ? :) for me personally, it would not be a deal breaker at all. :)

[michaelklishin]

If Augeas supports Erlang term files, it sounds like a good idea. Note that starting with 3.7.0 RabbitMQ will use an easier to parse config format by default but even then using Augeas may be worth it.

The 40 MB of added image size aspect doesn't concern me personally, bandwidth is getting cheaper predictably year after year and developer/operator time does not.

[jippi]

@michaelklishin they have http://augeas.net/docs/references/lenses/files/erlang-aug.html and a specific rabbitmq one http://augeas.net/docs/references/lenses/files/rabbitmq-aug.html :)

[tianon]

It still concerns me a bit that Augeas parses Erlang expressions with regular expressions in order to accomplish the task of modification. I seem to recall that @yosifkit found that we'd also need to install a newer RabbitMQ lense than is available in the Debian packages in order to parse the (arguably very simple) default configuration file we include.

IMO this newer INI-style format is going to be a simpler solution that should clean up our script pretty dramatically (especially and including being able to handle user-supplied files _way_ easier). :sweat_smile:

[jippi]

Augeas can also handle ini files :)

[Thubo]

I took the liberty and started working on the feature: https://github.com/Thubo/rabbitmq/tree/FEATURE-%23120

The augeas lenses are based on the work of Raphael Pinson, but extended them to fit (nearly) all options I could find in the official docs.

There are some open TODOs:

• Update the entrypoint.sh script
• Modify /etc/rabbitmq/enabled_plugins
• Some cleanup and more test

For now I've added the augeas lenses to this repo since it's not yet complete. Sooner or later these changes can be merged back to the hercules-team.

Please note, this is work in progress. Nevertheless I'm happy to hear suggestions how to improve.

[jippi]

with the new config file setup this isn't relevant any more