Rabbitmq-server: 3rd party plugin development with the monorepo

[lukebakken]

To reproduce:

git clone https://github.com/rabbitmq/rabbitmq-metronome
cd rabbitmq-metronome
make dist

Output in my env:

 DEPEND rabbitmq_federation.d
 ERLC   Elixir.RabbitMQ.CLI.Ctl.Commands.FederationStatusCommand.erl Elixir.RabbitMQ.CLI.Ctl.Commands.RestartFederationLinkCommand.erl rabbit_federation_app.erl rabbit_federation_db.erl rabbit_federation_event.erl rabbit_federation_exchange.erl rabbit_federation_exchange_link.erl rabbit_federation_exchange_link_sup_sup.erl rabbit_federation_link_sup.erl rabbit_federation_link_util.erl rabbit_federation_parameters.erl rabbit_federation_queue.erl rabbit_federation_queue_link.erl rabbit_federation_queue_link_sup_sup.erl rabbit_federation_status.erl rabbit_federation_sup.erl rabbit_federation_upstream.erl rabbit_federation_upstream_exchange.erl rabbit_federation_util.erl
src/rabbit_federation_util.erl:10: can't find include lib "rabbit/include/amqqueue.hrl"
src/rabbit_federation_util.erl:56: undefined macro 'is_amqqueue/1'
src/rabbit_federation_util.erl:60: undefined macro 'is_amqqueue/1'
src/rabbit_federation_util.erl:65: undefined macro 'is_amqqueue/1'
src/rabbit_federation_util.erl:15: function name/1 undefined
src/rabbit_federation_util.erl:15: function r/1 undefined
src/rabbit_federation_util.erl:15: function vhost/1 undefined
make[3]: *** [erlang.mk:5442: ebin/rabbitmq_federation.app] Error 1
make[2]: *** [erlang.mk:5184: app] Error 2
make[2]: Leaving directory '/home/lbakken/development/rabbitmq/rabbitmq-metronome/deps/rabbitmq_federation'
make[1]: *** [erlang.mk:4512: deps] Error 2
make[1]: Leaving directory '/home/lbakken/development/rabbitmq/rabbitmq-metronome/deps/rabbit'
make: *** [erlang.mk:4512: deps] Error 2

[michaelklishin]

Yes, we need to find another convention for plugins to integrate into the umbrella (monorepo). If you clone it under deps/ it should work in at least some cases.

[lukebakken]

Unfortunately in the above example rabbitmq/rabbitmq-server is cloned under deps/

[celvin]

any news on this?

[lhoguin]

There's a few options to solve this:

• Require development to be done inside the monorepo directly
• Use git-dir to fetch the dependencies (so rabbit will sit under deps/rabbit and not under deps/rabbit/deps/rabbit)
• Fetch the monorepo specially under monorepo, build once, and automatically add the paths to ERL_LIBS

[celvin]

@lhoguin which one would you recommend for a newbie and how would you execute it?

[lhoguin]

@celvin The above is about solving the issue permanently.

For a quick workaround, something like this seems to work fine:

git clone https://github.com/rabbitmq/rabbitmq-server
cd rabbitmq-server
make
cd ..
git clone https://github.com/rabbitmq/rabbitmq-metronome
cd rabbitmq-metronome
ln -s ../rabbitmq-server/deps
make dist

Then proceed as normal. You may also want a specific version/branch of rabbitmq-server, which should be switched to before running make.

[whitfin]

Do we have any idea when we'll pick a solution to this? And will the plugin development guides be updated to cover it?

[michaelklishin]

There is a solution, which is to clone your plugins under deps in the monorepo and make its Makefile include rabbitmq-components.mk and friends the way tier 1 plugins do.

The website is open source, so you are welcome to update the guide. This flow is not great but it is functional. Our team is very busy with other priorities so I doubt we will get to work on a 3rd party plugin flow any time soon.

[pjk25]

There have been some updates to rabbitmq-components.mk, found at the root of the monorepo, that should make this pretty straightforward now. rabbitmq-metronome has been updated to demonstrate with https://github.com/rabbitmq/rabbitmq-metronome/pull/14 and I believe the existing guide is applicable.

Additionally, for the adventurous, rabbitmq-metronome can also be tested via bazel since https://github.com/rabbitmq/rabbitmq-metronome/commit/0482d6a83b02dc2282c93db2323e25ee4a3c6edc