Rabbitmq-server: Retained messages not sent to wildcard subscribers

[csyllac]

Using the RabbitMQ Alpine docker image, which contains:
RabbitMQ 3.6.14 on Erlang 19.3

Custom Docker image created to add the:
RUN rabbitmq-plugins enable --offline rabbitmq_mqtt
EXPOSE 1883
Commands to enable MQTT.

Docker started with:
docker run -d --hostname rabbit --name rabbit -p 8080:15672 -e RABBITMQ_SERVER_ADDITIONAL_ERL_ARGS="-rabbitmq_mqtt.retained_message_store [mqtt.retained_message_store [rabbit_mqtt_retained_msg_store_dets] mqtt.retained_message_store_dets_sync_interval [2000]]" rabbit-mqtt
to enable message retention to disk.

There is what appears to be a bug in that retained messages are not sent to a receiver which starts after the retained message is set and which uses a wildcard to subscribe.

Simple send code:

import paho.mqtt.client as mqtt
import time

broker_address="172.17.0.2" 
client = mqtt.Client("sender", False, mqtt.MQTTv311)
client.connect(broker_address)

client.publish("/examples/test1", "test 1 Retained", retain=True)
client.loop()

Simple receive code:

import paho.mqtt.client as mqtt
import time

def on_message(client, userdata, message):
    print("received topic %s QoS %d RF %d received message %s" % (message.topic, message.qos, message.retain, str(message.payload.decode("utf-8"))))

broker_address="172.17.0.2" 
client = mqtt.Client("listener", False)
client.on_message=on_message
client.connect(broker_address)
client.subscribe("/examples/test1", 1)  <-- This works
client.subscribe("/examples/#", 1)      <-- This does not
client.loop_forever()

To reproduce the issue:
Comment out one of the two subscribes.
Start receiver python receive.py
Run sender python send.py

Messages will be sent and received as expected.
Exit receiver and restart it.

If the subscription is to the specific topic, the retained message is received.
If the subscription is to the wildcard topic, the retained message is not received.

Using VerneMQ, the retained message is always received using either subscription form.

[michaelklishin]

Retained messages have at least one known limitation: the store is node-local (that is documented). I can't think of a reason for wildcards to not have an effect with just one node, thank you for providing steps to reproduce.

[teemupiiroinenwirepas]

Do you have any estimation when this could be fixed?

[michaelklishin]

We do not make ETA promises.

[mdford]

Is this going to be fixed? I am having the same issue and just discovered this thread.

[michaelklishin]

@mdford it's not a priority but this is open source software, so those who need this badly are free to look into supporting wildcards (it's not entirely trivial, at least with the current plugin architecture).

[hardmax]

Hi,
Somebody know how solve this....I migrate all my plattform from mosquitto to Rabbitmq, but I dont know that Rabbit has this limitation..

For now I has to suscribe to each topic when I need recibe retained message, but maybe there are something to configure for this works... any idea?

regards

[fungiboletus]

Hi,

I'm not an Erlang developer, I couldn't implement a hello world, but from what I understand the handle_call for fetch should return a list of replies instead of a single reply and instead of ets:lookup, ets:match (or similar) should be called with a search query generated from the topic. However in case of many retained messages, the performance impact would be huge so I guess it should be a third retained message store. Perhaps some fancy indexing could be implemented.

https://github.com/rabbitmq/rabbitmq-mqtt/blob/b19eeb4edf1003ca2fb39884319a4755cda88a09/src/rabbit_mqtt_retainer.erl#L85-L91

https://github.com/rabbitmq/rabbitmq-mqtt/blob/d367b61e4c83121a41b8ec3f5131488c3de6bded/src/rabbit_mqtt_retained_msg_store_ets.erl#L52-L54

Learning Erlang is on my TODO list but I'm afraid I will not be able to send a pull-request anytime soon.

[michaelklishin]

A sequential scan over all retained messages would not be acceptable. RabbitMQ already has a trie-based topic routing implementation (one, two). It has to be adapted.

On top of this, retained message store really should be distributed, which is somewhat similar to what we did in rabbitmq/rabbitmq-mqtt#91 for duplicate client ID tracking. It is unlikely that we will delay 3.8 because of this issue.

[stfnbstl]

Is this already fixed or will it be fixed at all in the near future?

[michaelklishin]

It has not been. Unless someone would be interested in contributing (this is a fairly non-trivial task, although much smaller if limited to an in-memory implementation), this is unlikely to be addressed soon because there are much higher priority items for the core team, even in this plugin alone.

[travisghansen]

This seemingly non-compliant behavior is quite troubling as an end-user :( especially since it’s known and been floating around for 2.5 years..

[sobelman]

Such a common use case.... can't figure how its not supported yet.

[lukebakken]

can't figure how its not supported yet

Comments like the above do not help the discussion. This is the reason why this feature is not yet supported:

there are much higher priority items for the core team, even in this plugin alone

@sobelman and everyone else who has commented here, please take note of the "Sponsors" section of this feature added to RabbitMQ:

https://github.com/rabbitmq/rabbitmq-server/pull/2654

[Webbeh]

Hi everyone.

We recently switched to RabbitMQ from MQTT for professional reasons, and were disappointed to see this very troubling issue. We use MQTT for our legacy applications and most of them use wildcard-based subscriptions for various reasons, and it is not really possible for us to do the switch to fully qualified topic names in most of those applications.

Returning to Mosquitto for MQTT could be an option if Mosquitto didn't fail us due to the high volume of messages.

We would like to know what it would take for this issue to be resolved as quickly as possible without disrupting your schedule.

Thank you very much!

[lukebakken]

We would like to know what it would take for this issue to be resolved as quickly as possible without disrupting your schedule.

Pay VMware or another vendor to implement the feature. Erlang Solutions has provided excellent work for paying customers.

https://www.rabbitmq.com/#support

[Webbeh]

Thank you for your answer. We will contact them today.