Mosquitto: Broker stops retransmitting at QOS1 when client does not send PUBACK (max_inflight_messages=1)

[keytee]

We have an IoT system which has MQTT clients with limited resources. We are using QOS1 and the MQTT client can only handle one message at a time (no memory for queue). The concept is that if someone send 10 messages to that client (each client has its own topic) the client gets the message, process the message (send it over serial line) and if its OK send a PUBACK back. If something happens on the serial line and the client cannot forward the messages to is serial bus, then the client drops the MQTT message (not sending PUBACK back) and waits for the broker to retransmit the given message.

Because of the client can handle only one message at once, we are using max_inflight_messages = 1, which is working as expected (the broker not sending more than 1 message at a time) but after a client does not send PUBACK back, the broker not trying to resend the messages as expected (in every retry_interval time). The PINGREQ/PINGRESP works, and the client can send publish messages over the broker, the only problem is that client wont get any messages from the broker to the given topic until it reconnects.

Are we doing something wrong or are we misunderstanding max_inflight_messages functionality?

Tested with 1.4.15 and 1.5 on linux (Debian 9.5)

Thanks

[toast-uz]

Because I've not considered this situation, I checked the standard and found:

4.4 Message delivery retry
When a Client reconnects with CleanSession set to 0, both the Client and Server MUST
re-send any unacknowledged PUBLISH Packets (where QoS > 0) and PUBREL Packets
using their original Packet Identifiers [MQTT-4.4.0-1]. This is the only circumstance
where a Client or Server is REQUIRED to redeliver messages.

Therefore, the behavior of mosquitto broker might be normal. Your client can disconnect and re-connect with CleanSession set to 0, in order to force retransmission. Could you try?

[keytee]

MQTT reconnection after every serial error (when not sending PUBACK) is not an option, I think.
Anyway, we are using MQTT over TLS so an every time reconnection causing unnecessary network load.

From the mosquitto.conf man page:

retry_interval seconds
The integer number of seconds after a QoS=1 or QoS=2 message has been sent that mosquitto will wait before retrying when no response is received. If unset, defaults to 20 seconds. 
Reloaded on reload signal.

So, when client not sending PUBACK is the definition of no response is received by server. So after retry_interval seconds, de broker must resend the message.

[toast-uz]

The retry became strongly prohibited on MQTTv5. On MQTTv5 standard, "Clients and Servers MUST NOT resend messages at any other time" was added at the same place.

On the standard, you MUST reply PUBACK when you receive PUBLISH with qos 1. In the situation, I guess you should recover by yourself rather than rely on the broker.

Regarding retry_interval option seems no longer available now. By reading the source files, if you set the option, you can see the message "Warning: The retry_interval option is no longer available." Maybe there is a mistake on man page.

[karlp]

that retry interval is being dropped by the way, it's demonstrated to be a terribly poor idea in practice for..... everyone but you who wants to rely on it :)

For everyone else, a client that wasn't processing messages was ~never improving the situation by sending it more copies. (it's almost always overworked/swamped and the retransmissions don't help)

[toast-uz]

Thanks @karlp .
In my opinion, mqtt qos can operate under an ideal environment,, but can "almost all" under real. So, we cannot rely on qos if we need strict durability. In fact, mqttv5 will adjust with real by several new features, such as messages TTL and request-reply information.

On @keytee situation, I recommend to use qos0 and application level acknowledgement on another topic.

[keytee]

Thanks guys, if this is the correct way of working, we can live with that.
Please correct the man page to avoid the further misunderstandings about retransmission.

[toast-uz]

Note ./man/mosquitto.conf.xml has been already correct removing the description of retry_interval, but the web page was not revised.