Mqtt.js: odd behavior when subscribing to topics with wildcards

[rjbordon]

Hi there,

I've having some issues when subscribing to topics with wildcards. If I subscribe my client to "devices/+/activation" a message published to "devices/qwerty/metadata" if being processed by my client, which is in fact an error.

I'm using mosquitto as MQTT broker. This implementation supports wildcards for topics, and when I try this functionality with the mosquitto built-in client it works as expected. (just in case: http://mosquitto.org/man/mosquitto-8.html)

Does MQTT.js support this functionality? or I missing something else?

Thx in advance.

[mcollina]

MQTT.js has no special logic for subscribes and wildcards. Something weird
is going on :/.

[mcollina]

Can you post an example that replicate the issue?

[rjbordon]

Hi @mcollina,

Sure, below a sample you can use to replicate the issue:

var mqtt = require('mqtt'),
    host = '127.0.0.1',
    port = 1883,
    client = mqtt.createClient(port, host),
    pubArgs = {
        qos: 1,
        retain: false
    };

console.log('subscribing to devices/+/activation');
client.subscribe('devices/+/activation')
    .on('message', function(topic, message){
        console.log('message got to DEVICES ACTIVATION with topic: %s, message: %s', topic, message);
    });

console.log('subscribing to devices/+/metadata');
client.subscribe('devices/+/metadata')
    .on('message', function(topic, message){
        console.log('message got to DEVICES METADATA with topic: %s, message: %s', topic, message);
    });

var id = Math.floor(Math.random()*10000);
var topic = 'devices/' + id + '/activation';
console.log('publishing to %s', topic);
client.publish(topic, 'I\'m message for activation!!!', pubArgs);

What I'm seeing in my console is:

subscribing to devices/+/activation
subscribing to devices/+/metadata
publishing to devices/9956/activation
message got to DEVICES ACTIVATION with topic: devices/9956/activation, message: I'm message for activation!!!
message got to DEVICES METADATA with topic: devices/9956/activation, message: I'm message for activation!!!

Thx a lot for your help.

[mcollina]

The message event is emitted for every message received by the connection.
You should do it only once and then disambiguate.

Check out http://npm.im/ascoltatori because it implements the in-app
message disambiguation for Redis, MongoDB and also MQTT.

[rjbordon]

oh, got it, on method is attached to message event which is fired when any topic is fired against an instance of client.

thanks a lot for your help. very appreciated.

[VikramTiwari]

@mcollina Not exactly the same behaviour, but unable to publish/subscribe on the wildcard topics.

listening to mqtt sub -t 'hello/vikram/hi' -h 'test.mosquitto.org' -v
publishing to mqtt pub -t 'hello/+/hi' -h 'test.mosquitto.org' -m 'from MQTT.js'

getting no data on the subscriber side.
PS: Using node v4.1.0

[mcollina]

You need to:

mqtt sub -t 'hello/+/hi' -h 'test.mosquitto.org' -v
mqtt pub -t 'hello/vikram/hi' -h 'test.mosquitto.org' -m 'from MQTT.js'

[VikramTiwari]

Does this means I can subscribe on wildcard topics but can not publish to wildcard topics?

Edit:
Yes, can't publish on wildcard topics. Ref: https://developer.ibm.com/answers/questions/29899/publishing-topics-with-wildcards-not-working.html

[RangerMauve]

@VikramTiwari Yes. MQTT doesn't allow you to publish to topics with wildcards in them AFAIK.

You must publish to topics that match the wildcards.

[anhldbk]

@mcollina This issue still persists. Here is the demo code:

// test.js

var mqtt    = require('mqtt');

var client  = mqtt.connect('mqtt://test.mosquitto.org');
client.on('connect', function () {
  console.log('Connected')
  client.subscribe('#/:event/status') // <- using # will cause the problem below
});

client.on('offline', function () {
  console.log('Offline')
});

The output will be:

$ node test.js
Connected
Offline
Connected
Offline
Connected
Offline
Connected
Offline
Connected
Offline
Connected
Offline
Connected
Offline
Connected
Offline
Connected
...

[RangerMauve]

@anhldbk What makes you say that this is related to the wildcards? Is there an error you're seeing that would suggest that? Because it just looks like your socket is disconnecting and reconnecting for no particular reason.

[anhldbk]

@RangerMauve There are many reasons:

1. If I subscribe to another mailboxes like +/:event/status, data..., it works well.
2. My network connection is stable. Actually, I tried to run the program above with a local MQTT broker before using test.mosquitto.org

[RangerMauve]

Are you logging any errors on the client side or on the broker itself?

[anhldbk]

@RangerMauve There's no error logged on the client side.

client.on('error', function (error) {
  console.log('Error: ' + error) // <-- Nothing is printed out
});

[sandro-k]

@anhldbk the # has to be at the end like foo/bar/#. If you want to subscribe to everything just use #

#/:event/status === # because everything that is after the # is irrelevant. See http://www.hivemq.com/blog/mqtt-essentials-part-5-mqtt-topics-best-practices

[anhldbk]

@sandro-k Thank you for your explanation!

@mcollina Is it a good idea for mqtt.js to validate topics before sending the subscriptions to brokers?

[mcollina]

I think it might be a good idea! PR?

[anhldbk]

@mcollina I'm working on it.

[NancyZY]

@mcollina, @sandro-k , why I cannot accpt the message ?
sub topic : 'floor/+'
pub topic : 'floor/1'

[sandro-k]

@NancyZY sub to floor/+ and pub to floor/1 is valid. As already mentioned the MQTT-Spec is saying what is possible and what not. A good explanation can be found at http://www.hivemq.com/blog/mqtt-essentials-part-5-mqtt-topics-best-practices

[NancyZY]

@sandro-k ,yeah, I got it. I want to ask for another aspect: What is the limit of subscribe ? In other word, How many topics sub-client can accept at most? and How to solve it ? Does it have a method to test the limit?

[sandro-k]

42 - no just joking. As many as your System can handle CPU/RAM wise.

[NancyZY]

Ooh, I knew it , thank you! It is a few number. How can I deal with the situation? I want to deal with the cloud client which can sync status, then notify the mobile phone, it subscribe the topic like "floor/+" , but the number of publish topic is thousands. What can I do to run normally？

[sandro-k]

It is a scalability issue you are facing. Split up hierarchy to support multiple processes and threads so that each process is doing a portion a the topics.

[NancyZY]

Hi, the whole process that subscribing and publishing topic is right, BUT, sometimes it happened wrong , the console hints [Errno 54] Connection reset by peer, what is it? How can I solve it