Libzmq: ZMQ:4.2.0: Filters are dropped when number of filters is high(>1000) and PUB process is restarted

[victorserbu2709]

Scenario:

Consumer:
1 SUB socket which connects to a remote PUB socket.
Filters have this pattern: sprintf(filter, "%010d.u", i);

subscribe to 501 keys: 0000000000.u to 0000000500.u
receive 10 messages
subscribe to 501 keys: 0000000500.u to 0000001000.u

Producer:
1 PUB socket which binds
Each second it sends 2 messages using key 0000000000.u and 0000001000.u

Consumer receives 2 messages each seconds, but if producer is stopped and then started again, consumer will receive only 1 messages each second (sent to 0000000000.u) but it doesn't receive message sent to 0000001000.u)

Tested with zmq 4.2.0
Consumer: https://gist.github.com/victorserbu2709/aec0b6f677623baee63617c99b4f68ee
Producer: https://gist.github.com/victorserbu2709/b0cf0723e41bf21606251ecd22a0d772

[sublee]

I think it is related to a crash issue what I reported at #2252.

[sublee]

I guess 1000 the magic number is SNDHWM of a SUB socket. A SUB socket seems to drop SUBSCRIBE or UNSUBSCRIBE messages for PUB sockets when it reaches over SNDHWM.

[bluca]

Yes you are hitting the HWM when the socket reconnects. As the documentation says, the default is 1000. So when the sub reconnects it tries to send all subscribes at once and ends up dropping some.

Remember that a subscribe message is just a message, so it's affected by the HWM in the pipes like any other message.

Simply set the SNDHWM in your sub socket to be higher than the number of subscriptions you are planning to do. Setting the option in your test program is enough to make the problem disappear.

[bluca]

See the ZMTP specs for more details about the PUB-SUB protocol:

https://rfc.zeromq.org/spec:23/ZMTP/
https://rfc.zeromq.org/spec:29/PUBSUB/

[sublee]

So we should set SNDHWM of SUB sockets to be the potential maximum number of the all subscriptions from each projects. I can't guess the number for my project. So I simply set it as 0 which means "unlimited".

[bluca]

Also comment inline in xsub.cpp:

    //  Send it to the pipe.
    bool sent = pipe->write (&msg);
    //  If we reached the SNDHWM, and thus cannot send the subscription, drop
    //  the subscription message instead. This matches the behaviour of
    //  zmq_setsockopt(ZMQ_SUBSCRIBE, ...), which also drops subscriptions
    //  when the SNDHWM is reached.
    if (!sent)
        msg.close ();

[bluca]

Yes setting to unlimited will be fine for your case. Given the only thing a SUB socket sends is the subscription messages (and initial handshake of course) it's very unlikely that it could cause OOM issues.

[sublee]

I think this issue can be closed.