Pulsar: Large message size handling in Pulsar: Chunking for shared subscription
We have requirement of sending large message size over 250 MB for the shared subscription.
As the chunking feature is not enable for shared subscription while we are trying to send the message above 200MB we are getting below error.
2020-07-23 03:53:03,864 -0700 [bookie-io-1-10] ERROR org.apache.bookkeeper.proto.BookieRequestHandler - Unhandled exception occurred in I/O thread or handler
io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 406847488 byte(s) of direct memory (used: 1744830464, max: 2147483648)
amitkaushal16
>All comments
@amitkaushal16 yes, right now chunking feature is only available for non-shared subscription. It's little tricky for shared subscription as it requires sequential chunked-messages delivery to one consumer and also need to take other scenarios such as acking, redelivery, keeping multiple chunks in buffers, etc. We will look into this feature soon.
rdhabalia
on 23 Jul 2020
Related issues
rfuerst87
路
3Comments
frankjkelly
路
3Comments
qmnonic
路
3Comments
simoneau
路
3Comments
Tielem
路
3Comments
Most helpful comment
@amitkaushal16 yes, right now chunking feature is only available for non-shared subscription. It's little tricky for shared subscription as it requires sequential chunked-messages delivery to one consumer and also need to take other scenarios such as acking, redelivery, keeping multiple chunks in buffers, etc. We will look into this feature soon.