Aws-sdk-js: SQS send large messages

[sebastianvera]

Hello! Any plan on implementing what amazon-sqs-java-extended-client-lib does to allow sending messages larger than 256KB?

Thank you very much!

[n3xu5]

I am facing the same need, my initial idea was to create a wrapper for the SDK implementing similar behavior of the Java SDK.
Do you have something similar in mind with your suggestion?

[sebastianvera]

I am facing the same need, my initial idea was to create a wrapper for the SDK implementing similar behavior of the Java SDK.
Do you have something similar in mind with your suggestion?

Yes @n3xu5 , we ended up doing something very similar, just uploading large messages to a Storage Service and then downloading them 🤷‍♂️

[n3xu5]

Well, I will search one more time if nothing satisfactory comes up I will write an open source wrapper for the SDK.
I don't know if it's allowed to paste the link here, but I guess more people might find this useful.

[rustyforeal]

any updates on this?

[cjuega]

Hi @n3xu5,

I just finished an initial version of such library for nodejs. It just overwrites a few methods, however other methods should be overwritten to fully accomplish the mission.

So far:

• sendMessage uploads large messages to S3 whenever an S3 client is given.
• receiveMessage downloads large messages from S3 when required and it replaces messages' body.
• deleteMessage deletes S3 objects whenever the SQS message has a reference to S3.
• deleteMessageBatch (same as deleteMessage)

I built it as a mixin, so it doesn't depend on any specific version of aws-sdk, you just simply inject the aws-sdk you are using.

Right now it's a private repository, but if anybody is still interested on this, I will request to open source it.

[victor-xplore]

@cjuega an open source workaround would be nice

[jackvhall]

I just finished an initial version of such library for nodejs. It just overwrites a few methods, however other methods should be overwritten to fully accomplish the mission.

Hi @cjuega ,

My organization has a need for something like this. Please share? :)

[cjuega]

Hi @victor-xplore and @jackvhall,

Sorry for the delay. I finally convinced my company to open source the project. It will be available this week probably.

I'll post it here once it gets published.

[cjuega]

Hi again @victor-xplore and @jackvhall,

sorry for the late post. We finally managed to publish the library to npm.

If anybody needs SQS messages larger than 256KB, (s)he can use the following package:

https://www.npmjs.com/package/aws-sqs-xl-messages

[ajredniwja]

Since there is a library now, would like to close this issue, great work @cjuega