Masstransit: Add Amazon SNS/SQS Support

[bsommardahl]

Is it possible to extend MassTransit to use Amazon SQS instead of just RabbitMQ or MSMQ? What would be necessary?

[TravisTheTechie]

Transports are developed and delivered as external assemblies. As long as SQS has routing mechanisms, it should be possible to write a transport for it.

[bsommardahl]

How many days do you think that would take the average developer? What technologies would be needed to dev that?

[TravisTheTechie]

You can see the existing transports: https://github.com/MassTransit/MassTransit/tree/master/src/Transports

So I'd guess you'd need somewhere around 3-5k total line of code total create a robust transport. It's just written in .NET, need some async knowledge, and knowledge use of a client library for Amazon SQS. If there's no client library, you might need some more code to support the protocol at a lower level.

If I were to build a transport, knowing something about MassTransit, I'd look for a couple weeks to crank one out and have it tested, ready to roll out to a proving ground somewhere. If you are unfamiliar with MassTransit, I think you'd want more time for sure.

I think the question to drive "is that time worth it to invest" is around do you want the other stuff you can do on top of MT. If you just need a way to connect to the queue and serialize some data, MT is overkill. If you want to build complex, type-based routing or use state machines then it's all together different answer.

[bsommardahl]

My team is working on a rather large financial system and need some of the features that make the queue more robust. We would have to build them or "buy them", so it might be worth contributing. Know of any developers who have experience with this who might be able to contract to help us make this contribution?

[TravisTheTechie]

@haf did some transport work before, I don't know his situation around consulting. I doubt @phatboyg or @drusellers are interested in doing some consulting, but I could be wrong. That covers the major contributors.

[phatboyg]

If you have experience with SQS, and are familiar with the client library, and decide to take on the effort, I'd consider using MT3 as the basis for the work. It's a better model, IMHO, than the transport implementation that is in MT2. The code in MT2 works fine, but it makes certain assumptions around MSMQ and was adapted to work with RabbitMQ.

[bsommardahl]

Thanks for the advice Chris.

[phatboyg]

Did you end up making any progress on SQS? I'm thinking of adding it, but I need to understand what routing capabilities are available first.

Seems that SNS gives you topics, and SQS gives you queues, the combination of which could be a useful setup. Particularly if publish leveraged SNS, with delivery to SQS for consumers. Just a random thought.

[gertjvr]

Wondering if anybody has started with this issue, if not looking on starting with it in the near future :smile:

[bsommardahl]

Go for it.
On Wed, Feb 10, 2016 at 11:37 PM Gert Jansen van Rensburg <
[email protected]> wrote:

Wondering if anybody has started with this issue, if not looking on
starting with it in the near future [image: :smile:]

—
Reply to this email directly or view it on GitHub
https://github.com/MassTransit/MassTransit/issues/293#issuecomment-182720406
.

_Byron Sommardahl_
CTO, Co-Owner
Acklen Avenue | www.acklenavenue.com | 615-669-8239
632 Fogg Street Suite 8, Nashville, TN 37212

Acklen Avenue is a custom software development company committed
to creating quality software that people love to use.

[gtourkas]

@gertjvr @bsommardahl Has any of you picked this up or should i give it a try ?

[phatboyg]

If you'd like, I don't think anyone else has started on it yet.

[gertjvr]

@gtourkas I have had a few attempts at doing this, but get lost in the configuration part of the transport.

[charlesconklin]

@gertjvr Are you still working on this? I'm curious because my team wants to move to AWS in the near future and they don't want to use RabbitMQ.

[rschiefer]

+1 for adding SQS support to MassTransit

[alexgurman]

Same issue, got MT to work with RabbitMQ, but EA group wants us to use SQS.

[adolya]

Hi, we have also a requirement for this.
Page https://github.com/MassTransit/MassTransit/tree/master/src/Transports
doesn't exists (

[phatboyg]

Likely, not, Amazon MQ is possible though (it's just ActiveMQ)

[gertjvr]

Started working on AmazonSQS / SNS transport https://github.com/MassTransit/MassTransit/pull/1166