Masstransit: Generic Message Receiver

[phatboyg]

A simple way to use MassTransit to dispatch _messages_, without having to build a transport.

Similar to what was done to support Azure Functions, but not specific to any transport.

The generic receiver would be able to accept a real MassTransit message, or plain old JSON, or any type that has already been deserialized (think, <T>), and dispatch it through the MassTransit consumer pipeline.

Some basic extensibility points should be available, including a _Publish_ and _Send_ abstraction, that could be connected to anything (or nothing).

• [ ] Multiple overloads for message delivery
  
  
  
  • [ ] JSON message envelope (MassTransit)
  
  
  • [ ] JSON, raw, with message type overload
  
  
  • [x] <T>, an already deserialized message type
  
  
• [x] Exceptions should be propagated all the way up to the receiver's caller, so that they can be handled by the calling function.
• [ ] Publish abstraction, callback style for published messages
• [ ] Send abstraction, callback style for sent messages (how do we get an address?)

[floyd-may]

Watching this issue. Would like to see if I could make an attempt to wire up MT to Google Cloud Pub/Sub, this seems like a good start.

[phatboyg]

I'm considering native support for Google Pub/Sub, I just haven't dug into the API specification yet. Also, need to add support for GPB (and ultimately Avro). But that's a v6.1 thing.

[floyd-may]

Do you already have an issue in GitHub somewhere I could subscribe to for updates on GCP Pub/Sub support in MT?

[phatboyg]

Well, after looking at the API, I'm not exactly sure how I would _send_ - it seems there are only topic subscriptions. Which is half of the puzzle, and their API is pretty rough, sadly.

[floyd-may]

Ooh, yeah, I think I see what you mean. If I'm understanding correctly, GCP Pub/Sub subscriptions can only have a single topic, so a MT receive endpoint would need to listen to multiple Pub/Sub subscriptions; i.e. there isn't a (straightforward) way to have a single queue represent heterogeneous messages from multiple topics; it seems like their model is queue-per-topic-consumer rather than queue-per-consumer/endpoint. On the send side, it would seem that you'd have to possibly send multiple copies of the same message based on the exposed interfaces of the message? 🤢

At any rate, I'm still interested in following this topic as it progresses.

[phatboyg]

There is actually a way, I think ActiveMQ does the same thing by registering multiple consumers. So that would be a good start.

[phatboyg]

This is live, in v6.2, and I wrote about it: https://masstransit-project.com/2020/02/23/mediator/

Feedback is welcome.

[alexeyzimarev]

Looks great, we were bringing GreenPipes to some of our solutions, like EventHub consumer, but it was never very straightforward. Will be a lot simpler now. And if other tasks will eventually be done, making a fully functional event-oriented transport will be very easy to do.

[alexeyzimarev]

Send abstraction, callback style for sent messages (how do we get an address?)

I guess, as now. ISendEndpoint.Send with the address or using conventions. The address should be a simple string. Then, it could be, basically, anything - topic name, EventHub name, whatever.

[phatboyg]

Right. And with the new short addresses, like queue:name and topic:name it's easier as well.

[mastertheef]

So is there any way to send and consume messages to GCP PubSub with via mediator?

[phatboyg]

So is there any way to send and consume messages to GCP PubSub with via mediator?

You could consume by sending messages to mediator, but there is no facility to produce messages back to GCP.