Eshoponcontainers: [Question] New to API Gateway and BFF Patterns

[stumpykilo]

How and where do the controller methods like UpdateAllBasket in the Web.Shopping.HttpAggregator project get called? Do any of the services communicate with each other internally without the event bus?

[CESARDELATORRE]

The Http communication is as follows:
Client app --> API Gateway --> (Optional) HttpAggregator --> Internal Microservices
The HttpAggregator services are meant to be used only for certain Http calls that are more efficient if you aggregate them in a single call. Especially for mobile and SPA apps, it is more efficient instead of a chatty communication from remote apps.

The Event Bus is based on async communication based on RabbitMQ or Azure Service Bus and it is used for integration events derived from transactions that happened in any of the microservices and when other microservices need to be aware of those events, so they are subscribed to those events. But the Event Bus it just for events, not for queries, like Http calls.

Therefore, the HttpAggregator services are not using the event bus.

[stumpykilo]

Thanks @CESARDELATORRE for your response, that is really helpful. I'm also trying to understand how the eShopContainer services communicate between themselves (internally - querying), do they just use an HttClient call? Can you point me to an example of this in the source code?

Could you also point me to an example of where a mobile or SPA app is calling or consuming an HttpAggregator service?

[CESARDELATORRE]

@stumpykilo - This section from the book might be interesting for you:
https://docs.microsoft.com/en-us/dotnet/standard/microservices-architecture/architect-microservice-container-applications/communication-in-microservice-architecture

The internal microservices should have minimum or no Http communication at all between them, if possible. If you create long Http requests with many microservices within a single original Http request, your microservices won't be as autonomous as they could be and things can get pretty bad when you have any issue in one of them.
That is also a reason why you also might need async comm with the Event Bus, to have eventual consistency between the microservices thanks to the integration events.
Hope it helps. 👍

[stumpykilo]

Thanks for the link @CESARDELATORRE. What would justify the need to put Http communication between two internal services (say an Inventory service needs to do a simple lookup from an Item service)? Would you use the event bus to do that type of query? Would you or could you use an API Gateway type service to do that type of query?

I'm still not seeing how and where the HttpAggregator service is used or called in the source control. Could you point me to where that is?

Thanks again @CESARDELATORRE.

[kolluri-rk]

@CESARDELATORRE - after reading ocelot documentation and your blog post https://blogs.msdn.microsoft.com/cesardelatorre/2018/05/15/designing-and-implementing-api-gateways-with-ocelot-in-a-microservices-and-container-based-architecture/, I got very clear understanding on how it works. Thanks a lot for that detailed write up.

I have an architectural decision to make. Please let me know your comments after reading my problem.
Note: it may be bit lengthy, please bear with me.

We are designing a system composed of 6 microservices. 2 of them are Metering and Billing. Meter data will be created and maintained inside Metering service. Billing service needs very few attributes (5 attributes to be very specific) of Meter. I planned to duplicate that information using Integration events. Like publishing meter created event, meter changed event etc. to eventbus and subscribing for those events in billing service. So billing service is self sufficient to do its job.

One of my teammate has also gone through eShopOnContainers API gateway architecture and suggesting that to build an aggregate apigw, to pass those meter attributes along with billing command data to billing services. He thinks that for some reason if the updated meter data did not get to billing service database, billing may use wrong data.

I understand that default ocelot aggregate configuration/custom aggregates are designed to reduce chattiness between client and microservices. And I have one good use case as well. one of the page block needs data from 3 microservices. So i can certainly use this aggregate pattern. But I'm not sure if its a good approach to design apigw to pass additional information to the command issued from client to backend. With that the control flow would be, angular app issues create bill command --> received by apigw --> pull meter data from meter service --> construct actual create bill command required by billing service --> pass newly constructed command to billing service --> send response back to angular app.

Though this eliminates the need of duplicating meter data in billing service, I'm not sure if I'm violating any design principles or guidelines. I could vision that it may not be very scalable as we are hitting meter service every time while creating bill and also we can't post bill just in case if the meter service is down. Since here in our shop they are choosing consistency over scalability, what do you suggest on my teammate's proposal.

Thanks a lot for your patience for reading all the way through here :) Please let me know your comments when you get a chance.

[CESARDELATORRE]

@stumpykilo - HttpAggregator services are not used from C# code. They are called only from the Ocelot API Gateways and originally from the client apps. Take a look to the configuration.json like here:
https://github.com/dotnet-architecture/eShopOnContainers/blob/dev/src/ApiGateways/Web.Bff.Shopping/apigw/configuration.jsonIn there you can see there are routes for the "webshoppingagg" service/container.

About this question:
What would justify the need to put Http communication between two internal services (say an Inventory service needs to do a simple lookup from an Item service)? Would you use the event bus to do that type of query? Would you or could you use an API Gateway type service to do that type of query?

Usually, don't do that. Each microservice has to have all the data it needs for its processes. If you don't have the data you need, two things might happen:

1. That microservice's boundary might be wrong and it should be part of a larger microservice.
2. Duplicate that data (just for queries) in your microservice with eventual consistency based on async communication like when using the integration events with the Event Bus. Example is the price in the Catalog and the price in the Basket that is updated with eventual consistency with integration events.

The Event Bus is used only to propagate data through integration events, things that happened in the past in other microservices. So it is basically about updates. You don't use the Event Bus for queries.

[CESARDELATORRE]

@kolluri-rk - The HttpAggregation services should be used only in exceptional cases when it is clear that you need to improve the chattiness between client apps and the microservices.
The HttpAggregation services should not be used as an "business rules orchestrator" because in that moment you are tightly coupling the microservices with that front-end service and eventually, the HttpAggregator service will become a monolithic... breaking the tenets for microservices.

The approach is:

• Do the initial transactions you need to do in any microservice and if other microservices need to be aware, just subscribe them to the integration events. It is also a better approach for the future. If any other microservice needs to be aware, just subscribe it. You won't need to modify the HttpAggregator service. anytime you modify something like that you're opening a door to posibble bugs.
  Integration Events are more aligned to the OPEN/CLOSE principle in SOLID, but for distributed environments. Meaning, your app is open to changes by subscribing microservices to events, but you don't need to modify source code in the original code. :)

Basically, use the HttpAggregation services to only aggregate queries to reduce chattiness. But do not use it to agregate transactions/updates. That will be bad. It'll become a dangerous monolithic thing.

If a client app needs to trigger multiple independent transactions, just call to multiple microservices through the API Gateway, from the client app.
If you want to propagate data across multiple microservices, use async integration events with the Event Bus. Don't do that in the Aggregation service (It'll be a monolithic).

Hope it helps. :)

[kolluri-rk]

@CESARDELATORRE - Thank you so much for your guidance. Yes, its certainly helpful. You just double confirmed my understanding on sharing data between microservices with integration events. Thanks a lot for clarifying me why we should not use HttpAggregation services as business rules orchestrator.

But after looking at Web.Shopping.HttpAggregator/Controllers/BasketController/AddBasketItem, I'm trying to relate this to our business scenario. In AddBasketItem method, its pulling data from Catalog service and preparing BasketDataItem. Then its sent to Basket service. Similar functionality in UpdateAllBasket as well. Basket service is subscribed to Price changed event and Order started event, and perform necessary changes in it. With this, we are not replicating entire Catalog data to Basket service. We are just making catalog data available to Basket service through HttpAggregator. However changes are propagated with integration events.

I'm seeing similarity in our case as we just need few meter attributes in order to perform billing. I'm very much leaning towards maintaining that data through integration events (created/changed/deleted). However after seeing Web.Shopping.HttpAggregator, I'm just trying to evaluate if I can do similar thing while posting a bill. And Billing service will subscribe to meter attribute(s) changes and perform necessary actions. Either cancelling and re-billing pending bills of that meter or push them to a workflow for further review.

Please let me know your thoughts. Thanks again for constantly responding on this thread with your guidance.

[stumpykilo]

So much great and helpful content in this thread. Thanks @CESARDELATORRE and @kolluri-rk for all the replies and links. I'm sold on the api gateway approach and probably using Ocelot for our solution. One question I had was how would you do the gateways without docker? Would you just have everything that is in the ApiGw-Base in each Bff service with its own configuration.json? Thanks again for all of y'alls help!!

[CESARDELATORRE]

@stumpykilo - Without Docker, well, if you want to re-use implementation across multiple API Gateways, it won't be "so clean"...
One approach could be to have multiple IWebHost projects where the main files, the Program.cs and the Startup.cs would be "linked-files" pointing to the base project. Something like that...

Much cleaner when using the same Docker image, of course.. :)

[mvelosop]

Closing this issue now, will reopen if needed.