Generator-jhipster: Add RabbitMQ support with Spring Cloud

[danielpetisme]

Overview of the issue

Currently, when a developer wants to use Spring cloud the default choice Apache Kafka as message broker. By default, Spring Cloud provides 2 binders: Kafka and RabbitMQ.

Kafka is an excellent Message broker but it's overkill for small projects. Plus, it requires to manage zookeeper which is yet another component to maintain. RabbitMQ is a first-class message broker well known on the market.

Motivation for or Use Case

Reproduce the error

Related issues

Suggest a Fix

Most of the job is already done, after a first look it's all about providing the appropriate dependencies and a bit of YAML config (< 10 lines of code).

JHipster Version(s)

JHipster configuration

Entity configuration(s) entityName.json files generated in the .jhipster directory

Browsers and Operating System

• [ ] Checking this box is mandatory (this is just to show you read everything)

[PierreBesson]

I had thought of adding the support sometime ago as spring cloud stream offer the same abstraction for Rabbitmq and Kafka. Indeed it's not too hard to do. The time consuming part will be doing the docker-compose support. If you submit this as a PR, I can do some review and testing.

Initially for Kafka, the idea was to generate something more complex than what we have today. @jdubois had made a demo showing end to end streaming from Kafka to the front-end through websockets. But in the meantime the new angular was released and we moved to other things...

[jdubois]

Yes that would be great but we have so many opened tickets and issues... No time to add this kind of thing, that would require quite a lot of testing... How about doing a module? I'm sorry but at the moment I just can't follow everything

[danielpetisme]

@jdubois no worries. The main issue is that in the curent JHipster config Spring Cloud Stream is tightly coupled to Kafka. The files generation itself isn't hard but it will require some time to refactor the generators questions (Cloud Stream should be optional and then a question should ask to choose between the 2 binders).

I can provide the rabbitmq docker image and the maven|gradle configs (I am using it already) but won't have time for the generator :/.

@PierreBesson could you help ? Otherwise I propose to flag it as frozen until next year.

[PierreBesson]

@danielpetisme Yes I can help.
We can work on the PR together. Can you create a sample project modified from a kafka app so we can see what changes between the two configurations.

[pascalgrimaud]

Although there is few lines of code, not sure if we should add to the main generator-jhipster.

Like said, there are too much feature request and it would be exactly the same to do it as a module.

So for me, it should go to a module.

[deepu105]

We are in a kind of feature freeze for some time so its best to start it as a module and it bringing it into official generator can be reconsidered later. Closing the ticket for now but feel free to continue the discussion here

[PierreBesson]

For me this is an important feature for our spring cloud stack integration. Messaging is an important part of a complete microservice architecture and I think we should have this Rabbitmq option.

Then I also agree to close this for the moment as we have a lot of things currently in progress.

[danielpetisme]

@PierreBesson hsa promised I made a sample app with RabbitMq
https://github.com/danielpetisme/jhipster-sample-app-rabbitmq

You'll find 2 branches, master is the init of a monolith with Kafka support, binder-rabbit is ... the rabbitmq adapatation.
The diff:
https://github.com/danielpetisme/jhipster-sample-app-rabbitmq/compare/master...rabbit-binder

I made a REST Endpoint /api/messages,

• A producer sends the message in JSON format to a topic-jhipster destination
• A consumer reads the message from the topic-jhipster destination and save it into a list
  I've added a rabbit routing key just for the sake of demo

[danielpetisme]

Regarding the core vs module debate.

It's hard (an subjective) to define what should be or not into the core. For me, I don't see a clear line except the "AlreadyTooMuchFeaturesException" (which is understandable) between core and modules. I think jhipster is reaching a critical size and it's growing extremelly fast but that's another topic :wink:

[deepu105]

AlreadyTooMuchFeaturesException that's exactly the main problem as anything on the core adds the maintenance burden on the core team (which is pretty small and number of active maintainers among it is even smaller) we are already struggling to balance and JHipster is growing at an alarming rate and maintaining the level of quality we have now would be difficult if we keep on adding options without perfecting existing ones

[danielpetisme]

@deepu105 👍
Module or your external templates proposal could be next big architecture choice IMHO but it requires a core team decision.
Without adopting an Apache or JavaEE (RIP) approach Jhipster could be a kind of meta-blueprint and core-team supported minimal feature.
JHipster == orchestration between client, server, docker, db etc. and module are Angular, Spring, MongoDB, etc. Each module would be responsible of it's integration. Rather than having a huge central code generator and a lof of ifs.

Anyway, It's not the purpose of this issue (even if it's an interesting topic) so let's close it right now I gave @PierreBesson a sample app we'll see in the future months how things are moving forward.

[pascalgrimaud]

@danielpetisme : I discussed with Pierre today about this. I think it's really interesting, and it should definitely go to module.

You can host the repo on your github account, or we can create one common in hipster-labs and invite you as collaborator.
What do you prefer ?

[danielpetisme]

Hi,

As discussed I won't have much time to spend on it before end of year (more than the sample app linked above).
I would say in hispter-labs to permit @PierreBesson to work freely on the module if it has time. Otherwise I'll start a module early next year (2018).

Sounds good?

[frank8559]

Hello,

I would really like to try to help this beautiful project.
and I think that starting with a rabbitmq module seems like a good start. maybe I'm wrong. I am mainly Java backend developer, with angular knowledge but very little node.
But I'm wondering about the architecture of the generator. The border between core and module can be thin and I understand perfectly the problem.
For me everything that is not necessary to the basic operation of an application should be a module (kafka, elasticksearch for example).
I like the idea of external template for personalized code.
Why not take out kafka and do a spring cloud module that will eventually generate kafka and rabbitmq and others?

[PierreBesson]

I have initialized a new repository to host the module : https://github.com/hipster-labs/generator-jhipster-spring-cloud-stream
I have given @pascalgrimaud, @danielpetisme and @frank8559 write access to the project.
First we will develop the rabbitmq support and then work on moving the main generator kafka code to this new module. Hopefully this will be a proof of concept for modularizing our code base.

@jdubois, @deepu105 is this alright for you ?

[jdubois]

+1 for the RabbitMQ module of course
For Kafka: migrating it outside of the main generator would simplify our code, so in the long run that's better, but no need to rush here. And that's also more work, so let's not do it now, we really have too much opened tickets.

We also need to have a better marketplace for those: tags to find modules, and also have some "official" modules...

[antarus]

Can i help too ?

[PierreBesson]

Thanks for your PR on the module @antarus. I have added you to collaborators on the project.