Generator-jhipster: Add Couchbase support to jhipster

[tchlyah]

• Overview of the request

Couchbase is an open source, NoSQL document database, known for it's distributed architecture delivering consistent high performance, at any scale. It's also use N1QL, which extends SQL for JSON Document. And comes with a powerful and very complete administration interface.

The aim of this request is to add Couchbase DB compatibility to jhipster generated apps.

• Motivation for

As jhipster is using Spring Data for MongoDB, and there is an implementation for Couchbase (spring-data-couchbase) which is actively maintained, it is easy enough to add Couchbase support to jhipster.

• Suggest a Fix

I propose to implement the compatibility using :

• spring-data-couchbase, using N1QL repositories
• Couchmove : Java data migration tool for Couchbase, just like liquibase, but based on Flyway simplicity.
• testcontainers-java-module-couchbase : Testcontainer module for Couchbase, which uses docker to start an initialized instance of Couchbase for tests.

I already have a working version, with all tests passing, of a monolithic jhipster generated app based on Couchbase.
All I need is your confirmation to to begin work on a PR!

[deepu105]

I'm personally OK with it but I'll defer the decision here to @jdubois

[gzsombor]

Adding more backend are an interesting challenge which came up a couple of times. Unfortunately it's not easy to do it as a module 😕

[PierreBesson]

Personnaly I would love to see couchbase support in JHipster. Especially if it already has an integration with Spring data and a db migration tool (which it seems you are the developper).

But I have some reservation about it. Does couchbase have a big enough user base so that we will have feedback on it. Similarly I added Consul support almost a year ago and although I know some people do use it. I have had almost zero feedback and I have not even had the opportunity to use it myself.

So I'm not sure we should add yet another "exotic" option.

[gmarziou]

Similarly I added Consul support almost a year ago and although I know some people do use it. I have had almost zero feedback and I have not even had the opportunity to use it myself.

Here's one for you Pierre: I think it's great to have it in JHipster. I'm probably going to use it on a project that mixes java and C++ apps which are using TCP not HTTP. Eureka is fine as long as you stay in java world and you have sources for your apps. I like Consul's DNS forwarding for these legacy apps and also Vault is a great tool to pair with.

Sorry for the side discussion :)

Now about Couchbase, I haven't used it yet but I think your proposal is strong. Using TestContainers for testing is interesting even though some may argue that it adds docker requirement.

[jdubois]

+1 from me, let me explain my motivation here:

• As long as this uses Spring Data, it's probably quite close to what we already have with MongoDB - so for me this shouldn't be hard to maintain, and it shouldn't add a huge maintenance cost as this option is quite well separated from the others
• I've had discussions this week about migrating to the new Spring Data Cassandra -> this would makes even more sense
• And I had talks today with friends from Google to integrate Google Cloud Datastore, which doesn't work with Spring Data and is a proprietary competitor

Now, I agree with @PierreBesson that we shouldn't use databases with too few users, as the community interest won't be big enough. This is typically the issue today with Cassandra, as in reality nobody uses it (which is very disappointing to me). We'll see this in our stats once it's merged: if this is annoying to maintain, and if the stats are bad, we could remove it in the future.

[tchlyah]

But I have some reservation about it. Does couchbase have a big enough user base so that we will have feedback on it.

Couchbase is becoming widely used, here some examples :

• Criteo uses over 1000 nodes, with less that 100 ms latency, and 10M hits/sec.
• Linkedin : 400k op/sec on only 4 nodes
• Amadeus : 2,5M op/sec, 30 nodes, 1 Tb of Data
• Viber : migrated from MongoDB as it couldn't handle growth of data
• Paypal : 3-10 TB of data, 10 ms latency
• Pokemon Go
• Skyscanner

[ctamisier]

I'm using JHipster and need to store data into cassandra but not the users. I just have few users and other relations which totally fit in a SQL database. Then I need to store a lot of "Timestamped event" where the cassandra storage model is needed.
So I generated a project with PostgreSQL and then "get inspired" from the generated cassandra configuration code and apply it to my needs.
So I would suggest to eventually being able to have cassandra (or mongodb, couchbase..) to a specific domain (I mean not necessarily the users and authorities).

[sendilkumarn]

+1 from me for couchbase and of course we need to make sure that there is enough user base who will be using it.

recent interaction with someone in couchbase team says the same set of companies listed above. We have to see the trend of new users using couchbase, if that is substantial amount then we can decide to do that.

[deepu105]

@differentway so since majority agrees I think you have a green signal for PR :smile:
@ctauro this should be interesting to you

[sendilkumarn]

@deepu105 I was actually searching for @ctauro 😝

[pascalgrimaud]

+1 too, if:

• not hard to maintain
• you won't disappear :ghost: as soon as your PR is merged (so you can help us to maintain if needed)

[tchlyah]

• not hard to maintain

Until now there is no such big difference between MongoDB and Couchbase code, so I don't think it will be hard to maintain

• you won't disappear :ghost: as soon as your PR is merged (so you can help us to maintain if needed)

I won't! If I propose to do that it is because I intend to use it on my projects. I also have some other ideas for jhipster.

[stevehouel]

+1

[delkant]

Hey @differentway I think this is awesome.

I already have a working version, with all tests passing, of a monolithic jhipster generated app based on Couchbase.

Can your version be accessible already?

[tchlyah]

Can your version be accessible already?

You can find a first version here : generator-jhipster/couchbase.
For now it can generate a Couchbase monolithic app with JWT authentication only. It doesn't support entities for now, I'm working on it.
If you have any remarks please keep me in touch

[jdubois]

Any progress @tchlyah ? Should we keep this opened?

[tchlyah]

Any progress @tchlyah ? Should we keep this opened?

Working on the generator is not as easy as I tought, especially with a new database. For now I'm pretty close, you can follow my work on my repo (generator-jhipster/couchbase)

Here is my progress :

• [x] Database migration with Couchmove
• [x] Tests with Docker : testcontainers-java-module-couchbase
• [x] Monolithic app
• [x] Entities
• [x] Authentication :
  
  
  
  • [x] JWT
  
  
  • [x] OAuth2
  
  
  • [x] Session
  
  
  • [x] Social login
  
  
• [x] Microservices app
• [x] generator tests :
  
  
  
  • [x] Yeoman tests
  
  
  • [x] Travis sample tests
  
  
• [x] Docker compose scripts
• [ ] Cloud deployment (Heroku, Cloud foundry, ...)

So all that remain is a slightly bug in Couchmove and Cloud deployment.

BTW since I have no experience in any cloud plateform, help is welcome to quicly finish this last part!

[stevehouel]

I can help you if you want with cloud integration. Could you describe
remaining tasks for that part ??? I will get your branch asap and work on
it.
Le lun. 21 août 2017 à 19:43, Tayeb Chlyah [e-mail] a
écrit :

Any progress @tchlyah https://github.com/tchlyah ? Should we keep this
opened?

Working on the generator is not as easy as I tought, especially with a new
database. For now I'm pretty close, you can follow my work on my repo (
generator-jhipster/couchbase
https://github.com/differentway/generator-jhipster/tree/couchbase)

Here is my progress :

• Database migration with Couchmove
  https://github.com/differentway/couchmove
• Tests with Docker : testcontainers-java-module-couchbase
  https://github.com/differentway/testcontainers-java-module-couchbase
• Monolithic app
• Entities
• Authentication :
  
  
  
  • JWT
  
  
  • OAuth2
  
  
  • Session
  
  
  • Social login
  
  
• Microservices app
• generator tests :
  
  
  
  • Yeoman tests
  
  
  • Travis sample tests
  
  
• Docker compose scripts
• Cloud deployment (Heroku, Cloud foundry, ...)

So all that remain is a slightly bug in Couchmove
https://github.com/differentway/couchmove and Cloud deployment.

BTW since I have no experience in any cloud plateform, help is welcome to
quicly finish this last part!

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/jhipster/generator-jhipster/issues/6086#issuecomment-323807410,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ARWTrX-L9zE2G2a-g7sEi4GUUavhgQ_2ks5sacHZgaJpZM4OUVI1
.

[tchlyah]

I can help you if you want with cloud integration. Could you describe
remaining tasks for that part ??? I will get your branch asap and work on
it.

For now I didn't touch this part! So you can get my branch and see what I should do. Thanks a lot

[stevehouel]

Ok I do it asap :) and you r welcome :)
Le lun. 21 août 2017 à 20:07, Tayeb Chlyah [e-mail] a
écrit :

I can help you if you want with cloud integration. Could you describe
remaining tasks for that part ??? I will get your branch asap and work on
it.

For now I didn't touch this part! So you can get my branch and see what I
should do. Thanks a lot

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/jhipster/generator-jhipster/issues/6086#issuecomment-323813230,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ARWTrW0WpxeoZuhy64cL0EcQ1S0H9Ozdks5sacd7gaJpZM4OUVI1
.

[tchlyah]

Ok I do it asap :) and you r welcome :)

@stevehouel, any advancement?

[stevehouel]

No sorry a bit busy right now but till next week. After that i should be
free to work on it
Le jeu. 31 août 2017 à 14:30, Tayeb Chlyah [e-mail] a
écrit :

Ok I do it asap :) and you r welcome :)

@stevehouel https://github.com/stevehouel, any advancement?

—
You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub
https://github.com/jhipster/generator-jhipster/issues/6086#issuecomment-326280731,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ARWTre99NIxgv0AXczQB-pjIo5pMXq2Jks5sdqdogaJpZM4OUVI1
.

[pascalgrimaud]

No worry about Cloud deployment, it can be done in a later PR.

[tchlyah]

No worry about Cloud deployment, it can be done in a later PR.

Perfect then! I'm now on vacation, I'll be back next week, as soon as I return I will make the PR. Hope that @stevehouel will finish it before my PR 😉

[hdurix]

This can be closed now :-)
Next targets: Couchbase 5, Spring Data Couchbase 3, full text search, ...