Generator-jhipster: External templates in JHipster

[deepu105]

Reading https://twitter.com/nodenpm/status/909318429236219905 gave me an idea for our plugin system.

Background

Currently, we support modules but they can only add in functionality and they have to modify stuff we have generated using regex and stuff. It works but it's not the ideal way to extend the features supported by the generator.

There have been requests for stuff like Scala, Kotlin etc on the server side and Vue etc on the client side, supporting everything in the core generator would be cumbersome and already our core is humongous and its growing with each new feature we add. We already face issues in maintenance due to the size.

Some people might want to use a specific customization in a part of the generated code so for them they could use their own template for that alone without having to fork and maintain the entire generator.

This model also puts a lot of maintenance responsibility on us and makes the generator too monolithic and tightly coupled.

Solution

It's a good news that we already split our core to sub-generators with separate client and server. We also use the yeoman composability feature to compose these together. This gives us the opportunity to let people use external modules instead of ours for a part of the generator. For example today we compose client, server and language generators to create an app, we could let users use a different client instead of ours, this external client will be yeoman generator which would follow our guidelines to say provide Vue as the client or polymer or whatever they want. The same goes for server-side as well. A sample usage would look something like
Say there is a Kotlin server side provided in a generator called generator-jhipster-kotlin. The generator will expose three sub-generators like below

...
|_ generators
   |_ server
   |_ entity
   |_ service
|_ test
...

Now the user can execute JHipster and specify to use a specific external template

jhipster --template generator-jhipster-kotlin
Now when our CLI executes a sub-generator or when we compose a sub-generator we will first look at the specified template and see if it implements the required sub generator and use it else we fall back to the one from jhipster. So in this case, since the module has server sub gen we will use that instead of our own and let it handle the server side prompts, templates etc. This way the module maintainer is free to do whatever they want and does not have to rely on us for anything. We will record this in the .yo-rc.json and will use the sub-generator from the module for every command and fallback to ours when the module doesn't implement a sub-generator.
So when the user does jhipster entity myEntity it will execute the one from the kotlin module and not ours but for anything other than server, entity, service it will use the jhipster sub-generators.

I propose to start this with a new command line flags --template to begin with. It is easy to do as we already use composability so it's just changing a little bit code to call an external sub gen instead of internal in few places and writing up some docs for the module writers on how to utilize this. Future improvement could be;

1. Split server-side sub gen further so that server-side code, DB, testing can be further modularized and we can replace parts of the template like say --template generator-jhipster-neo4j or --template generator-jhipster-keycloak and so on.
2. Enables us to move our angularjs, angular, react clients into individual modules rather than having them all in the same generator this will reduce maintenance burden a bit and will be easy to let them evolve independently

@jhipster/developers let me know what you think.

TODO:

• [ ] Test edge cases
• [ ] Split entity subgen
• [ ] add tests where possible
• [ ] initial docs

[gmarziou]

Great idea.
I find "--template" a little confusing as we already use this word for different things.
What about "blueprint", "scaffold", "variant", "flavor" or anything else?

[deepu105]

Yes the --template is just an idea, we could find a better word of course

[gmarziou]

It could be hard to maintain coherence between all these variants.
To make sure Kotlin server works with React client as well as with Java server, we might need to standardize server REST API which probably means writing some independent compatibility tests, some kind of the TCK for JEE.

[deepu105]

I would expect the module maintainers to make sure that it works with the client or whatever JHipster has, that's the whole point of this. We would only mandate certain conventions

[sendilkumarn]

This is an awesome idea. But instead of overloading with some flags in
command line. Why can't we integrate it into our questions itself?

Thanks
Sendil Kumar N

On 18 Sep 2017 01:05, "Deepu K Sasidharan" [e-mail] wrote:

I would expect the module maintainers to make sure that it works with the
client or whatever JHipster has, that's the whole point of this. We would
only mandate certain conventions

—
You are receiving this because you are on a team that was mentioned.
Reply to this email directly, view it on GitHub
https://github.com/jhipster/generator-jhipster/issues/6372#issuecomment-330077699,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AL5LUv67IdpyDNfG30XXRXuhtJYZTDVXks5sjXR1gaJpZM4PaGwz
.

[atomfrede]

👍 I like the idea. If it makes maintaining modules more easy and independent from the main generator that would be awesome.

[Tcharl]

What about using xtend? The amount of work would be far more, but the templates would be far more easier, modular, extensible, ...

[deepu105]

@Tcharl could you be more specific I dont understand what you meant

[deepu105]

@sendilkumarn I would expect this to be a more power user feature and hence flag, ofcourse if it becomes very popular we could later make it a question

[Tcharl]

IMHO, yeoman generator is kind of 'limited' for the what is becoming jhipster: it lacks of some advanced feature that, for example, forbid to implement inheritance, complexify stuff (for example, you can't can't reuse some angular patterns in react) and lead to some ugly copy paste patterns, ...
Using advanced templating frameworks (like xtend backed by an xtext dsl) could help to have a better modularization, readability, etc. on the templating engine. However, it would mean a complete generator-jhipster (and modules) rewrite so it would be kind of an impacting stuff.

[deepu105]

@Tcharl we are not gonna rewrite the entire platform man, that's an insane amount of work. Agree Yeoman is limiting sometimes but it gets the work done and is easy to use and best of all its JS. So let's not pollute this thread with that discussion.

[Tcharl]

@deepu105 please accept my apologies. It was just a suggestion.

[deepu105]

@Tcharl hey no need to apologize, you did nothing wrong. I learned about xtend this way, my point was just to ensure that we dont divert from the topic at hand.

[Tcharl]

However, I like the idea of splitting the generator in multiple composable yo templates.

As long as the split is clear (from a consumer POV) on what is an external generators and what is given as option.

I even think that with a (slightly) better folder split, we could let the user override the part of any template he want:
For example, if we had a structure like this for angular:
entities/templates/client/angular/src/main/webapp/app/entities
|_____delete
|_____dialog
|_____details

Replacing one of this folder would override the generation (the delete component, the dialog one, or the details one).

It would let some company/contributors to override only the parts of template they want (with a little injection magic).
Am I totally out of scope?

[jdubois]

The main issue for me, as @gmarziou says, is that we would need some kind of TCK. And I have no idea how this could be written - that's a huge job as far as I can tell.
Then, I'd love to have different backends: Kotlin and Play look like the easiest to do, but I have a customer who could be interested by a .NET implementation... That's the whole point of microservices. Then, if we target too many different languages, maybe we need to totally switch to Consul (Eureka will work in Java of course, and in .NET with https://steeltoe.io/ - but at some point it will not be enough)

[deepu105]

@jdubois @gmarziou don't worry I have a clear idea of how to do this(I already did it partially in my old React module). I don't think it's a lot of work to start and I can, of course, write the initial docs as well.

[gmarziou]

API compatibility testing could be also be done by comparing generated swagger json spec at least for server variants using springfox

[deepu105]

If nobody has objections I will do a POC when I have time to kill.

[jdubois]

This could also be a good solution for companies who want their own
templates: for example they could fork our Angular templates to have their
own customized front end.
If this use case works I can find some people to help here.
Then this is not urgent - at the moment I'd rather focus on React support.

[deepu105]

Yes, this is not urgent but it's fairly easy to do, probably an hours work to code in the support and another hour to write some docs, I'll see when I have some time in between other stuff. Especially at times when you are doing some React stuff and get very frustrated with it :smiling_imp:

Once the support is added I'll talk to some interested people who wanted to do a Kotlin backend

[gmarziou]

As a side note while talking about templates, maybe we could benefit from angular-cli new feature: schematics.

[deepu105]

closing this as I'll manage the remaining TODOs