Openapi-generator: Switching template system from Mustache to Handlebars (or something else)

[wing328]

Description

There were discussions about switching from Mustache to Handlebars (or something else) before. The goal was to simplify the templates as Mustache is a logicless template system and therefore we will need to create a lot of mustache tag(e.g. isInteger, isHttpBasic, etc) every time we need to compare values.

As discussed in https://github.com/swagger-api/swagger-codegen/issues/6077, Handlebars seems to be a good replacement. Here are a couple of questions we need to address:

• are we going to support both at the same time?
• when will we remove Mustache template support?
• Is any tool/script to help migrate the templates from Mustache to Handlebar? (my understanding is that Handlebar is not 100% backward compatible with Mustache)
• for new generators, when will we recommend using Handlebar as the default?
• if users have already customized the Mustache templates, how can they migrate those to Handlebars? (in other words, are we going to provide a tool/script or just a migration guide and users need to manually update the customized Mustache templates)
• let's say we switch the template system from Mustache to Handlebars in v4.0.0, how are we going to sync enhancements, bug fixes from v3.x to v4.0.0 before v4.0.0 is officially released?

The goal of this discussion is to collect feedback from the community and formula a plan to replace Mustache if needed

openapi-generator version

Future major release (4.x or later)

Related issues/PRs

It was previously discussed in https://github.com/swagger-api/swagger-codegen/issues/6077

[wing328]

Here is my view:

• are we going to support both at the same time? Yes with mustache as the primary to start with and later switch to Handlebars as the primary
• when will we remove Mustache template support? If 5.x uses Handlebars as the primary template system, then we should Mustache template support in 6.x.
• Is any tool/script to help migrate the templates from Mustache to Handlebar? (my understanding is that Handlebar is not 100% backward compatible with Mustache) Not that I'm aware of. We can create scripts to ease the migration
• for new generators, when will we recommend using Handlebar as the default? Only after making Handlebar as the primary (default) template system
• if users have already customized the Mustache templates, how can they migrate those to Handlebars? (in other words, are we going to provide a tool/script or just a migration guide and users need to manually update the customized Mustache templates) Ideally we should provide a tool. Otherwise it's unlikely users will upgrade to the latst version of OpenAPI Generator
• let's say we switch the template system from Mustache to Handlebars in v4.0.0, how are we going to sync enhancements, bug fixes from v3.x to v4.0.0 before v4.0.0 is officially released? Ideally we should add the handlebar support and migrate the templates with a tool/script right before the 4.0.0 release so that we can continue to sync enhancements/bug fixes between major versions before that

[grokify]

Here are my thoughts:

• Should we switch to another templating system?
  
  
  
  • There are important benefits.
  
  
  • Both the Java and template code will be cleaner and easier to follow. Logicless templates are great when you actually have no logic, but when you start putting a lot of view logic in your code, it makes sense to consider more powerful templating systems.
  
  
  • More people can participate in building generators because less Java experience will be necessary for template builders. Template syntax is easier to learn and picking a solution that exists in many languages will may adoption easier (see next).
  
  
• Should we switch to Handlebars or something else?
  
  
  
  • An important characteristic for this project is that the template system be supported in many languages. This makes it more likely to be available in a template builder's language of choice and means it is more likely they are familiar with it and willing to invest their time using it, vs. a template system they otherwise would not use. Both Handlebars and Mustache are available in many languages but I'm not aware of other template systems like this.
  
  
• are we going to support both at the same time?
  
  
  
  • It's going to take a while for to transition all the generators and there may be things we learn along the way so supporting both makes a lot of sense.
  
  
• when will we remove Mustache template support?
  
  
  
  • Agree this should be 1 major release after Handlebars support is GA (generally available / non-experimental).
  
  
• Is any tool/script to help migrate the templates from Mustache to Handlebar? (my understanding is that Handlebar is not 100% backward compatible with Mustache)
  
  
  
  • There are a few areas of known incompatibility. We should investigate whether they affect our implementations or not. If they do affect us and there's no automatic transition, we should be able to document them.
  
  
• for new generators, when will we recommend using Handlebar as the default?
  
  
  
  • Recommendation can occur when Handlebars support is GA. I think it's ok for new generators to be built on Handlebars ahead of time if there is desire.
  
  
• if users have already customized the Mustache templates, how can they migrate those to Handlebars? (in other words, are we going to provide a tool/script or just a migration guide and users need to manually update the customized Mustache templates)
  
  
  
  • This is the hardest item for me. Do we know how many people have made customizations and how intensive they are?
  
  
  • The easier we can make the transition the better, but for some things like a tool/script, we should understand if it will help or not do the job. It may be worthwhile to transition some generators ourselves to get more information on what a tool/script could do. It may also be worthwhile to put out a guide and then see what the issues in transition are and how we could assist.
  
  
  • Some additional thoughts:
  
  
  • If the maintainers are available and have time, they may be more than happy to move to a system that is easier to maintain and understand.
  
  
  • If the maintainers are available and don't have time, they may have the same issue with merging current template modifications. It would be good to know if they are taking current modifications.
  
  
  • If the maintainer is no longer there, I've seen SDKs get deprecated.
  
  
• let's say we switch the template system from Mustache to Handlebars in v4.0.0, how are we going to sync enhancements, bug fixes from v3.x to v4.0.0 before v4.0.0 is officially released?
  
  
  
  • Since we are going to "add" Handlebars instead of switching, could we just add it in a 3.x release once it was functional? It could be experimental in 3.x and in beta in 4.x.
  
  

[jmini]

I hope @jimschubert will join this discussion because I think that he has some vision to change the architecture of the responsabilities between the component.

For the moment we have:

• DefaultGenerator: the main class that reads the spec, process them (delegating to the language specific layer), run the template engine and write the files. This is not Language specific.
• Generator classes (like JavaClientCodegen, GoClientCodegen, ...) that are a way to do language specific stuff.
• Codegen Classes like CodegenModel, CodegenParameter, ... that respresent the OpenAPI spec in a way that it is easy to write files (some treatment are made)

If I remember well, @jimschubert wanted to organize responsibilities in an other way.
If you want to introduce multiple template engine, thinking about who is responsible for what make sense.

[grokify]

If the interface to the generators, like JavaClientCodegen, GoClientCodegen, changes and we decide we need to have old and new systems running simultaneously because not all generators should be required to move at the same time, then one decision is whether a new template system would (a) only be on the new architecture or (b) be on the old and new architectures.

If the new template system is only on the new architecture, then it becomes more important to know the timing of the re-architecture.

Some of the re-architecture discussion in https://github.com/OpenAPITools/openapi-generator/issues/503. Should we have some wiki or repo markdown pages to discuss?

[chatelao]

Is Handlebars in the 4.0.x branch already available?

[grokify]

Is Handlebars in the 4.0.x branch already available?

You can see work being done on this in PR https://github.com/OpenAPITools/openapi-generator/pull/690 which hasn't been merged yet.

For now, you can get this in the rienafairefr/openapi-generator:templating branch.

[cognifloyd]

What about adding additional template engines that are never meant to become the default?

I'm in DevOps at a Python+Java (+ some JS though I haven't touched that yet) shop and rarely work with mustache or handlebars. I had to learn mustache to write the templates in #2270. Plus, the handlebars templates I've seen in swagger-codegen seem even more obtuse to me than mustache. Both feel clunky even if there are implementations in many languages.

I work with Jinja almost every day thanks to my DevOps work with Ansible, StackStorm, and other DevOps tools. So, for the python generators (this is the one I'm writing: #2270) in particular, I would prefer to work with Jinja templates to craft the python code. So for a python-focused generator I would like to use HubSpot/jinjava or something like that, but only for that generator. Of course, it's possible to write awful templates no matter the language, but allowing different languages to choose different template engines (assuming a java implementation is available) would be really nice.

[jimschubert]

@cognifloyd the long-term idea of #690 is to allow users to extend the "custom" template engines with whatever they prefer. For instance, to use Jinja this would require the user to create an adapter (or pull one from the community and add to claaspath), then explicitly select the template engine at execution time.

690 gets us a lot closer to that use case than we are today. I plan to get #690 into master this week, with a note that only Mustache is supported for embedded templates.

[dkarlovi]

Since https://github.com/OpenAPITools/openapi-generator/pull/2657 was merged, does that mean Handlebars is now fully supported?

[lindgrenfredrik]

I looking at migrating from Swagger-generator to Openapi-gen due to creating a issue on their git and not receiving any response for months

Currently have Handlebar templates for project so looking at running with it as template engine but found a issue with it:

In generateSupportingFiles in DefaultGenerator.java it looks if files end with correct template extension before compiling them as templates otherwise just writing them as files.
This works fine when mustache files registered for the different languages end with correct mustache but when switching engine it expects handlebars extension which isn't whats registered by "language" generator
The name conversion is done later in current setup (that is why it works for i.e. model files), which never gets called for Supporting files.

Any ideas on how to solve this properly?

• Allow mustache extension for any template engine?
• Convert names earlier?
  -- Register template files in some other manner to append correct ending?
• Run compile for all supporting files regardless of extension?

Should I write separate issue for this?

My temporary workaround is to allow mustache named targets to pass into compile section
https://github.com/OpenAPITools/openapi-generator/compare/master...lindgrenfredrik:aspnetcore_handlebars#diff-bd1652e990ffe227072a5c8908fd3054

[behrangsa]

Looks like this is merged to master. Is there a version based on master available?

[jimschubert]

@behrangsa I'm not sure what you're referring to as merged. We've had handlebars support throughout the 4.0 release. We don't have any built-in handlebars templates, and no real plans on the horizon to convert from mustache to handlebars as this would be an enormous effort.