Mustache.js: ES Module support

[zekth]

Hello!

I was wondering if there is a plan to support an ES module version of Mustache.js? If so i can contribute to it.

Why?
I'm working on the Deno project, and there is a discussion about templating (Deno only supports ES modules). Problem is there is no template renderer at the moment. One idea was to port mustache to Deno, so a port means a different code stream which needs maintainance. Having it directly in mustache would be a benefit for both projects i think. But in the case of mustache it means having a mustache.mjs added to the repo,edit: or change the mustache.js to make it ES module compatible.

Your thoughts?

ref: https://github.com/denoland/deno_std/issues/391

[phillipj]

Hi @zekth!

Exciting plans you've got with deno going forward. I'm very positive to making this project ES module compatible, and completely agree with your thoughts on avoiding a port.

You got any concrete thoughts what would be necessary to make this happen?

[zekth]

Here is a (really ugly) port i've done this morning (in typescript) https://github.com/zekth/deno_mustache/blob/master/mod.ts

i've ported some test just for the beginning.
The only concern i have is all the different environments you support with mustache, having the main js file ES module compatible is better than another mjs i think but i don't want to break anything :)

[phillipj]

Thanks a lot for the reference! 👍

Since I've got zero experience with deno, I'll fire off some trivial questions to trigger some discussions:

1. What requirements to deno have to make this work? Does it need to be a .mjs file or does it care about the type="module" of package.json?
2. Any file naming requirements?
3. Does TypeScript matter here somehow?

..and so on, any for-dummies-clues would be much appreciated. At the moment I've got too little knowledge about deno to think creatively about plausible ways of tackling this.

[zekth]

1. There is no special requirement as it only needs the library to be an ES module. Example lodash works without any porting. Loading of the module is made throught HTTP fetching, no package json or anything.
2. No naming convention
3. You can use TypeScript of JavaScript, Deno handles both.

For example using mustache in deno would look like:

import * as mustache from 'https://raw.githubusercontent.com/janl/mustache.js/master/mustache.js'
mustache.render(......

If you want more infos you can check this talk from Ryan: https://www.youtube.com/watch?v=z6JRlx5NC9E

[phillipj]

Cool!

So here's some ramblings to share some thoughts & context. My understanding of ES modules are that they're meant to be syntactically parsable in terms of what's exported. That's in dark contrast to its precursors like CommonJS, AMD etc which is a lot more dynamic in nature.

With that in mind, I'm also assuming the IIFE surrounding the body of mustache.js today, meant to feature detect the module system the code currently runs in, won't work in ES module land -- please correct me if I'm wrong!

That means there has to be a .js | .mjs | .ts file that has a plain and simple export ... like you have in your port: zekth/deno_mustache/mod.ts#L689.

The fun starts when we also want to keep the old behaviour, to stay compatible with projects using other module systems or not even a module system at all, hence the need for the mentioned IIFE wrapper :thinking: And as a reminder, that means projects running on servers and browsers.

Since we surely don't want to keep two different implementations (ES modules + the rest) in sync, it starts to feel like it's worth considering a build step. E.g. if we wrote the source code in ES module style, then a build step could convert it to non-ES module like we have today. Or the other way around if that's less intrusive.

As a final note, I'm a big fan of doing as small steps as possible. I'm generally positive to TypeScript, but is it okey for now if we keep focus on ES modules and do a different round focused on whether or not to convert to TypeScript/export typedefs?

Any other thoughts or corrections that comes to mind?

[zekth]

You're totally right, first step would be to add the TypeScript stack and use the compiler to output multiple files like commonjs / ES5 / ES6 and so on ( https://www.typescriptlang.org/docs/handbook/compiler-options.html ) . Using TS compiler does not force us to use TypeScript, we can use ES6 code also.

[phillipj]

Oh that's a great suggestion! Trying to use the TS compiler as an ordinary build tool from ES -> other module systems at first. That will make a plausible TS conversion later a lot less risky.

Are you able to give that approach a shot? Not necessarily solving everything in one go, but the first building blocks at least. Would be really valuable to have something concrete to look at and base future discussions around.

[phillipj]

Sooo I had a go at making a transition into an ES module.

My proof of concept ended up using rollup.js instead of the TypeScript compiler (or babel) primarily because of the UMD version they generate -- we still need that to keep compatibility to older module systems or no module system at all.

Any chance you're willing to give it a test run with deno to see if it works as you expect? phillipj/mustache.js#esm-ify mustache.mjs

[zekth]

@phillipj will do!

[zekth]

Some errors but i think those are minor fixes:

error TS2339: Property 'render' does not exist on type '{ name: string; version: string; tags: string[]; }'.

► file:///Users/vlegoff/projects/genesys/github/telemetry/t/mustache.ts:10:23

10 var output = mustache.render('{{title}} spends {{calc}}', view);
                         ~~~~~~

error TS2554: Expected 2 arguments, but got 1.

► https://raw.githubusercontent.com/phillipj/mustache.js/esm-ify/mustache.mjs:525:52

525   var context = (view instanceof Context) ? view : new Context(view);
                                                       ~~~~~~~~~~~~~~~~~

  An argument for 'parentContext' was not provided.

    ► https://raw.githubusercontent.com/phillipj/mustache.js/esm-ify/mustache.mjs:377:25

    377 function Context (view, parentContext) {
                                ~~~~~~~~~~~~~


error TS2339: Property 'escape' does not exist on type '{ name: string; version: string; tags: string[]; }'.

► https://raw.githubusercontent.com/phillipj/mustache.js/esm-ify/mustache.mjs:640:21

640     return mustache.escape(value);
                        ~~~~~~

error TS2339: Property 'clearCache' does not exist on type '{ name: string; version: string; tags: string[]; }'.

► https://raw.githubusercontent.com/phillipj/mustache.js/esm-ify/mustache.mjs:659:10

659 mustache.clearCache = function clearCache () {
             ~~~~~~~~~~

error TS2339: Property 'parse' does not exist on type '{ name: string; version: string; tags: string[]; }'.

► https://raw.githubusercontent.com/phillipj/mustache.js/esm-ify/mustache.mjs:668:10

668 mustache.parse = function parse (template, tags) {
             ~~~~~

error TS2339: Property 'render' does not exist on type '{ name: string; version: string; tags: string[]; }'.

► https://raw.githubusercontent.com/phillipj/mustache.js/esm-ify/mustache.mjs:678:10

678 mustache.render = function render (template, view, partials, tags) {
             ~~~~~~

error TS2339: Property 'to_html' does not exist on type '{ name: string; version: string; tags: string[]; }'.

► https://raw.githubusercontent.com/phillipj/mustache.js/esm-ify/mustache.mjs:690:10

690 mustache.to_html = function to_html (template, view, partials, send) {
             ~~~~~~~

error TS2339: Property 'render' does not exist on type '{ name: string; version: string; tags: string[]; }'.

► https://raw.githubusercontent.com/phillipj/mustache.js/esm-ify/mustache.mjs:693:25

693   var result = mustache.render(template, view, partials);
                            ~~~~~~

error TS2339: Property 'escape' does not exist on type '{ name: string; version: string; tags: string[]; }'.

► https://raw.githubusercontent.com/phillipj/mustache.js/esm-ify/mustache.mjs:704:10

704 mustache.escape = escapeHtml;
             ~~~~~~

error TS2339: Property 'Scanner' does not exist on type '{ name: string; version: string; tags: string[]; }'.

► https://raw.githubusercontent.com/phillipj/mustache.js/esm-ify/mustache.mjs:707:10

707 mustache.Scanner = Scanner;
             ~~~~~~~

error TS2339: Property 'Context' does not exist on type '{ name: string; version: string; tags: string[]; }'.

► https://raw.githubusercontent.com/phillipj/mustache.js/esm-ify/mustache.mjs:708:10

708 mustache.Context = Context;
             ~~~~~~~

error TS2339: Property 'Writer' does not exist on type '{ name: string; version: string; tags: string[]; }'.

► https://raw.githubusercontent.com/phillipj/mustache.js/esm-ify/mustache.mjs:709:10

709 mustache.Writer = Writer;
             ~~~~~~


Found 12 errors.

code:

import mustache from 'https://raw.githubusercontent.com/phillipj/mustache.js/esm-ify/mustache.mjs';

var view = {
  title: 'Joe',
  calc: function() {
    return 2 + 4;
  },
};

var output = mustache.render('{{title}} spends {{calc}}', view);

console.log(output);

want me to try to make a PR on your .mjs file?

[phillipj]

Thanks!

Realise now that I should shared a little more context about what I tried doing initially, sorry.

As we discussed earlier, my initial attempt with the TypeScript compiler also yielded quite a few errors, somewhat similar to the ones you provided now. Nevertheless it surprised me that it did generate output and it was very close to what I wanted.

What I wasn't pleased with was the UMD code it wrapped around the compiled output. Mainly two things:

1. It does not have the fallback of exposing the package contents onto the global scope (think window.Mustache). This is vital for our users that does not have a module system in place.
2. In projects with CommonJS, this package's contents is exposed as module.exports.default rather than module.exports. Although that might be the correct and expected behaviour when CommonJS require()s an ES module, it will break backwards compatibility, which I was hoping to avoid.

As my main goal was to first and foremost to transition the source code to be an ES module, not introducing TypeScript, I decided to try different compilers/bundlers to see if they did things differently to avoid the two challenges above.

Hence the reason I ended up with rollup.js. It's UMD output is what we need and it doesn't cause any breaking changes to users of this package.

Sooo to my actual question; do we need to care about TypeScript at the moment?

My understanding from earlier in this discussion was we could consider not fully transition to TypeScript yet, as it would help deno nonetheless if it is indeed an ES module, but I might have misunderstood that a bit?

[zekth]

I think the main problem is the first initialisation of mustache like here :
https://github.com/janl/mustache.js/blob/master/mustache.js#L14

and also this: https://github.com/janl/mustache.js/blob/master/mustache.js#L536
can be rewritten as new Context(view, null)

don't you think?

[phillipj]

I think the main problem is the first initialisation of mustache ..

You're probably right. In the .mjs version I tried moving that into the actual source code atleast, instead of being an object passed in from the UMD wrapper:

var mustache = {
  name: 'mustache.js',
  version: version,
  tags: [ '{{', '}}' ]
}

Would be cool to see your approach which would fix those TypeScript errors 👍

can be rewritten as new Context(view, null)

Almost.. Isn't new Context(view, undefined) the equivalent of not passing a second argument at all?

[zekth]

Yes correct about the new Context

I'll try something so :)

[zekth]

Here is the PR: https://github.com/phillipj/mustache.js/pull/1
CI is broken but i don't get why i got those linting messages

[phillipj]

Awesome, thanks a lot! Pretty busy the next couple of days, I'll do my best to review before the week ends.

[phillipj]

Status update; as I mentioned in the PR you opened against my esm-ify branch, I now got a few tests in place that ensures this package works as intended for different module systems we've supported for years in #724.

I'll let that brew for a couple of days as I just pushed a few improvements, just in case anyone has any objections or more tweaks in mind.

With those tests in place, I feel comfortable going forward transitioning this project to having the source code written as an ES module and build step to produce what we've got in mustache.js today -- that'll ofcourse be done in an upcoming PR.

[phillipj]

728 has been opened for public scrutiny.