Tools: Tagged template literals incorrectly transpiled to es5

Created on 18 Jan 2018 · 13Comments · Source: Polymer/tools

Tagged template literals are transpiled incorrectly to es5. _templateObject variables are placed outside script closures and end up on the global scope. This causes multiple tagged template literals to override each other.

I've investigated this, and it seems that babel is expecting to be in a module scope rather than a global scope. I am not sure if this is due to how babel is configured inside the polymer cli, or a bug within the babel transformer.

The same transpilation done by the typescript compiler provides valid output, it does not place variables outside closures.

This affects polyserve as well as polymer build. It happens on .js and .html files.

A minimal reproduction is like this:

Given two files:

html-file-a.html

<link rel="import" href="../bower_components/polymer/lib/utils/html-tag.html">
<link rel="import" href="./html-file-b.html">

<script>
  (function () {
    const a = window.Polymer.html`<div>A</div>`;
  })();
</script>

html-file-b.html

<script>
  (function () {
    const b = window.Polymer.html`<div>B</div>`;
  })();
</script>

And the following polymer.json:

{
  "entrypoint": "index.html",
  "shell": "src/html-file-a.html",
  "fragments": [
  ],
  "sources": [
    "src/**/*"
  ],
  "extraDependencies": [
    "bower_components/webcomponentsjs/*"
  ],
  "builds": [
    {
      "preset": "es5-bundled"
    }
  ]
}

Produces the following output:

<html><head></head>
<body>
  <div hidden="" by-polymer-bundler="">
    <script>... insertions by polymer omitted for brevity...</script>
    <script>var _templateObject = babelHelpers.taggedTemplateLiteral(["<div>B</div>"], ["<div>B</div>"]); (function () { window.Polymer.html(_templateObject) })();</script>
    <script>var _templateObject = babelHelpers.taggedTemplateLiteral(["<div>A</div>"], ["<div>A</div>"]); (function () { window.Polymer.html(_templateObject) })();</script>
  </div>
</body>
</html>

Versions:
Polymer CLI v1.5.7
Polymer Build v2.1.1
Polymer Bundler v3.1.1

cli High Available Bug

Source

LarsDenBakker

Most helpful comment

I implemented Option 1 (update variable name of Babel output) in #962. Seems to work with the repro from @LarsDenBakker: https://github.com/LarsDenBakker/polymer-cli-tagged-template.

tony19 on 19 Feb 2018

🎉4

All 13 comments

The above example is a minimal reproduction of the issue.

The actual issue arises when a template is evaluated later than the script was run. For example I often use functions that dynamically returns a template:

(function (NAMESPACE) {
    NAMESPACE.templateFactory = (template) => window.Polymer.html`<div>${template}</div>`;
})(window.NAMESPACE = window.NAMESPACE || {});

This transpiles to:


var _templateObject = babelHelpers.taggedTemplateLiteral(["<div>", "</div>"], ["<div>", "</div>"]);
(function (a) {
    a.templateFactory = function (a) { return window.Polymer.html(_templateObject, a) }
})(window.NAMESPACE = window.NAMESPACE || {});

By the time the templateFactory() is called, the _templateObject it references is a global variable which by then is overwritten by another script.

LarsDenBakker on 18 Jan 2018

I've created a repository that demonstrates the issue: https://github.com/LarsDenBakker/polymer-cli-tagged-template

Observe the difference in output in the browser when running the unbuilt code, the es6 bundled code and the es5 bundled code. Unbuilt and es6 bundled are ok, es5 bundled is not.

The same issue is explained here: https://stackoverflow.com/questions/37858707/babeljs-transpile-es2015-to-es5-template-literal-tag-function-issues

If the modules option on this line: https://github.com/Polymer/polymer-cli/blob/master/src/build/optimize-streams.ts#L26 is changed from false to 'umd', each file/script tag is wrapped in an iife which makes things work again. However, because the polymer bundler preserves the original script tags this will result in a lot of extra boilerplate.

LarsDenBakker on 20 Jan 2018

Now that html tag template literal got released in 2.4, more poeple will start using this and run into this issue which is not at all clear to debug :(

LarsDenBakker on 31 Jan 2018

👍1

The nature of this bug is in the nature of Babel which assumes that you will bundle you code into modules after transpiling. Though for some reason it is not mentioned here https://github.com/babel/babel/blob/master/doc/design/compiler-assumptions.md

I did some research about the ways to solve this:

Changing bundler logic to rename all _templateObject vars making the name unique across all concatenated files
➕ can be changed on Polymer CLI side
➕ aligned with Babel assumption about what bundler is supposed to do after transpiling
➖ tightly coupled to Babel implementation details
Wrapping each HTML import's script in IIFE after transpiling (this plugin can be used https://www.npmjs.com/package/babel-plugin-iife-wrap)
➕ can be changed on Polymer CLI side
➕ modularizes code even more (may be helpful during transition period to ES modules)
➖ changes the convention about HTML Imports code which is executed by default in the global scope
➖ backwards incompatible
Create _templateObject in the same scope as the template literal itself or just call a helper directly (my draft implementation is here https://github.com/babel/babel/compare/6.x...bashmish:template-literals-in-html-imports?expand=1)
➖ very unlikely to be accepted by Babel because it means some benefits for a deprecated HTML Imports standard at the account of all the rest use-cases
➕ but can be fixed in the fork of relevant babel plugins as Polymer CLI does not expose it's Babel configuration to the end-user
Similar to 1: proposing unique name generation to Babel based on UUID algorithm, so that names are uniqeu regardless of how Babel is run
➖ very unlikely to be accepted by Babel because it means some benefits for a deprecated HTML Imports standard at the account of all the rest use-cases

My personal favourite is number 3. The fix for upcoming Babel 7 is even simpler. Though number 2 may be much easier and more useful in the long run despite the fact that it breaks the convention about HTML Imports.

I wonder what you think?

bashmish on 6 Feb 2018

👍1

I think I need to dive deeper into the ES spec as it was recently changed and Babel here kinda follows the spec rather than just trying to optimise it's own overhead. Some useful information regarding this caching strategy is mention here and is all of a sudden related to lit-html https://github.com/Microsoft/TypeScript/wiki/What's-new-in-TypeScript#cache-tagged-template-objects-in-modules

bashmish on 8 Feb 2018

Now that html tag template literal got released in 2.4, more poeple will start using this and run into this issue which is not at all clear to debug :(

Hi :wave: that's me. Nothing like spending a half of your Saturday thinking you'd missed a angle bracket or comma somewhere, and read through half a dozen unrelated components before getting down to the real issue here. 🤦‍♂️

If I understand @bashmish's research on possible solutions, I'd be a big fan of Option 3b, calling the helper directly, though agree that getting Babel support for that seems like an up hill battle. After that, I like Option 1 for it's simplicity, we're already a bit attached to the Babel implementation details as it is, and lazily prevents from having to create new documentation to prevent possible issues around leaving components and mixins in the global scope.

Westbrook on 18 Feb 2018

I implemented Option 1 (update variable name of Babel output) in #962. Seems to work with the repro from @LarsDenBakker: https://github.com/LarsDenBakker/polymer-cli-tagged-template.

tony19 on 19 Feb 2018

🎉4

Babel has indicated that they are expecting a scope per file, and are not going to do anything to fix this. https://github.com/babel/babel/issues/7321#issuecomment-370316245

They propose a workaround: https://github.com/TrySound/babel-plugin-iife-wrap but I think @tony19's merge request less intrusive.

LarsDenBakker on 5 Mar 2018

This was mentioned as Option 2 in one of my previous commits. After a few round of investigations, I also believe that this would be the most reliable solution. And the main reason for that is that assumption of babel. In the long run we can run into other issues with babel, not only with tagged template literals, and only way to eliminate this is to meet babel's requirements to our code.

bashmish on 5 Mar 2018

@justinfagnani @aomarks This issue was fixed in a PR, but never got released. Is it possible to get a release of this before the Polymer 3 build tooling release?

LarsDenBakker on 22 Apr 2018

@justinfagnani @aomarks This issue was fixed in a PR, but never got released. Is it possible to get a release of this before the Polymer 3 build tooling release?

We are regularly publishing -pre releases: you can get it with npm install polymer-cli@next. This version is backwards compatible with today's @latest, although you can expect some instability as it has a lot of active development. Does that work for you until the final next version is published?

aomarks on 24 Apr 2018

Unfortunately I cant, its part of our company's build pipeline. Getting an unstable version in there wont be possible.

LarsDenBakker on 24 Apr 2018

I am having major problems with a tagged string literal templater function which uses string manipulation... i had to convert a websites javascript into a webpack compitable structure with imports/exports on a multi page website.

And ive now spend over 72hours of my own time, trying to figure out what the hell is going on.

Apparently webpack bundles tagged templates wrong. if you have a tagged template function and you call it from within an object etc myObj = {test: templater(string literal ticks start)my template function ${'manipulation'}(string literal ticks end) and then call the myObj.test() from a packed webpack module... it wount recognize it as a function.