Lit-element: Static build for CDN distribution

[slightlyoff]

Per a F2F conversation with @justinfagnani, wanted to capture agreement to add a single, modern-browser-exclusive build of LitElement and the most frequently used lit-html directives to the distributed NPM module. The goal is to enable easy inclusion of LitElement from a single source file.

Rationale

LitElement, when built, is tiny! That isn't apparent when using it from, e.g., unpkg. The need for the Polymer CLI tools also increases the friction to getting started with the edit/compile cycle that historically makes web development so much fun.

Instead of trying to boil the whole ocean by providing every build variant, or having to compromise on size to accomodate legacy, the idea here is to get developers started quickly while providing an easier path to "real" LitElement usage via local builds.

Part of this approach might need to add guards to the build to give up when loaded in the presence of existing LitElement (to prevent over-inclusion) and console logging to flag that this isn't the most efficient approach.

[justinfagnani]

The idea we discussed was to build a fairly reasonable set of bundles for LitElement and lit-html, and publish them to npm under dist/ folders. The bundles would be minified ES2017 modules. LitElement would include UpdateingElement, but not lit-html - it would reference the lit-html bundle. The lit-html bundle would include everything reachable from lit-html.js, but not directives. They would be minified separately.

[jrobinson01]

thoughts on using https://github.com/pikapkg/web instead of a CDN?

[justinfagnani]

To clarify, this isn't for local installations via the npm registry, and we won't be changing the main and module fields of package.json. For most users nothing will change at all.

This is simply a way for us to distribute fairly well optimized bundles to CDNs that host npm files so that people can use them easily. We're a bit unusual in that we only publish plain ES2017 modules. Most packages right now still publish minified bundles in a variety of module formats. Our current approach is more optimal for local development, but gives a bad impression for people directly loading files from a npm CDN.

dist/ might be a bad folder name - prebuilt/ may be better. We'll put a README in the folder saying not to use these builds for production (unless you have a npm CDN that's intended for production use) and probably add a console warning too.

[slightlyoff]

Is this in progress now?

[e111077]

Regarding pikapkg, the CDN launched today which serves a minified bundle at https://cdn.pika.dev/lit-element

[justinfagnani]

Ugh... that looks like it bundles lit-html, so if you import lit-html separately to use directives, it won't work.

[chase-moskal]

@e111077

Regarding pikapkg, the CDN launched today which serves a minified bundle at https://cdn.pika.dev/lit-element

that pika cdn bundling feature looks super cool!

@justinfagnani

Ugh... that looks like it bundles lit-html

i think i'm missing something here — i thought the pika bundle is exactly what people have been asking for? please bear with me as i think aloud here, i'll try to express why this is confusing to me

what's the point in providing a lit-element bundle when it doesn't include lit-html in the bundle?

• isn't the point of the bundle to make lit-element "grab-and-go" ready from unpkg or similar — without requiring the user to worry about the bare specifier resolution step (a bundler or es-module-shims)?
• if the lit-element bundle references lit-html, doesn't that require bare specifier resolution? and then doesn't that defeat the purpose of providing a bundle?
• i worry there's a circular thing going on? we want a bundle so we can avoid bare specifier resolution, right? but we also want the bundle to require bare specifier resolution for lit-html?

so if you import lit-html separately to use directives, it won't work.

also, as soon as there is talk about importing lit-html separately, like in a peer dependency situation — i think that invalidates the convenience bundle use-case — because the user is now responsible to mediate how lit-html is shared in the application amongst multiple libraries, the user should be controlling bare specifier resolution with their own bundler, or with import maps; they have now outgrown the capabilities of a convenience bundle

maybe my assumptions are wrong here — maybe avoidance of the bare specifier step is not considered important for lit-element's ease-of-use in this case? so users are still expected to implement bare specifier resolution themselves via bundling or es-module-shims in order to get started using these lit-element bundles? perhaps then it's really about the minification and consolidation of http-requests.. however if they're forced to using a bundler to use these bundles, isn't that minification and consolidation redundant, and already in the user's wheelhouse?

do the bundles reference each other with bare specifiers that need to be resolved? are the bundles intended to be imported directly (grab-and-go, no bare specifiers), or with a resolution step?

i was expecting the goal would have been a bundle that looks just like the one that pika provides — however i understand there's a peer dependency problem with the way pika bundles those sub dependencies — but isn't the correct solution for that, to move on from the convenience bundle and use the es modules and bare specifier resolution by configuring import maps or your own bundler?

perhaps pika cdn covers the "avoiding bare specifier resolution" use-case?

and maybe the lit-element bundles proposed in this thread are for a different use-case? i'd like to understand what this use-case is all about

ultimately, in an ideal world, it makes sense to me that packages like lit-element would continue to only expose es modules, and all of these bundling, minification, and optimization concerns, would be left to dedicated tools like bundlers and cdn's like this new pika thing

please inform me! cheers friends! :wave:

[jfbrennan]

Came back to LitElement hoping this was possible now. Darn.
This is what you can do with Hyper that I wish I could do with Lit:

<body>
  <my-foo></my-foo>

  <script src="https://cdn.jsdelivr.net/npm/[email protected]/min.js"></script>
  <script>
    // A better Custom Element experience using Hyper
    class MyFoo extends HyperHTMLElement {
      created() { 
        this.render(); 
      }

      render() {
        return this.html`<div>You're doing it Peter!!!!</div>`;
      }
    }

    MyFoo.define('my-foo');
  </script>
</body>

If I really need to I can also go all the way and set up and configure a build pipeline to use an installed/imported version of Hyper, but it's not required. I love this "old-school" way...other modern libs like Moment, Vue, Riot, and more offer this option so anybody can start trying the lib in like less than 60 seconds. It's awesome!

[justinfagnani]

@jfbrennan
Note that you can do the exact same thing with LitElement and unpkg:

<body>
  <my-foo></my-foo>
  <script type="module">
    import {LitElement, html} from 'https://unpkg.com/lit-element?module';

    class MyFoo extends LitElement {

      render() {
        return html`<div>You're doing it Peter!!!!</div>`;
      }
    }
    customElements.define('my-foo', MyFoo);
  </script>
</body>

[antonioaltamura]

Hi, can I know the reason behind the choice of imposing a transformation step (with Polymer CLI) just to import LitElement? I'm genuinely curious.

[chase-moskal]

@antonioaltamura -- you don't need a transformation step for lit-element -- that's what's great about tagged-template literals instead of jsx

justin's post above demonstrates usage of lit-element in plain html without any build step :+1:

[antonioaltamura]

Nop, a remote request is not an option for me. Can I do the same with the local LitElement package?

[slightlyoff]

Pretty bummed this isn't sorted yet. Full-rollup packages for lit-html and lit-element seem like they'd be pretty straightforward from the build perspective. Yes, larger, but is that so problematic?

[chase-moskal]

@antonioaltamura

Nop, a remote request is not an option for me. Can I do the same with the local LitElement package?

you certainly can serve from your local node_modules -- maybe use es-module-shims and supply an import map that points to packages in your node_modules directory :)

👋 chase

[chase-moskal]

@slightlyoff

Pretty bummed this isn't sorted yet. Full-rollup packages for lit-html and lit-element seem like they'd be pretty straightforward from the build perspective. Yes, larger, but is that so problematic?

yes, distributing bundles in modern esm packages is problematic -- here's why

1. first, your bundle is already right here!

   • https://cdn.pika.dev/lit-element/^2.2.1
   • import {LitElement} from "https://cdn.pika.dev/lit-element/^2.2.1"
   • pika automatically generates bundles of every modern esm npm package
   • anybody looking for a bundle should be encouraged to use this one (or run rollup themselves)

2. so what's the harm in libraries distributing bundles to users?

   • it sets a bad example for modern library design: many users of a bundle simply won't understand when they are creating a peer-dependency nightmare which will lead to people loading many redundant copies of the same code
     
     
     
     • when you use a few libraries which have peer dependencies, and you use them in bundle form, you'll end up loading those peer dependencies redundantly -- now imagine loading ten lit component bundles -- you're now loading ten times more lit than you bargained for
     
     
     • lit-element is currently a clean and well-formed example of an ideal modern esm package following the emerging best practices, and it's hard to use incorrectly -- i'm hoping it can stay that way so i can keep using it as an example of how to structure a good library
     
     
     • tldr; applications should be bundling/minifying/optimizing/etc -- whereas libraries should expose only es modules
     
     
   • and hey, what's wrong with those handy pika bundles, anyways?
     
     
     
     • pika bundles are all-inclusive and convenient for instant experimentation; no build, no npm, no complexity -- just a url and a webpage -- i think it's what your looking for :)
     
     
     • plus, just imagine: if we make the bundle easily accessible, people will likely bundle the bundle into their bundle -- ridiculous! -- since they're already bundling, they ought to use the es modules -- and the other workflow to consider is simple webnative imports, and that's why lit-element exposes es modules
     
     
     • so i see lit-element currently satisfies three use cases:
       
       
       
       
       
       1. convenient experimentation (pika bundle!)
       
       
       
       2. app dev who bundles (es modules in rollup!)
       
       
       
       3. ripped and sweet app dev who uses es-module-shims (es modules!)
       
       
       
       4. is there another use-case we should consider?
       
       
       
     
     

one last thought: yeah, some people think that they want a bundle, and some people think they want common-js; i'll even bet somebody thinks they want window globals too -- so let's just leave those people behind! -- after all, they've got pika and other solutions to handle their confusion and/or legacy needs

👋 chase

[stevenpeh]

pika/unpkg and es-module-shims are definitely workable workarounds. But that's all they are when you don't/can't use a bundle for whatever reason.

You'd think a framework should at the very least support a simple use case where folks can use it out of the box without a bundler or extra workarounds!

For me in a corporate environment, we cannot even hit the external URLs for pika and unpkg. I can experiment find with it in our POC environment where we have access. I absolutely do not want to use a bundle just for this because we don't use it today for other web apps we build internally - yes we've managed to avoid an extra level of complexity - K.I.S.S.

But this framework... just to even get started I have to bite the bundler bullet or do the workaround dance, pretty silly.

[web-padawan]

@stevenpeh please note that you can now use Snowpack: https://www.snowpack.dev/#litelement

It's solving the problem of need for a bundler by processing files on install via Rollup.
Actually it's the same as https://github.com/Polymer/lit-element/issues/603#issuecomment-470659306 (previously known as @pika/web)

[e111077]

Glad to see that Snowpack provides a solution, but unfortunately, it doesn't seem to solve the use case of the initial issue as it still requires you to set up an npm / yarn environment and install. This is a weird problem as it will have to require the CDN to know what packages are being used and resolve the dep tree as there can only be one version of lit-html on the page for directives not to break.

We did something similar back in the day for webcomponents.org where you essentially provide it with a package.json (or bower) and it'll create a static link for the bundles. Though I don't believe it had the niceties of minification etc.

Edit: I'm dumb and missed the comment above Serhii's

[stevenpeh]

@stevenpeh please note that you can now use Snowpack: https://www.snowpack.dev/#litelement

It's solving the problem of need for a bundler by processing files on install via Rollup.
Actually it's the same as #603 (comment) (previously known as @pika/web)

Thanks, I'll give it a try. The previous pika example was a bit confusing, it seem more like a CDN reference similar to unpkg rather than providing a localized, remote dependency free and bundler free option. I'll test out snowpack.

That said. The official docs should provide a recommended way to do this, even if it means using external tools like snowpack. A good framework should always provide a means of it with as little dependency as possible. In this case the minimal dependency would be a one off initial setup time dependency, better than runtime dependency (CDNs) or per build dependency (bundlers).

Ultimately, the roadmap should aim at "none". None as in no extra tooling setup - the framework is self-contained with whatever dependency it needs in a localized manner.

[stevenpeh]

Ok I can get this working using snowpack to do a one off processing of my project folder to convert node_modules to web_modules. However as mentioned ideally the framework should provide a way to consume without all these workaround where a bundler like Webpack isn't use.

In fact I looked around and found the another WC framework, Slim.js, allows just that. In fact they provide 2 ways to do it without external tools like snowpack if I don't want to use bundlers. They provide an explicit "no_modules" version I can import as plain script imports or if I want to use es6 modules can I also import they regular js as script modules explicitly in my index.html

[LarsDenBakker]

How do you see this working for components which are published and reusable? How should they declare dependency on lit-element or lit-html? Should they also ship a build that can be loaded with a script tag, duplicating lit-html + lit-element for each component?

You could somewhat solve this for just lit-html or lit-element, but as an ecosystem of cross-dependent packages this isn't possible. Es modules are implemented in all browsers, they are the best way to share and ship code. Because the node_modules folder is in a different directory in your repository and when installed in another project, you can't set explicit paths to the module folder for importing dependencies. So you need to use something like bare imports for now.

[gmurray81]

Yeah, we have a scenario where one of our packages has a peer dependency on lit-html. This is fine for when people are using a modern build toolchain with our package. But we also provide a UMD bundle for the package, and because lit-html provides no UMD, bundle option, this provides no end of troubles. Is the only solution to ingest lit-html into the umd bundle?? Publish an alternate lit-html that does provide a UMD bundle?

[justinfagnani]

@gmurray81 If you provide a non-module bundle, you should probably include everything needed in the bundle. lit-html doesn't seem right for a peer dependency either.

[gmurray81]

Wouldn't issues arise if there were multiple nested distinct versions of lit-html?

[Alex-PK]

Hello, chiming in with my use case. Hope it helps.

I am not using npm or node locally, so I can't use webpack or any other js bundling system. I am actually developing a (bunch of) Go website, and wanted to serve some web components to be used on the pages, without having a full SPA.

Using import {html, LitElement} from 'https://unpkg.com/lit-element?module' fires off 22 HTTP requests, for a total of 69 Kb compressed (163 uncompressed) and takes 1.29 seconds.

Using import {html, LitElement} from 'https://cdn.pika.dev/lit-element' fires off only 4 requests, but one of them is a polyfill.js (89 Kb compressed, 441 uncompressed) that doesn't appear to be needed. The total is 123 Kb compressed and 540 uncompressed, for a total of 0.92 seconds.

None of them seems ideal. Having a single 70 Kb bundle to download would probably take less than 200 ms.

[IanBellomy]

Related:
The unpkg urls are great for deployment (if you aren't trying to cache them with a service worker) but tsc / VSCode doesn't do typing for remote imports which makes _doing work_ difficult with unpkg in place.

See: https://github.com/microsoft/TypeScript/issues/29854

Edit:
If installing locally is ok, it's not so bad to pass through the typings, i.e.
declare module "https://unpkg.com/lit-element?module"{ export * from "lit-element" } // etc.
Per https://github.com/Microsoft/TypeScript/issues/28985#issuecomment-472743014

[chase-moskal]

i made this simple <demo-counter> codepen demo featuring import maps via es-module-shims (~50 lines)

and yes, a bundle would load faster than using import maps, however, keep in mind: bundles are properly optimized at the application-level, not at the level of libraries like lit-element. if you want refined performance, your application should run a bundler like rollup, webpack, or snowpack as a part of the build routine

[webfolderio]

I built lit-element bundled with lit-html. It doesn't require node and supports both iife and ES module. It's suitable for non-node js environment usage.

[Vincent-Carrier]

@webfolderio Those are pretty large bundles (100KB plus)... Not very practical for production.

Gonna chip in with my use case: I'm building a blog with Hugo and I'm using lit-element to embed dynamic content in my markdown. Right now I'm using Rollup to bundle everything but it honestly feels very overkill for what is essentially a static website. It adds another build step and I have to run the Rollup watcher in another terminal tab, set up different bundles for each page, etc...

What I'd like to see is a small, minified ESM distribution I can download myself (could be a zip file for all I care) and use without any special tooling. Not a single 50KB+ file, nor a thousand little 100B files, but something in between, where files can be loaded as needed.

Keep up the good work!

[slightlyoff]

Just noting that I hit this again and it's still not fixed.

@justinfagnani: please advise.

[chase-moskal]

@justinfagnani, @slightlyoff, @Vincent-Carrier

what do you think about adding a new readme section about this? here's my attempt, and i think these two strategies should practically cover most use-cases, what do you think?

### Bundles, minification, and other optimizations
- The lit-element package keeps it simple by shipping plain es modules
- **For convenience and experimentation,** we recommend loading the lit-element 
    bundle straight from the [Skypack CDN](https://www.skypack.dev/)
    ```js
    import {LitElement} from "https://cdn.skypack.dev/lit-element"
    ```
    alternatively, see [es-module-shims](https://github.com/guybedford/es-module-shims) 
    to get started using in-browser [import maps](https://github.com/WICG/import-maps)
- **For production applications,** we recommend using a bundler 
    like [Rollup](https://rollupjs.org/guide/en/) or [Webpack](https://webpack.js.org/) 
    to optimize your whole application, because such optimizations are best handled 
    at the application-level rather than per-library
    - use bare-specifier imports in your code, like `import {LitElement} from "lit-element"`
    - your bundler will resolve "lit-element" as configured by your package.json or import map
    - if you're writing a library, place lit-element in your package.json `peerDependencies`
    - this strategy avoids easy-to-miss peer dependency problems, which could 
      otherwise cause duplicate code being loaded
    - this strategy also allows your bundler to leverage tree shaking to avoid unused code

if this is sufficient and clear, maybe this issue could be closed

my biggest concern with directly shipping a bundle, is that hordes of well-intentioned web component library authors will load the bundle instead of the modules, accidentally forcing all downstream applications to redundantly load duplicate copies of lit-element/lit-html

optimizations must be done at the application-level, where the bundler really has all the necessary information — whereas library-level optimizations are fundamentally premature, incomplete, and error-prone in terms of causing wasteful bloat — libraries can avoid all this and even keep it simple by cleanly providing modules

  :wave: chase

[gmurray81]

@chase-moskal why not have the pre-bundled version emit a warning on the console that this may not be the appropriate configuration for a production application?

The issue with the skypack approach is that it can't work in pre-esm browsers, and there are scenarios where you may want to use lit-element/lit-html in pre-esm browsers without having to involve transpilation/bundling tooling.

[gmurray81]

On a side node, conventions are a useful shortcut to avoid having to consider all possible creative avenues for use. Its conventional to include older style bundles in an npm package, and while you may be able to rationalize the need for these away, you are actually cutting off avenues for use that are a bit outside the golden path. Following the conventions, while seemingly onerous, enables more flexible use, and avoids suprising consumers of the package expecting more conventional outputs and layout.

[justinfagnani]

This remains a tricky and pretty over-constrained issue. The reason it isn't solved yet is because every "solution" we know of violates one of the constraints, and it's not so easy to pick from the options. I still believe that the format we publish is the least opinionated and least hazardous (without some breaking changes that we are in fact doing in 2.0, but I'll get to that...)

Here are some of our main goals and constraints we consider when publishing the library:

1. Pay-as-you-go: Don't load code that's not used by an application or component. And accomplish that without a build step.
2. Avoid module duplication: Don't allow (or encourage) multiple dependents to depend on different builds. Also, accomplish this without tools.
3. Don't unnecessarily transpile -> Use faster/smaller code in modern browsers
4. Minimal-to-no build steps required, yet
5. Works with common tools and build systems

What we've done to meet those are:

• Publish multiple entrypoints, such as lit-html/directives/cache.js to be pay-as-you-go
• Import lit-html via bare module specifiers. This was the hardest decision to make, because there's a tension between (4) and (5) but it came down to the fact that complete solutions to (4) would make fragile assumptions about package managers and servers and would break tools that assume Node module resolution. So we used bare-specifiers only from cross-package imports, built dev servers to perform Node module resolution (and encouraged other to do the same), and helped with the Import Maps specification.
• Publish modern (for the time) ES2017, and only ES2017. It's difficult to impossible to uncompile ES5 to modern, so we have to at least publish modern JS. But we also want to avoid module duplication from some code importing other builds, so we can only publish ES2017. Other true frameworks that encourage publishing multiple builds then have to deal with detecting and removing duplication with tools, and that's a mess we didn't want to get into.
• Publish only standard JS modules. No UMD or CommonJS. This is because of the duplication issue again.
• Don't do any bundling or minification. We see that as an application concern and one that can be done better by tools with a whole-program view.

This was a very unusual way to publish a JS package at the time, but I'll note that it's the simplest solution and many more packages are following suit now.

Now, there are some other concerns that aren't fully addressed by this approach:

• Apps that support IE11 and other older browsers need to compile our code. Many build systems are not set up to compile packages in node_modules/ by default. This hasn't been too big of a blocked for most people as they've been able to modify their config, or don't support IE11.
• Our unbuilt file-size on the wire isn't what we report on our site as the minified-gzipped size of the core library. This is one of @slightlyoff's main concerns as I understand it. The concern is that people who check on an unbuilt demo will think that lit-element is much bigger than it will be in production. I've definitely seen this a couple of times, but it's mostly a marketing concern as we see very few attempts to push the unminified source straight to production. Almost all sites are built through a bundler that does the right thing. Those who are trying to go bundler-free are usually savvy enough to still optimize files if they want to.
• CDN usage can be slow if the CDN doesn't build files. This is true with simpler CDNs like unpkg.com, but Pika/Skypack/Snowpack and jspm are optimizing files now.
• We still aren't completely tool-free because of bare-module specifiers. Developers still need at least a dev server that rewrites import specifiers. This was the tradeoff we made at the time, and hopefully as Chrome ships Import Maps this will begin to change and enable truly buildless dev again.

So I get that those issues are still a concern for some people. Solutions like adding a pre-build and bundled version of lit-element to the package have not reached clear consensus on the team because of the duplication hazzard.

However, things have changed in the ecosystem since lit-element 1.0 which lets us address those issues a bit more completely:

• Node and many tools now support package exports with conditions. So we can publish minified sources for production and unminified sources for development. It seems like the condition name "development" is becoming a defacto standard, so tools can have a single flag to switch their mode across all packages. We're doing this in lit-element 3.0 and the minified files are highly tuned to save every last byte given our specific code and conventions. Since we now have a built version, we're also including more dev-mode-only code (like extra warnings) that is eliminated from the prod build. We're also testing both prod and dev builds against the whole test suite to make sure nothing's broken.

This makes lit-element and lit-html's wire numbers basically the same for demos and in apps with their own build system

• Package exports also let us control the entrypoints to the package, so even for the unbuild dev-mode we can hide implementation modules from importers. Because of this we can bundle the entry points. We still have multiple entrypoints for directives, decorators, and other helpers, but the main library is one file in the prod version now.
• Bundlephobia, Skypack and other similar tools are much better about reporting realistic module sizes, so we can also back up our marketing numbers with third-party verification.
• More and more packages are publishing modern JS and dropping IE11 support.
• Import Maps are about to ship in Chrome

So I think lit-element 3.0 will be in much better shape relative to the remaining concerns. We may still yet want to add some sort of "classic" build for some use cases, but we still need to evaluate that wrt module duplication hazzards. One possibility is that package exports aren't enforced by older tools or tool-less workflows, so we can publish a build, but not include it in the package exports, and most build systems will reject that import.

We're trying to get lit-element 3.0 and lit-html 3.0 released very early this year, so hopefully we'll be able to close the issue then. I'll ping this issue on the next pre-release when we have a few more docs to see if we can get some feedback on how well it addresses things.