Parcel: 🙋 Feature request (?) - Code splitting with dynamically resolved path

[benrobinson]

This is a feature request unless I missed something. I've been poking around and found some similar issues (https://github.com/parcel-bundler/parcel/issues/905), but nothing that seems to address this directly -- apologies if I missed it.

My guess is that this currently isn't supported, but here's what I'm trying to do...

Given modules with an interface like:

class Module {
  render = () => { /* ... renders something or whatever */ }
}

I want to resolve the specific module that will be imported at runtime following some kind of event in the browser:

function renderSomeModule(moduleName) {
  import(`./modules/${moduleName}`)
    .then(module => module.render())
}

Currently, the above example fails, but regular dynamic imports work fine like this (with the name/path already resolved in import):

function renderSpecificModule() {
  import(`./modules/SpecificModule`)
    .then(module => module.render())
}

I've read the code splitting doc, which provides an example of how to resolve imports ad hoc with a map of imports that might be resolved async, but ideally I don't want to write a big list of every possible module that I might be loading because I have a fairly large collection of potential modules that could be loaded. In other words, I don't want to do what's suggested here https://parceljs.org/code_splitting.html, like:

// Setup a map of page names to dynamic imports.
// These are not loaded until they are used.
const pages = {
  about: import('./pages/about'),
  blog: import('./pages/blog')
};

// ^^^ Ideally I don't want to do this part manually, because it would be a big list to maintain

async function renderPage(name) {
  // Lazily load the requested page.
  const page = await pages[name];
  return page.render();
}

Instead, I would want Parcel to be able to figure out which files are possible "options" to be loaded based on the directory path + variable wildcard I have provided in import(./path/${actualModuleWildcard}) or import(./${pathWildcard}/${moduleWildcard})

This is possible in webpack, but with some magic that I believe probably generates code equivalent to the code splitting example above:

// Here's how doing it in webpack looks:

function renderSomeModule(moduleName) {
  import(/* webpackChunkName: `[name]` */ `../modules/${moduleName}`)
    .then(module => module.render())
}

Like I say, my guess is that this is basically not something that is supported by Parcel (yet), but I was just wondering if I was overlooking some obvious pattern or configuration that would make this work at the moment.

🎛 Configuration (.babelrc, package.json, cli command)

It's not really a bug, because regular code splitting as described in docs works fine and I've found no docs to indicate that what I want should work, but here's my .babelrc configuration, which is really the only configuration I have:

{
  "plugins": [
    "transform-class-properties",
    "transform-runtime",
    "syntax-dynamic-import"
  ],
  "presets": [ "env", "stage-0", "react"]
}

🤔 Expected Behavior

Ideally, it'd be great if imports like this:

import(./path/`${someModule}`)

Would work out of the box in Parcel, with the actual implementation of resolving wildcard imports happening behind the scenes (and without the use of magic configuration comments a la Webpack.)

😯 Current Behavior

Currently, if attempting to load a module this way, I get an error in the browser at runtime:

Whereas if the module path is fully present in import() it resolves the module in the browser just fine.

💁 Possible Solution

I'd imagine that a possible solution would involve checking the path specified, and collecting + bundling any legitimate module import possibilities that match that path.

A drawback might be a longer build time, I'd think.

🔦 Context

Because this currently works in Webpack, but I like Parcel's overall approach better, it's one of the main things preventing me from switching over.

🌍 Your Environment

| Software | Version(s) |
| ---------------- | ---------- |
| Parcel | 1.6.2 |
| Node | v8.9.0 |
| npm/Yarn | npm version 5.5.1 |
| Operating System | macOS High Sierra 10.13.3 |

[davidnagli]

Ok so I'm a bit confused on whether or not you care about code splitting.

In the beginning it seemed like you where exclusively talking about code splitting, but then towards the end you kind of just made it seem like you just need some general way to do wildcard imports, not necessarily code splitting.

---

In the case that it's the latter (you DON'T need code splitting), the answer is simple: We support wildcard globs out of the box! 🚀

You should be able to just do:

import stuff from './path/*'

and that should import everything under the path directory into an object called stuff.

You might wanna take a look at #125 (particularly, Devon's comment)

---

In the case it was the former, and you were actually DO need code splitting, the answer is a bit trickier. In order to split the code, you need to actually know which code needs to be split into each bundle during build-time, which is impossible if its determined 100% dynamically during runtime.

Even if you DONT use dynamically generated template literals in your imports and choose to use globs, it'll already be more complicated and might now work as expected... at least according to Devon in his comment on #125.

I hope that made some sense 😇 😂

[benrobinson]

Hey, thanks for getting back to me so quickly -- I'd missed the glob support, and the discussion in #125 is definitely useful. It seems like I may be able to get it to do what I want, but I'll have to play around and see.

Just to clarify, I do want code splitting... What I'm doing is mounting React components ad-hoc, so given a src directory with an unknown number of valid component files, I only want to load the component files relevant to a given page rather than loading a big aggregate bundle of all of the components when only a few of them are used. I want to load only a small entry point file (index.js) in the browser to begin with, which will detect the mount locations of necessary components and give the instruction to load only the necessary corresponding bundles.

So given a src folder like this:

src/
  index.js
  components/
    A.jsx
    B.jsx
    C.jsx

The entry point file is what gets loaded by the browser, and it has something like this:

const renderComponents = () => []
  .slice
  .call(document.getElementsByClassName('js-component'))
  .map(renderComponent);

function renderComponent(component) {
  const componentName = component.getAttribute('data-component-name');
  import(`.components/${componentName}`) // will be 'A', 'B', or 'C', or it'll fail with the below error.
    .then(c => { /* render the component */})
    .catch(err => {
      console.err(err); 
      console.err(`Failed to load ${componentName}`);
    });
}

window.addEventListener('load', renderComponents);

So at bundling, I want bundle files something like:

dist/
  indexBundle.js
  Abundle.js
  Bbundle.js
  Cbundle.s

If the only component mount location found on the page is for component 'A', then I would want only these two bundles to be loaded by the browser:

dist/indexBundle.js
dist/Abundle.js

And if I had a more complex structure like this (which I don't currently, this is just as an example):

src/
  index.js
  components/
    foo/
      A.jsx
      B.jsx
    bar/
      A.jsx
      B.jsx

Ideally I want to be able to do something like this:

import(`components/${type}/${componentName}`)

And have it load only something like this:

dist/indexBundle.js
dist/foo/ABundle.js
dist/bar/ABundle.js
dist/bar/BBundle.js

Like I say, I currently do this with webpack, and my belief is that they must do glob load up front during bundling, but resolve individual files in the glob to their own bundles in dist output (mapping the src name to the dist name), so loading them separately at runtime is possible.

Based on what you sent, it seems like I may be able to find a way to do this same thing myself, so I'll mess around and see what I can come up with. Will let you know what I figure out. Thanks again!

[benrobinson]

This comment by chee represents what I'd want in terms of output:

In my case, I actually want Parcel to go ahead and bundle every single valid module file in the specified path, I just want it to bundle each file separately, not as one big glob bundle. If I request a module at runtime, its bundle gets loaded. If I don't request it, Parcel has still built it, it just doesn't get loaded at runtime.

To paraphrase chee's comment but differentiate, I'm fine with Parcel bundling every matching file because that might just mean a longer build time, I just want to avoid loading every bundled file if not every file is needed. Feel free to let me know if I'm overlooking something that makes this a crazy request...

[benrobinson]

In the case it was the former, and you were actually DO need code splitting, the answer is a bit trickier. In order to split the code, you need to actually know which code needs to be split into each bundle during build-time, which is impossible if its determined 100% dynamically during runtime.

I just wanted to clarify further... in my hypothetical scenario, which module bundle is actually loaded would be 100% dynamic at runtime, but which module bundles need to be built would be determined by what valid module files were in the path at build time. If there were 20 src module files matching src/components/*.jsx, I would want there to be 20 dist module bundle files matching dist/components/*bundle.js. Please let me know if anything is still unclear!

[benrobinson]

OK, so I figured out one way I can basically do what I want, without using globs...

Given a directory structure like:

src/
  index.js
  makeImports.js
  modules/
    a.js
    b.js

makeImports.js would be a node utility file that does the meta-step of writing the file that defines any dynamic module imports that I want to make available based on what's in the filesystem:

// makeImports.js
const path = require('path');
const fs = require('fs');

const rootPath = path.resolve(__dirname);

// Load all JS files from a specific path, and just assume they're modules -- could be more done to verify they are in fact valid modules.
const dynamicImportFiles = fs
  .readdirSync(`${rootPath}/modules`)
  .filter(moduleName => /(\.jsx|\.js)/g.test(moduleName))
  .map(moduleName => {
    mn = moduleName.replace(/(\.jsx|\.js)/g, '');
    return `${mn}: import('./modules/${mn}')`
  })
  .join(', ');

// Write an imports.js file that contains the individual dynamic import()s
fs.writeFileSync(`${rootPath}/imports.js`, `
  const imports = {${dynamicImportFiles}};
  export default imports;
`);

The output file from the above would look like:

// imports.js -- generated by makeImports.js
const imports = {a: import('./modules/a'), b: import('./modules/b')};
export default imports;

And index.js would then import the chunked imports from imports.js:

// index.js
import imports from './imports';

const renderComponent = (component) => 
  imports[component]
    .then(c => { /* render the component */});

And I'd set up my package.json with a script command that would just ensure that makeImports was always run before parcel:

// package.json
{
  ...
  "scripts": {
    "parcel-run": "node src/makeImports.js && parcel src/index.html"
  },
  ....
}

This basically works for my purposes at the moment, although it could certainly be improved in terms of generality in the makeImports.js file.

Really, this might be something that's a better fit implemented this way than as part of Parcel since it may be that anybody trying to do something like this will need to just come up with their own logic for how to determine what needs to be in their imports, but with some more thought about how to generalize, maybe it could be a cool feature.

Anyway, thanks -- Parcel has been great to work with so far!

[shunia]

Is there any way to be sure in the building process that parcel knows exactly what the dynamic value is?
Or do import or require understands dynamic value in the path in any version of ECMA standards?
Or how other bundlers implement this feature and make everything possible?

If any of the question can be answered, as detailed as possible, I think it will be huge help to the team to find out a way to implement this inside parcel.

@pipbeard Just suggestion.

[Mouvedia]

@shunia webpackMode

[benrobinson]

@shunia as @Mouvedia suggested above, I was definitely basing my idea of how this might work on webpackMode.

I'll do some poking around as far as ECMA standards, but for my use case, I would really have been happy with it just recursively trying to resolve/build dist files for each individual valid JS module under the specified path/pattern at build time, to be then later actually added to the page (from dist) at runtime. Obviously, my approach of just handling this myself as a pre-parcel step also solved my problem, so if this isn't a commonly needed feature for others then it may not be worthwhile.

[benrobinson]

@shunia here is the TC39 proposal that appears to address this issue:

https://github.com/tc39/proposal-dynamic-import

Really, the main difference between why this would be desirable in parcel over glob imports is that while glob imports make it easy on the developer who is importing lots of possible files without knowing which in particular will be used at runtime, this would still make it easy on the developer, while also ensuring that only the required modules were loaded at runtime, rather than just loading all possible files matching the pattern, which is obviously less efficient otherwise.

I may poke around and try to come up with a PR for this issue.

[DeMoorJasper]

I'll close this issue as it's very similar to #125