Parcel: JSON with references to hashed files

[hirasso]

Hi there!

Now this would be amazing: Is there a way to tell parcel to put a .json file inside the ./dist folder that contains references to hashed files using their original names? I'm imagining something like the following:

Pseudo-code

{
  "some-file.ext": "some-file.content-hash.ext"
}

Usage Example

{
  "favicon.png" : "favicon.09bcacef.png",
  "house.svg": "house.33b919c8.svg"
}

That would come in very handy when also loading bundled files from outside a parcel bundle, like for example a PHP environment.

[DeMoorJasper]

Wouldn't be hard to create a plugin for it

[hirasso]

Wouldn't be hard to create a plugin for it

...not if I would know what I'm doing 😉

Do you have 1-2 hints on where a plugin would have to hook into the parcel code to do that? I'm imagining an event or something similar that gets fired with all bundled assets...

[DeMoorJasper]

I have a SSR framework, that actually takes full control over naming and such, so it's probably a little overkill. But might be nice to learn some things from. https://github.com/DeMoorJasper/blazingly

I think it's just easier if I would explain though.

A parcel plugin gets the bundler object, so you can listen for bundler.on('bundled'). Which will emit after each bundle and it has a callback with the mainBundle as it's argument (which you can use if bundler.bundleNameMap isn't enough)

// This is a plugin
module.exports = function (bundler) {
  bundler.on('bundled', () => {
    let aNiceMapWithAllNamesMappedToHashes = bundler.bundleNameMap;
    // Save it to a JSON File...
  });
};

[hirasso]

Thanks for this hint! When I call console.log(bundleNameMap) though, I get crazy file names pointing to the hashed file names. To stick with my original example:

Map {
  '1fba6721b387fcaf61c5a08533b919c8.svg' => 'house.33b919c8.svg',
  'aee30a003f28ca77bfde9e54efbd8b79.png' => 'favicon.09bcacef.png'
}

Is there also a point in the bundling process, where I can get the real original filenames, or a way to retrieve them using the keys of the bundleNameMap?

EDIT

I thought maybe I could retrieve them using bundler.bundleHashes, but they seem to not be related to each other.

[DeMoorJasper]

I kinda was afraid that was gonna be the case. You could manually create a similar map by going through each bundle recursively and using the name of the entryAsset (and skipping if one does not exist - this is in case of virtual things like HMR Js, sourcemaps, ...)

bundle.entryAsset.name => Original filename
bundle.childBundles => Iterate this recursively and you should be able to get a nice list
bundle.name => The hashed output filename

[hirasso]

Does this look good to you?

bundler.on('bundled', () => {
  let manifest = {};
  for( let bundle of bundler.mainBundle.childBundles ) {
    manifest[bundle.entryAsset.basename] = path.basename( bundle.name );
    for( let childBundle of bundle.childBundles ) {
      // skip if entryAsset is null
      if( !childBundle.entryAsset ) {
        continue;
      }
      manifest[childBundle.entryAsset.basename] = path.basename( childBundle.name );
    }
  }
  // TODO: Save it to a JSON File...
});

[DeMoorJasper]

I would make it recursive, as each childBundle can have childBundles of it's own, it's a tree of bundles.

[hirasso]

Of course, makes sense. That's just a first draft. I also noticed that checking for !childBundle.entryAsset doesn't work, since the entryAsset for css for example is null, even if the file exists. I'll better use fs.existsSync(bundle.name), right?

[DeMoorJasper]

Thought every bundle had an entryAsset if it wasn't virtual.

You could also use Array.from(bundle.assets.values())[0] or something, to get the original name I think

[hirasso]

Works great, thanks! Here's the cleaned up code:

bundler.on('bundled', () => {
  let manifest = addToManifest( {}, bundler.mainBundle.childBundles );
  console.log( manifest );
  // TODO: Save it to a JSON File...
});
function addToManifest( manifest, bundles ) {
  for( let bundle of bundles ) {
    let original = Array.from( bundle.assets.values() )[0].basename;
    let hashed = path.basename( bundle.name );
    manifest[original] = path.basename( hashed );
    manifest = addToManifest( manifest, bundle.childBundles )
  }
  return manifest;
}

[DeMoorJasper]

Glad I could help :)

[hirasso]

Oh sorry, just realized that Array.from( bundle.assets.values() )[0].basename doesn't quite work. I get unexpected results:

{
  "admin.js": "admin.map",
  "admin.scss": "admin.css",
  "app.js": "app.map",
  "flickity.css": "app.css",
  "house.svg": "house.33b919c8.svg"
}

I'll have another look next week sometime with a fresh head and report back as soon as I found a working solution.

[DeMoorJasper]

Yeah figured there are some bugs in that code.

Should be something like this:

bundler.on('bundled', () => {
  let manifest = addToManifest(bundler.mainBundle);
  console.log(manifest);
});
function addToManifest(bundle, manifest = {}) {
  manifest[Array.from(bundle.assets.values())[0].basename] = path.basename(bundle.name);

  if (bundle.childBundles.size() > 0) {
    for (let childBundle of bundle.childBundles.values()) {
      manifest = addToManifest(childBundle, manifest);
    }
  }
  return manifest;
}

[hirasso]

I think I found the solution. Seems like if entryAsset is null, the output file doesn't contain a hash. So we can just use it's name in that case (now also with the fs.writeFile):

const bundler = new Bundler(files, options);
const bundle = await bundler.bundle();
// build the manifest once on initialization
generateManifest( bundler );
// rebuild the manifest on every change (watch mode)
bundler.on('bundled', () => generateManifest( bundler ) );

/**
 * Create a manifest file that contains references to hashed assets
 * @param {Class} bundler – the parcel bundler instance
 */
function generateManifest( bundler ) {
  bundler.on('bundled', () => {
    let manifest = addToManifest( bundler.mainBundle );
    fs.writeFile(`${bundler.options.outDir}/manifest.json`, JSON.stringify(manifest, null, 2), 'utf8', function (err) {
      if (err) { return console.log(err); }
    });
  });
}

/**
 * Add bundeled assets to manifest.json
 * @param {Class} bundle – the parcel bundle
 * @param {Object} manifest – the manifest object
 */
function addToManifest( bundle, manifest = {} ) {
  // if this bundle has a name (e.g. it exists), add it to the manifest
  // (bundles with multiple entryPoints don't have a name at their root)
  if (bundle.name) {
    let distName = path.basename( bundle.name );
    let srcName = ((bundle || {}).entryAsset || {}).basename;
    // Some assets (namely assets with type `css`) don't seem to have an entryAsset defined in parcel. 
    // In that case, just use the distName. It will result in { "style.css" : "style.css" }
    if( !srcName ) {
      srcName = distName;
    }
    manifest[srcName] = distName;
  }

  if (bundle.childBundles.size > 0) {
    for (let childBundle of bundle.childBundles.values()) {
      manifest = addToManifest( childBundle, manifest )
    }
  }
  return manifest;
}

Edit

Updated the script to accept both single or multiple parcel entry files

[hirasso]

Oh sorry, moved to fast. I'll read through your code the next days. It's getting late over here in Germany ;)

[hirasso]

@DeMoorJasper So, I think I found the issue. Maybe it is related to my setup. I'm using three .js files as entry points for parcel:

const bundler = new Bundler('src/*.js', options);

That leads to the following structure for my bundler.mainBundle:

Bundle {
  type: undefined,
  name: undefined,
  parentBundle: undefined,
  entryAsset: null,
  assets: Set {},
  childBundles: 
   Set {
     Bundle {
     type: 'js',
     name: './dist/script-one.js',
     parentBundle: [Circular],
     entryAsset: [Object],
     assets: [Object],
     childBundles: [Object],
     siblingBundles: [Object],
     siblingBundlesMap: [Object],
     offsets: Map {},
     totalSize: 0,
     bundleTime: 0,
     isolated: undefined },
     Bundle {
     type: 'js',
     name: './dist/script-two.js',
     parentBundle: [Circular],
     entryAsset: [Object],
     assets: [Object],
     childBundles: [Object],
     siblingBundles: [Object],
     siblingBundlesMap: [Object],
     offsets: Map {},
     totalSize: 0,
     bundleTime: 0,
     isolated: undefined },
     Bundle {
     type: 'js',
     name: './dist/script-three.js',
     parentBundle: [Circular],
     entryAsset: [Object],
     assets: [Object],
     childBundles: [Object],
     siblingBundles: [Object],
     siblingBundlesMap: [Object],
     offsets: Map {},
     totalSize: 0,
     bundleTime: 0,
     isolated: undefined } },
  siblingBundles: Set {},
  siblingBundlesMap: Map {},
  offsets: Map {},
  totalSize: 0,
  bundleTime: 0,
  isolated: undefined }

...wich means thatArray.from(bundle.assets.values())[0].basename in your code will be undefined when the function is first being called.

My script skips the mainBundle and just traverses through the children. It works fine with my setup. I'll keep on using it for a bit and update the code here if I run into issues.

[DeMoorJasper]

Ow yeah that makes sense, as it's a tree, it needs a root, which in this case is an empty bundle, as we have to have something to append the child bundles too.

This is definitely setup related, as a setup with only one entrypoint will have that as it's entrypoint instead of a virtual bundle.

[hirasso]

Sorry for the last [deleted] comment, the 'bundled'-event works just fine from inside a parcel-plugin.

[hirasso]

Created a parcel plugin for this:

https://www.npmjs.com/package/parcel-plugin-manifest
https://github.com/hirasso/parcel-plugin-manifest

Would be very grateful for any feedback – it's my first npm package ever. Also I don't know if this would need a test. If so, I would need to learn how to write tests, as well – never have done that either 😅...so many things to do for such a simple script 😉

[DeMoorJasper]

@hirasso tests are usually a good idea. I personally don't do it untill other people actually start using it. Feel free to add this plugin to https://github.com/parcel-bundler/awesome-parcel

[DeMoorJasper]

Gonna close this issue as it has been resolved :)

[hirasso]

Yeah, I get it :) I started writing my first test, great fun!
While doing that I realized that I can't rely on the entryAsset for my goal. There are just too many edge cases. I would really need to know what the hash of the bundle is. Would it be possible to add it as an entry to the bundle? For example:

Bundle {
  type: 'css',
  name: './dist/app.cbac04c8.css',
  hash: 'cbac04c8', // this would finally solve all my problems I guess...
  plainName: './dist/app.css', // ...or even this ;)
  parentBundle: 
   Bundle {
     type: 'html',
     [...]

[hirasso]

...or I might even have found a bug in parcel? I keep on hanging on the fact that a .scss file imported in a .js file doesn't have an entryAsset...

[hirasso]

OMG, just found out that there already is a plugin exactly doing this!! 😱

https://github.com/mugi-uno/parcel-plugin-bundle-manifest

I'll take mine down.