Fractal: Fractal as static site generator and/or plugin?

[shawnbot]

Forgive me if this way off-base, but re: modularity discussed in #197, you might consider either supporting plugins for a popular static site generator (via a common API) or essentially turning Fractal into a static site generator plugin that can be run in isolation. That way, Fractal could leverage an ecosystem of plugins that already exist (e.g. for compiling Sass or rendering Markdown) rather than needing to build and support its own.

For instance, Metalsmith does one thing well that (to my eye, at least) appears to map directly to at least one part of Fractal: it walks a directory and creates an in-memory "file system" object with both site- and file-specific metadata, then provides a plugin API for modifying any/all of them. In practice, this would mean replacing fs.js/ffs with Metalsmith and rewriting the core to work with a its file system object and data objects. Fractal's "core" Metalsmith plugin could then either just build the component data structure, or load the theme specified in the options and tell it to render:

// options here can come either directly from the JS API or
// from the dependent module's package.json
modul.exports = function plugin(options) {
  return function fractalize(files, metalsmith, done) {
    const metadata = metalsmith.metadata();
    const opts = Object.assign({}, metadata.fractal, options);
    const fractal = metadata.fractal = new Fractal(opts);
    return async.eachSeries(
      Object.keys(files),
      // this is a fictional API, but something like this would need to exist
      (path, next) => fractal.addFile(path, files[path], next),
      // finally, reconcile all the files into their respective components
      error => error ? done(error) : fractal.build(done)
    );
  };
};

The Fractal CLI could then use other plugins to do the watching, serving w/live reloading, etc.

Metalsmith is just an example—there are a number of similar frameworks, each with their own ecosystems. There's obviously a good deal of risk involved in hitching Fractal to the success of another framework, but I think it might be worth it if you can leverage the tools and talents of an existing community.

[allmarkedup]

@shawnbot not way off base at all! In fact the very first prototype of Fractal I put together attempted to use Metalsmith as it's core filesystem parser. I ran into a fair few issues at the time although many of them may have been because I was still not completely clear as to the shape of the thing that I was trying to build at that point.

I'll certainly look at it (or similar projects) again for v2 development as I completely agree with you about the advantages of building on top of an existing community and plugin ecosystem. Metalsmith is certainly a good fit for a part of the parsing process; however the situation I don't want to end up in is one where differences in the approach of Metalsmith to Fractal (and frankly some of the quirks of Fractal itself!) end up limiting functionality that can be built into the core.

I guess an example of this would be the 'multiple component sources' feature discussed in the other issue - I'm sure that this is _possible_ if we were using Metalsmith under the hood but it's not necessarily in line with it's core model. I don't want to end up having to build so much 'wrapper' code around another tool that it ends up being more complex that just writing a more focussed equivalent specifically for Fractal.

Another thing that occurs to me is that fundamentally Metalsmith has a pretty 1:1 relationship with input file => output file, whereas the 'compile' step involved in Fractal (where multiple files are effectively grouped so as to be treated as one component) falls outside of that mental model. Your example shows how that step could be handled outside of Metalsmith but some part of me is slightly unsure about saying "you can use Metalsmith plugins up until here (i.e. the file parsing process) but then not here (i.e. extending component configuration/capabilities)."

Hopefully that makes some sense. But I do like the thought of building on Metalsmith (or similar) if the tradeoffs are worth it, so I'll spend a little time looking into it. And if it's not suitable in iteslf I do think that the simplicity of the Metalsmith approach and the ease of plugin development is certainly something worth emulating for the Fractal component parser/compiler if possible.

Might be also worth noting that my current first pass at rewriting the base filesystem parser actually builds on the glob-stream and glob-watcher packages developed for use in Gulp - another tool that I think has a lot of potential similarities with Fractal in the initial parse stage requirements. Indeed the File package I've recently added is modelled around a read-only, Fractal-specific version of Gulp's Vinyl file format.

[allmarkedup]

@shawnbot I've been looking into this a little today and I don't think that Metalsmith is actually going to be a good fit, although not necessarily because of the reasons I first thought! One of the main ones is that the mutability of key properties (like paths) would be very problematic for Fractal, so a lot of valid metalsmith plugins, if used, would cause issues down the line. It could certainly be worked around but I'm not sure that the end result would be much of a net gain for Fractal.

However, I like the simplicity and plugin-based nature of the metalsmith model, so I'm going to take another look at if the core filesystem parser could be re-implemented in a similar fashion, with a couple of tweaks to ensure that plugins don't unintentionally clash with information the component 'compile' step would need further down the line.

[shawnbot]

Sounds good, @allmarkedup! Glad to hear this isn't off-base, and I agree that Metalsmith may not be the right framework or ecosystem to plug into. If you can find a solid foundation in another framework, I think it could really help focus Fractal on what it does best. 🍻

[micahgodbolt]

I'm giving this a try in that I'm creating a new site that uses fractal templates to drive metalsmith's layout files. Basically setting partials directory as fractal/components and using {{> component/component }} to pull templates in.

I did run into one issue with metalsmith-layouts where non template files in the component folder were causing problems.

Here's the current proposed fix for that.

https://github.com/superwolff/metalsmith-layouts/pull/116

I"ll be sure to keep everyone updated on my progress of combining the two.

[ismay]

@micahgodbolt Btw, metalsmith-in-place v2 was just released, which is in my opinion better suited for templating than layouts in most use-cases. It uses jstransformers and can easily be extended with a custom rendering engine if needed. Might be useful.

[micahgodbolt]

@ismay But i still want to write my blog posts in markdown and then render them through a template. That's what metalsmith-layouts is for, right?

[ismay]

Well layouts is just a polyfill for templating engines that don't support extending templates (like handlebars). So it's functionally the same as in-place, but with a polyfill.

So for example, for your blog. Jstransformers (and thus in-place) transform based on file extension, so you'd name a file that you want to process blog-post.swig.md. Which would run the file through a markdown processor and then through a swig processor (right-to-left). Swig (like a lot of other templating languages) supports extends, so you could do:

{% extends "./layout.swig" %}

{% block body %}What's in here will be put in the "body" block of layout.swig{% endblock %}

In your template. And a lot more. With a decent templating engine like swig in-place offers a much more enjoyable and powerful workflow than layouts imho.

[micahgodbolt]

@ismay thanks for the clarification! I'm wanting to find a way to keep my markdown files agnostic of the way they get rendered (other than index.html in layout key). Maybe I'll just end up writing my own twig render inline in metalsmith. Anyway, I'll take the rest of this convo out of this closed fractal issue :D