Jekyll: The Future of Theme Gems [Discussion]

[ashmaroli]

The future of theme gems as I see it..:
Requires Jekyll 4.0

Self-Contained Modules

Theme gems will be self-contained _insta-jekyll_-modules. All they need would be content. This allows users to freely switch between themes without breaking their site or losing their customizations. For this to happen a few things will be prescribed and expected to be followed. A theme-gem ecosystem would be:

• SCAFFOLD DIRECTORIES [ _layouts | _includes | _sass | _data | assets ]
• every theme gem shall contain a configuration file named the same as the gem-name e.g. minima.yml and it will reside inside _data/
• core styles and settings will be defined in minima.yml and referenced in templates with liquid using the theme namespace. e.g. {{ theme.myvariable }}
• the layout names will be standard for all theme-gems:
  
  
  
  • home for index
  
  
  • post for posts
  
  
  • page for pages
  
  
  • default as the parent layout for above layouts
  
  

When overriding defaults, the user creates the required file / copies from the gem_dir. into a directory named the same as the theme gem in use. e.g. minima/_data/minima.yml | minima/_layouts/ | minima/_includes/ etc

Jekyll 4.0 shall look for these directories first in the site.root and then in the gem.

Following this _convention_ ensures easy switching between theme gems at the same time preserving their customizations.

[vwochnik]

+1

[oncleben31]

_data theme folder could host locale strings files for themes with i18n features.

[ashmaroli]

@parkr, what do you think about modifying lib/~/data_reader.rb to include _data/ inside theme gems..?

[mmistakes]

I'm with @oncleben31 and @ashmaroli.

Bundling in _data files would be very useful. One of my themes currently uses a data file for all of the localized text found in its UI elements. Asking users to manually copy over that file when using a theme gem is kind of a bummer.

[jaybe-jekyll]

This is a neat topic of discussion. In recent related threads, themes have been enhanced to include seeded assets/, etc. for the purposes of providing a full, complete, and out-of-box-functional theme/site. Extending inclusion to other, additional areas such as pre-seeded/configured _data/ is exciting.

[ashmaroli]

I've created a plugin that somewhat realizes the idea being discussed here..

[arthurattwell]

"Jekyll 4.0 shall look for these directories first in the site.root and then in the gem."–@ashmaroli

Wouldn't it be clearer to keep themes in a _themes directory? A site root might contain any number of content directories, and grouping themes in _themes would make it easier to navigate content directories and probably easier for new users to understand.

[ashmaroli]

Wouldn't it be clearer to keep themes in a _themes directory?

Nope. Themes will continue to be located in the users Gem Paths as defined in the user's gem env

[DirtyF]

Could we have both worlds - that is a _theme directory pointing to the theme's gem files?

[pathawks]

Could we have both worlds - that is a _theme directory pointing to the theme's gem files?

I don't understand what you mean; could you elaborate?

[DirtyF]

@pathawks since theme's files are stored in the user's gems directory, could we create a _theme symlink that points to this directory, so that users can cd _theme to easily list theme's files or be able to copy files with cp _theme/_layouts/footer.html _layouts/footer.html for instance?

The idea is to provide a shortcut, but users should be aware that they can not modifiy gem's files.

[ashmaroli]

could we create a _theme symlink that points to this directory

@DirtyF, since symlinks do not have a solid support on Windows, this idea gets a :-1: from me

[arthurattwell]

Themes will continue to be located in the users Gem Paths as defined in the user's gem env

@ashmaroli Perhaps I'm not following your original proposal properly:

When overriding defaults, the user creates the required file / copies from the gem_dir. into a directory named the same as the theme gem in use. e.g. minima/_data/minima.yml | minima/_layouts/ | minima/_includes/ etc

Jekyll 4.0 shall look for these directories first in the site.root and then in the gem.

You're saying, if I understand correctly, that to override a theme default, I'd create, say, minima/_layouts/default.html in my Jekyll site.root. I'm suggesting that -- since one might be trying out several themes at any one time -- it may be better to create _themes/minima/_layouts/default.html. So Jekyll 4.0 would look for these directories in site.root/_themes first, and then in the gem.

Or am I misunderstanding what you mean by site.root?

[ashmaroli]

Or am I misunderstanding what you mean by site.root?

I see what you meant by your earlier comment, now. So, a :+1: to having _themes to house all theme-specific overrides at site.root in v4.0

[jekyllbot]

This issue has been automatically marked as stale because it has not been commented on for at least two months.

The resources of the Jekyll team are limited, and so we are asking for your help.

If this is a bug and you can still reproduce this error on the 3.3-stable or master branch, please reply with all of the information you have about it in order to keep the issue open.

If this is a feature request, please consider building it first as a plugin. Jekyll 3 introduced hooks which provide convenient access points throughout the Jekyll build pipeline whereby most needs can be fulfilled. If this is something that cannot be built as a plugin, then please provide more information about why in order to keep this issue open.

This issue will automatically be closed in two months if no further activity occurs. Thank you for all your contributions.

[ashmaroli]

Happy New Year @jekyllbot !! :laughing:

[shawnbot]

I agree with some folks here that supporting a _themes directory would be really, really useful. For one, it would allow people to publish sites to GitHub Pages (and other Jekyll-based platforms) without being constrained to whitelisted themes, and remove the burden of having to "unbundle" themes using the awkward process described in the Jekyll docs.

Here are some use cases to consider:

• A user wants to use a theme that isn't supported by GitHub Pages.
• A user wants to contribute changes to a theme, but doesn't know how to test it locally because they aren't familiar with Ruby and/or Gems, which makes testing locally difficult or impossible.
• A user wants the ability to easily switch between themes, or even build a site multiple times with different themes.

In all of these cases, support for a _themes directory has the potential to make the user's life easier. They can try a theme out by copying it to _themes and updating their Jekyll config, rather than monkeying with Bundler. They can publish their theme to GitHub without having to know how to publish a Gem. Theme users can install and keep the theme up-to-date with git submodules. And users can remove the theme later by removing a single directory, rather than having to hunt down all of its files individually (which is what people must do now when they follow these instructions.

The documentation itself alludes to "regular themes", which aren't really a thing without support for the _themes directory. I think the single-directory format would be a lot easier for some users to work with and allow folks to experiment more easily and with less risk of ending up in a bad state.

[gboone]

+1, @shawnbot. In addition to removing a theme being easier in a directory, updating it will be as well.

Another advantage is the ability have WordPress-style "child themes" that inherit the basic styles and templates from a parent but allow a user to build and override them as needed. #5343 has some discussion of this, too. Managing the templates of regular and "child" themes would be tremendously easier in a _themes directory for template designers who aren't familiar with Ruby and Gems.

[pathawks]

So then instead of have two different places to look for a layout/include, there would be three. 😕

[ashmaroli]

there would be three. :confused:

If backwards compatibility is not on the cards for 4.0

• then templates need not reside at the source root but should rather be housed within _themes
  theme-gem < overrides in _themes
• reduce fragmentation by enforcing a limited scope:
  all themes to have a common name-set for layouts:
  base => default.html index => home.html pages => page.html posts => post.html
  if that's too extreme, layouts shouldn't be hard-coded within documents. Use Front Matter Defaults.
  other (remaining) layouts to be converted into includes.
• penalize inclusion of {% include -- %} within markdown or content documents. All of that should be handled by layouts / includes.

With the above, trying out various themes become less stressful.

[shawnbot]

@pathawks gem-based themes can already inherit from one another in sneaky ways, e.g. by including other themes' Sass files. At the very least, I think directory-based themes, whether they inherit from other themes or not, would be easier to debug because you can search (with find, grep, etc.) the filesystem for similarly named files or contents. This is a lot harder to do with gem-based themes because the gem files typically live outside the working directory.

[pathawks]

I didn't think we had any concept of child themes yet.

It seems like including a theme in the working directory defeats the entire purpose of having a theme. Perhaps we should add more documentation explaining that a theme does not have to be published to RubyGems, or even exist on a remote server at all; Gems can already just be a directory.

[ashmaroli]

theme does not have to be published to RubyGems, or even exist on a remote server at all; Gems can already just be a directory

Isn't versioning and updating easier if the gem is hosted on say RubyGems being the whole point of having gem-based themes

[pathawks]

@ashmaroli Not if you are the only user of your theme.

I think the big win of Gem based themes is (should be) ease of installation.

[ashmaroli]

Ease of installation, yes: Gemify a theme and use it on more than one projects by the same author
Just have the Gemfile reference the theme..

But, we're discussing about switching various themes, previewing them, making adjustments, try out another theme, save overrides for previous themes, etc etc

[shawnbot]

Sorry if I wasn't clear: I meant that it's possible to do sneaky things within a Gem that allow you to informally "inherit" from another Gem-based theme; not that theme inheritance is a supported feature, nor that it should be.

I also agree that Gem-based themes are much easier to install for people who are familiar with Gems and Ruby. However, they're not usable on GitHub Pages unless they're on GitHub's very short whitelist. And for people who are less familiar with Ruby but know their way around the filesystem enough to put a simple Jekyll site together, I think that directory-based themes would be a huge boon.

Some context: I work for the U.S. government and am on the Web Design Standards team. We're looking at creating a Jekyll theme for the Standards that people with little to no git or GitHub (let alone Ruby or Gem) experience can use and deploy on GitHub Pages and other Jekyll-based platforms, such as Federalist. Federalist builds Jekyll sites in containerized environments and doesn't restrict Gems, so we can scaffold new sites with a Gem-based theme there. But some agencies and departments can and do publish directly to GitHub Pages, which means that we need to provide additional tooling to "unbundle" a Gem-based Standards theme and apply updates to the theme. If directory-based themes were a thing, then we could scaffold those sites with _themes/uswds set up as a git submodule that points to the theme repo, and every hosting environment would be able to handle it.

Directory-based themes are also easier for users to run locally because they don't necessarily involve having to use Bundler at all: clone the repo with --depth 2 (or use the GitHub desktop app), and if you have Jekyll installed globally you can just run jekyll serve in the project directory. 💥

In short: directory-based themes offer a simpler mechanism for distribution that's compatible with hosting environments that don't allow arbitrary Gems, and are potentially much easier for users to run locally.

[ashmaroli]

This discussion seems to have strayed far from my original intent.

@shawnbot, According to you, the chief necessity of directory-based themes is to be able to host on GitHub Pages using git submodules that'll will cater to people who have little to no prior experience with Git or GitHub Pages.
Confusing..

But the common ground for everyone in favor of a _themes directory is that Jekyll no longer looks for the SCAFFOLD DIRECTORIES — _layouts, _includes, _sass, _assets at the root of the site's source but rather in _themes/[THEME-NAME]/

:+1:

[shawnbot]

Yeah, I'm sorry if I hijacked the thread. I just wanted to +1000 the _themes suggestion proposed by others before me, and provide some more specific use cases.

I think the core concept of scaffold directories makes sense, and I assume that the Gem-based theme implementation already provides some way to prepend resolution paths for different types of files (includes, layouts, Sass partials). If I knew Ruby well enough I'd dive right in and see if I could hack together a solution, but I humbly defer to the maintainers and Jekyll hackers on whether this is an easily implemented feature.

Here's how I would expect it to work, assuming theme: name in the Jekyll config:

1. Attempt to require the gem name. If that succeeds, add its "scaffold directories" to the appropriate resolution paths, and perform any other Gem-based theme initialization necessary.
2. Otherwise, check for the existence of _themes/{name} in the source dir. If that exists, add all of its scaffold directories to the resolution paths instead.
3. If the directory doesn't exist, throw an error.

From there on out, everything _should_ just work the same way, right?

[adamvoss]

Another thing the structure presented would seem to go towards enabling would be having different files render with different themes in a single project (i.e. specifying theme in addition to the layout in a file's front matter). This is something I never thought I would want until I just did.

[ashmaroli]

would be having different files render with different themes in a single project

@vossad01 I never expected that. But out of curiosity, why would anyone want different files rendered with varying themes?

[arthurattwell]

@ashmaroli @vossad01 +1 for different files with varying themes. FWIW, we have a niche use case for this: we use Jekyll to produce books as websites, ebooks and in print. One Jekyll site might include several different books. At the moment, if we use themes, all books in a site must look roughly the same (i.e. they are based on the same theme). If different files could use different themes, one Jekyll site could contain any number of very different books.

[ashmaroli]

@arthurattwell Interesting use case, The way I see it, since the books are part of the same site, there is at least some common ground where everything converge to, and therefore the themes themselves must have a common ancestor.

The best solution hence is to develop an in-house theme to cater to this niche scenario and implement it via Front Matter Defaults:

• each book is a collection item with entries as chapters: /_book_1, /_book_2

• delegate these "books" with a layout of their own:

  ```
  defaults:

  • scope:
    path: ""
    type: book_1
    values:
    layout: book_1
  • scope:
    path: ""
    type: book_2
    values:
    layout: book_2
    ```
• these individual layouts descend from the parent layout say bookforge.html

this allows scalability and write new layouts / use existing ones for new books

[adamvoss]

I never expected that. But out of curiosity, why would anyone want different files rendered with varying themes?

Sorry I thought I responded to this previously.

What I recalled originally triggered this was GitHub Pages. At least some of those themes are only really built for a single page. When a project wants to grow beyond that (ex. add documentation pages) it may be perfectly willing to accept a different theme for the new (documentation) pages to to avoid/delay spending the time working on themes or being forced to change the appearance of its current page.

The other case that I thought of because of projects I have worked near, but was not actually attempted is similar to @arthurattwell's. The case of a project where its main website and its documentation website were developed separately, and ended up with different themes. As they have grown, it has become clear there would be efficiencies in merging the two. The ability to have each keep their own theme reduces the barrier for merging them and lets the work of refactoring to a common theme be able to more easily happen over time.

[parkr]

What unresolved questions are still present from this discussion? Can we close this issue and open targeted issues?

[ashmaroli]

While I'm happy with all the ideas that got discussed here, I admit that now there isn't a clear focus on what to take away from here and what not to..

My opinion is to let this remain open (to maintain visibility) and lock it to prevent further noise. New users can simply express their reaction on existing posts which will be eventually be re-visited once development on v4.0 starts.

Closing this would only cause it to drown in the sea of past issues..

[DirtyF]

I admit that now there isn't a clear focus on what to take away from here and what not to.

So let's slice this in carpaccio. Already remove _themes directory support from this thread, if you could slice your ideas in smaller parts, maybe this can go forward.

[jekyllbot]

This issue has been automatically marked as stale because it has not been commented on for at least two months.

The resources of the Jekyll team are limited, and so we are asking for your help.

If this is a bug and you can still reproduce this error on the 3.3-stable or master branch, please reply with all of the information you have about it in order to keep the issue open.

If this is a feature request, please consider building it first as a plugin. Jekyll 3 introduced hooks which provide convenient access points throughout the Jekyll build pipeline whereby most needs can be fulfilled. If this is something that cannot be built as a plugin, then please provide more information about why in order to keep this issue open.

This issue will automatically be closed in two months if no further activity occurs. Thank you for all your contributions.