Jupyter-book: Explore using another SSG (Nikola, Hugo, Gatsby?) to build the site

[choldgraf]

In a recent twitter thread, @jph00 and @damianavila suggested that Nikola might be a good option for a project like Jupyter Book. Others have also suggested that either Hugo or Gatsby is a good choice. This is an issue to keep track of whether this is worth pursuing.

Feel free to add to either of the lists below.

Reasons for changing

• Jekyll is fairly slow (it takes many seconds to build a site or update after content is changed for non-trivial-sized books).
• Ruby is a gigantic pain to install and has a very different experience on different operating systems.

Generic downsides

Some general challenges that would come with a new SSG

• An up-front cost of switching over the templates / config / build / documentation to nikola instead of jekyll
• Another confusion cost of changing the usage pattern for users
• Users would now have to build the raw HTML and host that online (e.g. with gh-pages or netlify). gh-pages would no longer auto-build for them
• We'd lose some plugins like jekyll-scholar, for which nothing like this exists in nikola

SSG-specific pros

Hugo

• Super easy to install
• Has a book theme already built in: themes.gohugo.io/hugo-book demo here
• Has pandoc conversion support (but I'm not sure how to use it exactly...)

Gatsby

• React seems super cool (though I don't have much expertise)
• Nteract has a lot of great react work done with notebook interfaces (https://components.nteract.io/) and will probably continue improving this
• There is already an ipynb transformer for Gatsby (https://www.gatsbyjs.org/packages/@gatsby-contrib/gatsby-transformer-ipynb/)
• this is probably the most useful skillet to learn from a web UI standpoint...

Nikola

• It can use Mako for templating, instead of liquid, which could potentially result in cleaner templates etc.
• It uses Python, which more people in the jupyter ecosystem already know
• Nikola also has first-class jupyter notebook support
• It looks like porting from jekyll would be really easy

[yuvipanda]

What are your thoughts on some abstraction that lets people use their static site generator of choice? I think everyone's gonna have a favorite that fits their use case better than others, and they'll all have upsides and downsides. You can't pick one that works for everyone.

[choldgraf]

I think that's related to #42 as well, yeah? I think in general I'm +1 on removing the requirement that people use particular build systems. On the other, how much complexity would it add to accommodate this?

[gaow]

It uses Python, not Ruby, to build. This means we avoid the hard dependency on Ruby and all of the confusion that it (may) bring.

This can be addressed in part by using a docker image for the jekyll part -- this is in fact how I use jupyter-book. The Dockerfile has 4 lines:

    FROM jekyll/jekyll:3.8.5
    COPY Gemfile .
    RUN apk --update add make gcc g++ libc-dev
    RUN bundler install

Then you run

docker build -t jekyll-book .

to build the image, and

docker run --rm --security-opt label:disable  \
   -v /some/path:/some/path  -t -w=/work/directory -u 1000:1000   \
   jekyll-book bundle exec jekyll build

where -v and -w options are to mount proper paths and to set workdir, respectively, as needed. -u is my UID mapping. This long docker command can easily be built to some command alias or to be dynamically generated for users if you use Python to detect some of these parameters.

I agree for those who does not use Ruby the setup can be a pain. But docker users might find this alternative setup very convenient.

[choldgraf]

wow that is very cool! thanks for sharing...maybe we can incorporate this workflow into the docs somehow

[emdupre]

Has there been any consideration of Hugo instead of Nikola ?

I've found a few anecdotes where folks discussing moving from Nikola to Hugo, and it seems like they have a number of themes that we could format content into.

I haven't tried either SSG, so I'm just curious if anyone's already thought about this option !

EDIT: There's also a direct comparison of Hugo and Jekyll here, for what it's worth. And a tool to directly import Jekyll into Hugo !

[choldgraf]

@emdupre I've heard that Hugo is very good as well (and probably the most well-polished of all the SSGs). Then there's also gatsby which uses react (and might let us do things like use nteract components to render site elements).

The main question I have when it comes to other SSGs is: will it be easier for users to install the stack on their computer (aka, could we get to the point where we assume anybody could build the HTML directly, instead of uploading the markdown files to github). Right now I don't think we can assume this, because installing Ruby kinda sucks if you're not on *nix :-/

[emdupre]

Yes, unfortunately....

The reason I pointed to Hugo was it seems like the development community there is quite active, and there are already tools to move from Jekyll to Hugo (and some nice Hugo templates). But I haven't tried installing Go across systems, so I can't comment to ease of use.

[yuvipanda]

For me personally, the best thing about Hugo is that you do not need to install Go on the target system. It's a single executable you can download in any fashion on all 3 major platforms, and works without the user ever having to know it's written in Go.

[choldgraf]

@yuvipanda that is indeed an extremely compelling use-case :-)

@emdupre do you think you'd have time to give a shot at the "auto convert" functionality for Hugo in order to see how much of a pain it'd be to convert over? I think the biggest challenge would be converting over the liquid syntax...

[emdupre]

That is a great reason, @yuvipanda ! Now I'm even more excited to try it out.

I'll give it a spin @choldgraf and report back :smile:

[choldgraf]

yayyy thanks!

[akhmerov]

I've had a decent success with mkdocs + mkdocs-material theme, see https://solidstate.quantumtinkerer.tudelft.nl . Didn't figure out how to hook up bibtex yet, but on the other hand it seems that the markdown used by mkdocs-material is a superset of the one used in the jupyter-book. And for a project like this, I feel, a sufficiently flexible markup language is important (I believe jupyter-book has much less than bookdown+rmarkdown. I believe that asciidoc would be a much better format in the long run, but unfortunately it seems to be way less popular.

[choldgraf]

Another point in Hugo's favor:

Apparently there's a Hugo Book theme:

https://themes.gohugo.io/hugo-book/

This might make the port a much easier proposition...here's a demo:

https://themes.gohugo.io/theme/hugo-book/

[emdupre]

Had not seen the book theme ! Great find. Tried it out and still having style related warnings and one SCSS error:

Can keep poking at it, though !

[choldgraf]

thanks for looking into it @emdupre , I'm working through the Hugo docs too :-)

Another thing to think about is that nteract has done a bunch of awesome work in building in notebook components (https://components.nteract.io/) and Gatsby (SSG based on react) actually has a transformer just for notebooks so you wouldn't need to convert them into something else before importing them. so that might be another option (especially given that I suspect there will be more notebook-specific work done in react compared with hugo)

(I've tried to compile all of this into the top level comment)

[rahuldave]

I spent some time this weekend trying jupyter-book out. The jekyll dependencies were a PITA on my mac and I failed. I managed to get it to work on a digital-ocean ubuntu 18.04 droplet (but the redirection is broken out of the box which should be a minor fix).

Having to have 5-6 packages do native builds including a C compiler just seems wrong.

I noticed that once the toc.yml and markdowns are built jekyll takes over. Nikola and hugo both support a "_data" directory kind of notion which should make toc's easy. Nikola uses jinja templates and thus porting the templates might be easier. Hugo's one executable makes it super user friendly.

[choldgraf]

@rahuldave interesting, I didn't know nikola supports liquid. I wonder if that means accomplishing the same thing with nikola would be super, super simple to implement as a current fix (we could keep exploring Hugo as well but it could be more of a long-term and "only if worth it" kinda thing) https://getnikola.com/creating-a-theme.html

[choldgraf]

hmmm, looking into it a bit more, I think that nikola uses jinja, and not liquid...though they seem pretty close to one another?

[rahuldave]

nikola supports jinja not liquid, but they are very similar...

[choldgraf]

Another note, looks like someone has a notebook to Hugo converter https://discourse.gohugo.io/t/ipython-notebook-support/3211/10

[choldgraf]

another quick update: I managed to get the hugo book theme working pretty quickly! Here's what a rendered page looks like with really minimal CSS changes:

obviously still no interact buttons, javascript, etc...but what do folks think about this as a base to work from?

[emdupre]

That's great !!! I'm all in on switching over -- thanks for working on this :tada:

[rahuldave]

This is super! Is there a branch to play with this?

[choldgraf]

@rahuldave totally still in a proof-of-concept mode now, but I'll try to get a PR done soon :-)

[jasmainak]

@choldgraf recommended me to comment here. Maybe I try to rephrase what I asked before. As a user who switches between different computers, I find it easier to just push to github and wait a few moments rather than figure out the dependency issues on each workstation.

What do other similar libraries do? sphinx-gallery and RISE come to my mind. In RISE, I can click a button and shows me the html -- with no dependencies as far as I can tell. So my naive question is -- is it possible to make this work by copying a few javascript files or is the SSG dependency something that one cannot get around?

[epogrebnyak]

Yes, unfortunately....

The reason I pointed to Hugo was it seems like the development community there is quite active, and there are already tools to move from Jekyll to Hugo (and some nice Hugo templates). But I haven't tried installing Go across systems, so I can't comment to ease of use.

You do notneed to install Go to use Hugo, Hugo is written in Go, but distributed as a binary. You just use an executable file for your system.

[choldgraf]

closing as this will be superceded by https://beta.jupyterbook.org/intro.html