Zfs: Github wiki may not render ZFS-on-Linux-Module-Parameters page due to it's size

[gmelikov]

Describe the problem you're observing

Github wiki can't render huge pages.
For example, I've added small update https://github.com/openzfs/zfs/wiki/ZFS-on-Linux-Module-Parameters/_compare/8510c572f34a04745bbe3b26a8eab710b07d1dbf...1c7f3b2000524567a4dba2d7c5b3d7826e068a74
and Github broke this page with error:

Looks like we need to pregenerate at least this page and host somewhere else.

Describe how to reproduce the problem

Add 20-30 symbols to page https://github.com/openzfs/zfs/wiki/ZFS-on-Linux-Module-Parameters

@richardelling you may be interested in this problem. I've shortened my change to make it work again.

[richardelling]

haha, seems someone forgot how big the internet is :-)
We can break it into several pages. The index is automatically generated from the contents, so
while we'll lose the all-in-one-page searchability, we won't lose content. Any thoughts on how to
break apart the content? An easy first step is to break out the various kernel modules.

[gmelikov]

To be honest, maybe it's the signal to think about appropriate documentation format and presentation place all over the OpenZFS project? I was thinking about using RST for that, track it in git and publicate on website (open-zfs.org). Github can partially show it's preview, many projects are doing so and it works good.

Github wiki has these problems for me:

• Not straightforward for new contributions (all or nothing, full edit right must be granted)
• Table of contents is manually generated
• Bad navigation (one manual structure without autogeneration of subpages)
• Can't be reused to generate for. ex. man page
• Page size limit (this issue problem)
• It's already not an only place with documentation, our docs are very fragmented (open-zfs.org, man pages, github wiki, other must-have to read things like delphix blog for raidz write amplification)

[richardelling]

man pages are not appropriate for this material. They really only work when they fit on a single screen. Also tables in man pages are a crying shame.
github wikis are managed through git, so I don't see RST adding any value.

[gmelikov]

@richardelling I agree with you that man pages can't get all docs effectively, but that was not a main point.

github wikis are managed through git, so I don't see RST adding any value.
But new contributors can't create a PR for Github wiki, that is a problem too.

Main point that I wanted to describe is: we have a problem with documentation, it's very fragmented, and github wiki is not the best instrument for that too (I described my thoughts earlier here). Yes, we can locally rework parameters page to live with it some time.

I propose to think about documentation strategically, have it in one place and with appropriate instrument. Github wiki + markdown can't give us good docs (mainly because it doesn't have methods to structurize it and generate ToCs). Default mediawiki (open-zfs.org) can't get a good generated structure too (but it may have it with addons, which is not so bad too).

Example of docs with RST and generated html:

• source https://github.com/openstack/keystone/tree/master/doc/source
• html https://docs.openstack.org/keystone/latest/index.html

I'm not insisting on RST, though, it's just en example of generated documentation, which is easy to contribute to and looks good in browser. One small bonus - part of docs may be generated from code source too.

So i propose to discuss strategic documentation course, maybe even on next Leadership Meeting, what do you think about it @ahrens @behlendorf ? Or maybe I'm wrong and everybody is ok with this situation.

[mskarbek]

+1 for keeping source in git for publication on openzfs.org. That way anybody could contribute small changes like notes about compatibility and gotchas in specific distros to avoid, for example, more issues like #10257. At least for me, it's easier to just commit that note then asking for permissions and work with GitHub wiki.

[behlendorf]

This sounds like a good topic for the Leadership Meeting. I agree, the current documentation situation isn't great and it would be nice to have everything in one place.

I really like what @anderbubble has started putting together at readthedocs. Personally, I'd be all for moving this, or something like it, in to the OpenZFS organization to make it easy for others to contribute.

• https://openzfs.readthedocs.io/en/latest/index.html
• https://github.com/anderbubble/openzfs-docs

If we could identify a few people willing to work on this we could get the ball rolling.

[anderbubble]

I haven't put much effort into this lately (frankly my momentum died when I had to start over due to licensing restrictions on the man pages) but I am absolutely interested to be involved in whatever capacity I'm given, up to and including leading the effort. Knowing that I'd be working with upstream support would go a long way!

[gmelikov]

I commit to move our github wiki on such RST-based platform this month so to look at it in real world, if there is nobody who'll do it faster (one-month doc hackaton, yay).

@anderbubble if you won't mind - I'll use some of your wrappers as a starting point, and maybe we can merge it later, looks like topics don't cross).

My plan:

• prepare a machinery for doc generation
• move github wiki into it (I'll be here in a month, this work is best as a one-person project, I think)
• publish it somewhere in official place
• move everything else we already have on it slowly (at this point anybody's help will be welcome)

[behlendorf]

@gmelikov that sounds great. Thanks!

[anderbubble]

@gmelikov what can I do to help?

[gmelikov]

So, I've succesfully moved full wiki into RST format, looks pretty, code and preview:

• https://github.com/gmelikov/openzfs-docs
• https://gmelikov.github.io/openzfs-docs/

Cleaned up something, reappear some pages missed in sidebar. Tried to make commits neat to review, initial commit is a full wiki repo converted by pandoc into RST without any changes from my side. Next commits are manual adoptation and cleanup, sphinx configuration. make html will generate documentation.

I like the process, looks like it's a good start to get it somewhere in official place, so we can redirect old github wiki into it.

There are several ways:

• integrate it into https://github.com/openzfs/zfs in docs folder in same branch
• use it as a separate repo, like https://github.com/zfsonlinux/zfsonlinux.github.com
• use it in different branch of main repo (I'm against it)

First variant gives us atomic updates of code and documentation, but will mess issues and PRs. Different repo is a good to start variant. @behlendorf what do you think about it?

Next thing I want to do is to add man pages into it, but before that feedback may help to choose a path.

@anderbubble after merging this one into something official it would be great to prepare some Basic concepts, workflow will be through a usual PR requests, if you may want to continue this thing.

[richardelling]

thanks @gmelikov ! However, IMHO the pages look better in github than on your wiki. Why? Because serif fonts are fine for the printed page, but are difficult to read online. It is a simple fix, I'm sure.

While I'm here, the separate repo approach is best.

[gmelikov]

@richardelling fortunately there are many ready to use themes and they are 100% customizable, I'll work out the font question, thank you for feedback, style sense is not my best part :)

Looks like sphinx can generate pretty man pages too! Will try pandoc on our man pages some time too, rst-based documentation converted to man and html pages is an interesting thing. In that case web doc and man pages may have some common rst parts.

[gmelikov]

I've changed the theme, it has better fonts and is more mobile-friendly.

[behlendorf]

That looks great. Setting up a separate repository sounds like the best approach to me. As a next step what do you about transferring your openzfs-docs github repository in to the openzfs organization? Then we could set a few people up with committer privileges (perhaps @gmelikov, @anderbubble, @richardelling?) and everyone can work through PRs to update the content.

[gmelikov]

@behlendorf I'm all to do this, tried to transfer it into openzfs org but I don't have create rights in it, so I've just made a request to transfer it to your personal account @behlendorf .

Can I help to deploy it?

• /docs/requirements.txt has all deps
• cd ./docs/ && make html will generate html into ./docs/_build/html/

[gmelikov]

We've published it!

• https://openzfs.github.io/openzfs-docs/
• https://github.com/openzfs/openzfs-docs
• Autodeploy was configured
• Later we may move to docs.openzfs.org.

I've already updated github wiki to redirect to new location.

@richardelling @anderbubble , I'd be glad to find someone who ready to review me there. And, of course, everybody is welcome with contributions!

[anderbubble]

@gmelikov thanks for including me! I'll look over it, and am glad to have a place to contribute docs that I feel really good about.

I have openzfs.readthedocs.io, but always with the intent to relinquish it if the OpenZFS project wanted it. Would there be a value in configuring it to build these docs?

[gmelikov]

@anderbubble thank you for an offer, we chose to publish via github pages, in this case we can build whatever we want and how we want via github actions.