Venus: Set up docs to be editable by PR

[mishmosh]

Description

It's time to enable editing ~the wiki~ [edit: the docs] by PR. Currently only people with write access can edit the wiki, so we lose out on lots of potentially great contributions.

This blog post documents how to enable PRs, but keep the findability of a wiki.
https://www.growingwiththeweb.com/2016/07/enabling-pull-requests-on-github-wikis.html

Acceptance criteria

• The workflow is low-burden on infra (and this is confirmed by infra team)
• Assuming green light, implement it
• When this is complete, someone without write permissions to go-filecoin can submit a PR to edit the docs.

Risks + pitfalls

• Watch out for fiddly permissions

Where to begin

[rosalinekarr]

@ognots, can you look into this blog post when you have the time and add some details about translating some of the TravisCI instructions into CircleCI instructions for our specific setup?

[eshon]

Here's a proposal suggested by @yusefnapora in discussion with @michellebrous. This way Devs and Contributors can keep their docs + guides in their home repo for easy updating.

This would allow us to start a cross-repo Docs portal for the Filecoin Project as a whole as other projects add docs later on.

Steps:

• Move the current go-filecoin Wiki into a new filecoin-project/gofilecoin/docs folder - allows for community PRs
• Pull <your_repo>/docs/ folders across the filecoin project into a new filecoin-project/metadocs repo (git submodules + cron? Some other trigger + watch? Ask yusef for help.)
• Build a Docs website / portal from the new metadocs repo.

Docs Example using Docsify:

Here's the current Wiki as a Docsify site. Edit on Github will launch an editable file if you have permissions, otherwise it shows a "please fork and create a PR" message.

(Btw, Docsify is dynamic - not a static site generator - so as soon as you update an .md file the website will update soon via gh-pages. Is this the best Docs Framework to use? If you've got another let's discuss, we've been doing a comparison. Docsify is open source so we can customize this theme, add a global nav header, add Algolia search for free - suggested by Mhz - add more cool toppings, the works.)

[mishmosh]

Love it! This proposal hits the top criteria in my mind:

• docs will be editable by PR -- enabling community contributions, and ensuring updates are reviewed by more than 1 person.
• minimal deployment burden
• supports smarter nav

[yusefnapora]

Algolia! That's the doc search engine whose name I couldn't remember :)

I haven't used Docsify but have read their docs, and it seems nice. The results are definitely nice to navigate and read, and I like the low-friction to editing.

For building the metadocs repo, we could maybe have a CI job on each main repo (go-filecoin, etc) that triggers a job on the metadocs repo using the Circle CI API if the docs content changes. For that to work, we need to generate a Circle CI api token and put it somewhere the other repos can grab it, like a Circle CI context variable.

[eshon]

If we plan to use Docsify, to support the Edit on Github plugin button directly to the original repo, we'd need to hack this Docsify plugin to refer to a possible different Github repo within subdirectories. I think it's feasible.

Multi-repo seems to be a common docs site generation problem (e.g. https://antora.org/ solves it but we definitely don't want to move away from markdown)

[anorth]

I would like to retain the ability for committers and other blessed contributors to be able to edit wiki directly, without requiring a PR review cycle. I think that lowering the barrier to updating the docs will lead to more thorough and up-to-date docs.

IMO the wiki is the best UI option available to us for this. I also think that [Wiki Links] are easier than copy/pasting absolute (or crafting relative) URLs, but am willing to sacrifice this for other goals.

My personal preference is for the workflow referred to in the link in the OP. Another repo for the docs is also acceptable, with the direct edit in GitHub functionality. Both allow for PRs.

Note that the specs team are also separately working on publishing docs direct from a repo: https://filecoin-project.github.io/specs/docs/network-protocols/. It would be unfortunate if we ended up with two different systems for essentially the same thing. cc @dignifiedquire @pooja

I'll support whatever decision is made by someone willing to do the work, though.

[eshon]

Took a closer look at Yusef's good work in https://github.com/filecoin-project/engine/issues/26 and successfully set up the 2-way sync between a test Wiki and a PR-able Docs repo (and Docs site). So we can keep the Wiki as-is and devs can keep making edits there. (A human will need to review docs PRs and possible merge conflicts in a CI once in a while.)

I think as long as any docs are in Markdown (or close enough) we can reach for a consistent, searchable docs portal for all Filecoin repos with good search and analytics later on.

[michellebrous]

Thanks @eshon! So to clarify, the Wiki and the PR-able Docs Repo (and Site) will exist separately, but are synced so that any changes to the Wiki will trigger a PR to the Docs Repo. Will the Docs Site be out of sync with the Wiki until the triggered PRs are manually approved? And this means the public can view two sources of documentation: the Wiki and the Docs Site?

Just want to make sure I understand this flow clearly.

[eshon]

Still working on the CI script and will probably need review from yusef and/or infra but...

Wiki => MetaDocs: automatic syncing on Wiki edit

Any go-filecoin Wiki update triggers a Github webhook that auto-updates the MetaDocs repo (it'll be automatically pushed via CircleCI, not by PR).

MetaDocs => Wiki: by PR or local changes

Yep, any PRs filed in the MetaDocs repo will be out of sync until approved - PRs usually take a while for someone to review anyway. Once they are approved & merged (or if anyone with permission makes local changes to the MetaDocs repo) the CI script will automatically push them to the Wiki. Or a human can update the Wiki manually based on those PRs if they know there will be git merge conflicts.

And yeah the public will be able to find 2 sources of documentation and make PR requests to MetaDocs. Wiki and MetaDocs should always be in sync unless there's a CI error / merge conflict - there'll be emails notifying someone. We're just trying out a MetaDocs portal to see if it makes sense to do. Wiki-wise many devs really like Github wikis (e.g. https://github.com/ethereum/wiki/wiki endures) so its fine to keep that around too.

All markdown files can use Github Wiki-style markdown - Docsify can be extended to render the weird Wiki image directory tags so the source files can always be identical.

I am having fun learning CircleCI.

[mishmosh]

@eshon can you propose a release checklist here?

[eshon]

The 2-way Github wiki sync script is very kludgy and having a wiki as well as docs repo can be confusing for community contributors.

Moving all documentation to filecoin-project/docs would better enable community PRs in a logical repo like ipfs/docs as well as support Suggest an Edit links. (Someone's already proposed a Chinese translation of the go-filecoin wiki via PR to filecoin-project/docs.)

Proposal:

• Move the wiki to a subdirectory of filecoin-project/docs as a tutorial, then add redirects on the wiki. Keep troubleshooting content on the go-filecoin wiki so its easy for the dev team to update.
• Use filecoin-project/docs to host all documentation content.
• Deploy a Docs portal to docs.filecoin.io e.g. https://filecoin-project.github.io/metadocs/ (theme will change as we add more content)

Steps to move to this workflow:

• [x] Ask go-filecoin team to transition away from having the Getting Started tutorial on their wiki and move it to filecoin-project/docs/go-filecoin-tutorial
• [x] Add a redirect note on the go-filecoin WIki to the Docs site but keep sections like Troubleshooting, Known Issues there
• [x] Ask Infra to set up this CI script with a machine user, it syncs the docs content repo and docs website repo
• [x] Ask Infra to map docs.filecoin.io and filecoin.io/docs to the docs website gh-pages, also ask them to map blog.filecoin.io
• [x] Update all editable links on Filecoin.io and community channels

I'm aware there isn't much Documentation content and will work on trying to create more in the coming sprints.

@mishmosh - that's all I can think of for now

[anorth]

Ok

On Fri., 26 Jul. 2019, 20:16 Eva, [e-mail] wrote:

The 2-way Github wiki sync https://github.com/protocol/infra/issues/477
script is very kludgy and having a wiki as well as docs repo can be
confusing for community contributors.

Moving all documentation to filecoin-project/docs
https://github.com/filecoin-project/docs would better enable community
PRs in a logical repo like ipfs/docs as well as support Suggest an Edit
links. (Someone's already proposed a Chinese translation of the
go-filecoin wiki https://github.com/filecoin-project/docs/pull/1/files
via PR to filecoin-project/docs.)

Proposal:

• Move the wiki to a subdirectory of filecoin-project/docs as a
  tutorial, then add redirects on the wiki. Keep troubleshooting content on
  the go-filecoin wiki so its easy for the dev team to update.
• Use filecoin-project/docs to host all documentation content.
• Deploy a Docs portal to docs.filecoin.io e.g.
  https://filecoin-project.github.io/metadocs/ (theme will change as we
  add more content)

Steps to move to this workflow:

• Ask go-filecoin team to transition away from having the Getting
  Started tutorial on their wiki and move it to
  filecoin-project/docs/go-filecoin-tutorial
• Add a redirect note on the go-filecoin WIki to the Docs site but
  keep sections like Troubleshooting, Known Issues there
• Ask Infra to set up this CI script
  https://github.com/filecoin-project/metadocs/blob/master/.circleci/config.yml
  with a machine user, it syncs the docs content repo and docs website repo
• Ask Infra to map docs.filecoin.io and filecoin.io/docs to the docs
  website gh-pages, also ask them to map blog.filecoin.io
• Update all editable links on Filecoin.io and community channels

I'm aware there isn't much Documentation content and will work on trying
to create more in the coming sprints.

@mishmosh https://github.com/mishmosh - that's all I can think of for
now

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/filecoin-project/go-filecoin/issues/1895?email_source=notifications&email_token=AADMW6VCQZGYP4MSKVBVAYTQBLFHHA5CNFSM4GXBOQ2KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD24FFKI#issuecomment-515396265,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AADMW6WOHY6MBJQSNYGAKZTQBLFHHANCNFSM4GXBOQ2A
.

[mishmosh]

Does the last step include announcing the change & inviting PRs in community channels? If so, LGTM!

@eshon and @ognots: Is next Friday Aug 2 a reasonable date to land this? (@ognots I think infra help is only needed for items 3 and 4)

[ognots]

I can definitely perform steps 3 and 4, but a Friday deadline is tight as I have a large queue of small requests. I'm on holiday August 8, so will definitely aim to have it done by August 7

[mishmosh]

@ognots thanks! I've created https://github.com/filecoin-project/go-filecoin-infra/issues/273 and https://github.com/filecoin-project/go-filecoin-infra/issues/274.

[ognots]

@mishmosh can this be closed?

[eshon]

The final step is deleting content from most pages from the Wiki and pointing those pages to the equivalent pages at https://docs.filecoin.io/go-filecoin-tutorial/Home.html.

If the go-filecoin team is ok with that and made aware today I can do that tomorrow.

[mishmosh]

This is a heads up to @filecoin-project/go-filecoin-committers that the great migration (of docs source from wiki to repo) will be happening tomorrow. As a result we'll be able to accept docs contributions from the whole community. Thanks @eshon for driving this to the end!