Netlify-cms: Docs require fundamental infrastructure overhaul to fully integrate with CI/CD pipeline.

[silverwebb925]

Is your feature request related to a problem? Please describe:

Yes. The Docs will _never_ stay up-to-date or comprehensive enough with the existing documentation development pipeline.

Individual section/articles have excellent content (barring minor outdated items), but what I’m seeing* implies that while the project has a good CI/CD pipeline for features and bugs, the Docs themselves are progressing almost as a separate development stream that has a more traditional trajectory. Conceptually, the current PR process _should_ keep the Docs updated... but the pipeline doesn't have enough built-in validation to really _ensure_ that the docs stay up-to-date with the state of the CMS.

I can already tell that everyone is onboard to bring the Docs up to speed, but I really must emphasize that _good documentation and smooth documentation workflow is essential to the sustainability of this project._ The accessibility of the Docs is directly tied to NetlifyCMS adoption, and the Docs provide a major route to bring new contributors into the fold (here I am, after all).

*I've read through the GitHub Issue archive pretty extensively, specifically paying attention to reorganization efforts related to the Docs, and have listed the most relevant issues under Additional Context at the end. Please let me know if there are additional issues to add for reference.

Personal Note:

I have a professional background in technical graphic design and master planning/documentation, as well as significant experience automating complex design/documentation tasks via JavaScript… but I've only recently begun pursuing software engineering as a career. Please forgive _and correct_ me if ever fail to grasp something you might consider basic in terms of CS or NetlifyCMS itself. I am a strong believer in RTFM, and appreciate any links or resources you want to share. Thank you!

Describe the solution:

I propose that fully integrating the documentation pipeline with the rest of the project's CI/CD should jump to (near) the top of the list for NetlifyCMS development priorities, and that only the bare-minimum of manual effort be expended to reorganize or revise the Docs within the current infrastructure. Once a more robust content schema and contribution workflow is in place, reorganizing and updating the docs happen as a matter of course and it will happen much faster and more accurately than it can now.

I have a lot of ideas regarding Docs design, organization, content, and workflows, but here’s a first-pass at big-picture goals and a more granule task list as a potential place to start.

Ranked Big-picture Goals:
I. Establish Docs master schema to bring content into alignment across sections and articles.
II. Establish prescriptive content standards applied through automated validation.
III. Revise the backend CMS configuration to extend Open Authoring capabilities while maintaining content schema.
IV. Revise the frontend Docs UI to aid navigation and improve UX, including no-investment feedback options.

Possible Initial Task List

• [x] 1. Incorporate documentation linting as a standard test for all PR’s. Even if the linter’s output still requires human attention, it will at least begin providing metrics validating both the integrity of the Docs and conversely helping to ensure future content will align with previously established standards. Is there a linter that is specific to validating NetlifyCMS code that could be adapted to run on the Docs as well? PR Opened for this step: #4320
• [ ] 2. Tie documentation linting to automated Docs content updates, flagging automatic changes for manual review. Start small, such as updating syntax of fields, such as changing value_field to valueField.
• [ ] 3. Incorporate basic tracking data into articles, including automatically tagging articles with original submission date, date of last update, last validation date (and PR#?), history of last few authors (including bots), NetlifyCMS version compatibility, etc. Exposing some of this tracking data to the Docs end users should help drive confidence in the efficacy of the Docs and create a basis for better feedback when items can easily be verified to be out of date.
• [ ] 4. Increase user feedback by implementing no-investment feedback options. No sign-in article flagging, etc.
• [ ] 5. Improve the Open Authoring workflow by exposing more settings to non-technical users.

This article highlights many of the advantages I would expect to see from implementing documentation linting.

I am unfamiliar with implementing or customizing linters, and only just discovered they already exist for documentation (it’s always validating to find out someone has built and is using something you’ve only just dreamed up).
I would love to hear from more experienced contributors about the idea of overhauling the entire process of documentation and if documentation linting is the right place to start.

——————————————
A few more ideas that are related to the big picture goals I am suggesting.

Content Schema
In general, I’d ideally like to see the configuration of the Docs CMS much more abstracted, if not completely decoupled from the page layouts the users see.
It will always be important to see and edit articles in a WYSIWYG fashion, but perhaps each article is implemented via a content template with the individual fields correlated across articles. Consistency versus flexibility will be an interesting balance, but most documentation really can be broken down into a finite set of information modes allowing for a consistently implemented schema. In terms of the existing Docs, a good example of this cross-correlation is that many articles are already loosely following this structure:

• Concepts
• Tech Specs
• Tutorial
• Advanced Options

So instead of choosing between writing articles in a mode suitable for every user group or writing articles that leave out user groups... what if content was abstracted and (more) standardized so it could be compiled differently for each user groups (Designers, Engineers, Authors, Contributors, etc). Likewise, this information could be edited across articles more easily… for example, maybe I’m only interested in ensuring a consistent voice across all Concept sections. I could fire up the Concepts Glossary, and scroll through all the article headings at once with the preview pane showing the full context for any entry I'm actively editing.

UI Design Thoughts
Navigating the Docs is somewhat cumbersome on each of the front and back ends, and both could use UI changes to improve the UX. I’m trying to avoid extensive brainstorming about the CMS UI for the moment because improvements to the Docs Open Authoring experience might be better addressed by working through the Issues captured in the in-progress UX Project related to #2557... and a lot of the CMS UI might be dictated by whatever content schema is adopted.

Here are some specific UI/UX ideas for the frontend:
I visited docs.netlify.com for the first time today and realized I basically described that UI.
Is it possible to source/port and adopt the UI directly from docs.netlify.com? Restyled to match the NetlifyCMS branding of course.
This would remove a _lot_ of work from revamping the Docs frontend, and shouldn't cost Netlify.com anything to provide the existing design files, right?

Additional Context - Related Issues:

Docs Content & Org

598 - Build a more organized outline for docs

1038 - Improve contributing guidelines (OPEN)

1340 - Create a user guide for content editors (OPEN)

1632 - Add a style guide to documentation contributor info

1692 - Reorganize the docs with a strong focus on the user (OPEN)

2336 - Improve Contrib Guide and Create Style Guide

3764 - A proposal for reorganizing the NetlifyCMS Docs navigation (OPEN)

3766 - feat: reorganizing the NetlifyCMS Docs navigation

Structured Testing

1013 - Determine testing strategies

UX Design

2557 (OPEN Project)

[silverwebb925]

Having read through my initial post, I’d like to step back a moment and fully acknowledge the amazing work that’s gone into the Docs already, both from a conceptual and planning standpoint and in terms of the actual content describing NetlifyCMS. You've all done really great work, and I have SO much respect for the many contributors to such a useful reference.

Calling for an overhaul of the Docs infrastructure is intended not as a critique of your work, but is an effort to ensure that the Docs _continue_ to be the best documentation possible. (and hopefully with less manual labor)

Much of what I’ve outlined above directly relates to or is adjacent to ideas laid out in other issues, and I believe taking concrete steps to update the infrastructure of the Docs will also incorporate and promote many of the other documentation-related Issues already on the table.

As part of acknowledging others’ work on the Docs, I went through the Issues I cited (and any issues they cited), and have compiled a list of contributors already involved with the Docs. (the list is non-hierarchical and includes organizers and contributors with massive input alongside people with single posts simply expressing interest in helping with documentation). Apologies if I've missed anyone...

@verythorough
@erquhart
@tech4him1
@tomrutgers
@ademuro
@dopry
@hdoro
@Benaiah
@erezrokah
@talves
@jorear81
@phae
@JRPfeiffer1
@NaNdreas
@ragasirtahk
@eamonnbell
@loremipson
@cory-wc
@jerdog
@t1merickson
@owenhoskins
@ziburski
@kurtgcollins
@MimiDumpling
@nathankitchen
@jeremyzilar

Additionally, I noticed @jeddy3 and @lunaceee in a thread about structured testing and would appreciate any input you may have regarding documentation linting.

Lastly, @austincondiff, it seems like there may be significant overlap between improving the Docs and the UX improvements you’ve designed.

[silverwebb925]

In order to examine the idea of implementing a master schema, I went through and typed up the current organization of the docs down to the individual headers in each article (see attached). I used this to find patterns within the structure, as well as reading through the content itself and from this, I created a recursive content schema to act as a rough organizational framework for each level of information (Sections, Articles, Headers, and even Content paragraphs)

To see how this schema did in practice, I worked through the Concept section (Intro to Netlify CMS) and I'm pretty happy with the flow of the content after rearranging it to match the recursive schema. Most of the excerpts in the Recursive Outline are pulled directly from the existing documentation, though I did add some where there were gaps. One thing I like about the recursive schema is that it provides a pretty straightforward validation from the get-go (even if I did it all manually this time).

I only roughed in the rest of the sections in my working copy, but the schema appears to hold up and allows for more modular pieces of content.

Please note that the attached recursive outline is _not_ a representation of an actual menu nor a proposal to minutely number the Docs. It is a technical _org chart_ describing the structure of the content. Also, even IF this schema holds up, it is only a rough theoretical structure describing the general organization and developing a prescriptive system for content generation/validation will require significantly more detailed guidelines.

That said, this seems like a step in the right direction. Let me know if I can answer any questions about this schema, especially as it relates to the Big Picture Goals in my original post.

Thanks again!
20200909_Docs_Recursive Outline.pdf
20200909_Docs_Current_State.pdf

[erezrokah]

Thank you @silverwebb925 for doing such an extensive research. Great work!
What is the next step we should do (with emphasis on speed)?
Should we add formatting/linting to the docs? Even if the set of rules it not yet fully defined yet, that should be a quick win.
The tricky thing would be to make sure the linter is aligned with the way the CMS normalises content.

It would be great to have the documentation work as a part of the development workflow and have a schema enforced.

Obviously we would happily accept PRs for improving the docs.

[erezrokah]

2. such as changing value_field to valueField. (pretty sure the Relational Widget documentation is out of date.. its on my radar for a PR)

This is another issue we have - the docs represent the code on the default branch, and not the CMS version published to npm. Ideally we would have docs per version dropdown.

[silverwebb925]

You're welcome, @erezrokah!

What is the next step we should do (with emphasis on speed)?
Should we add formatting/linting to the docs? Even if the set of rules it not yet fully defined yet, that should be a quick win.
The tricky thing would be to make sure the linter is aligned with the way the CMS normalizes content.

The next crucial step is adding a linter for the Docs as soon as possible.

I'd like to push for integrating a Natural Language Processing (NLP) linter with the default NetlifyCMS content editor (and not just for the Docs development). Adding NLP for real-time feedback in the content editor would be a HUGE win for NetlifyCMS, since there are no CMS's that can lint style and usage as a built-in feature (that I've seen).

I would be happy to help develop both a default configuration as well as a Docs-specific module as soon as a linter is chosen and moved into production.

I highly recommend reading the attached paper and/or watching the related presentation to understand how revolutionary real-time prose linting can be as part of a content editor.
I had thought to suggest ProseLint itself after watching these, but the project seems almost abandoned on GitHub.

"Linting science prose and the science of prose linting", Pacer and Suchow 2016
20min video presentation Pacer

Prose Linters Research:
Vale
write good
Natural
textlint
ProseLint - seems like it started really strongly in 2016, but may now be abandoned? If so, it's too bad.

[erezrokah]

Having real time feedback in the editor would be awesome, however that might a bigger effort than liniting our docs (I think we would need to improve the editor itself first).
I think the first step would be to set up the infrastructure that will allow others to contribute to the docs and your initiative in a structured way.
Once we have a good dev/contribution pipeline we can incrementally increase our efforts, WDYT?

[silverwebb925]

I agree. Which contributors need to be tapped to make decisions about what linter(s) to use and how/where to insert them into the PR process?

[erezrokah]

How about raising this in our Slack https://netlifycms.slack.com/archives/CPRR9L929?

I'm also good with you submitting a PR with the change - that usually starts a discussion.