Logstash: Generate versioned plugin documentation

[dedemorton]

The Problem
Currently, the documentation for Logstash plugins reflects the version of each plugin that was bundled with the release. If the user updates a plugin after installing Logstash, it can be challenging for the user to find docs that match the specific version of the plugin that is installed. The docs in the master branch contain info about the latest version of the plugin, and the docs in the current branch contain info about the plugin version bundled with Logstash, but what if the user has a different (intermediary) version of a plugin installed? Short of looking at the Ruby source code in GitHub, there is no easy way to figure out what is supported.

We tried to mitigate this problem by adding version info to the published plugin docs, but that has resulted in more confusion and doesn't solve the main problem: users need to be able to access accurate documentation that reflects the version of the plugin that's installed.

Proposal
We need to provide versioned plugin documentation so the user can access content that accurately reflects the version of the plugin that they have installed.

To accomplish this goal, we need to:

• [x] Design the information architecture for navigating to/from the versioned content for each plugin. This might be a link that points to "Other versions" or possibly even a list of all available versions.
• [x] Determine the design and location of the versioned content:
• what the pages will look like
• where they will live
• how the user will navigate between different versions of the content (drop-down list? links in the TOC?)
• how the user will navigate back to the main Logstash doc, etc
• [x] Modify the docs infrastructure to build plugin-specific content. For every version of the plugin that's released, we should generate a new version of the plugin documentation and make it available on the docs website. We'll need to make the IDs in each file unique (possibly by including the version number) so that the book builds without errors.
• [ ] Automate the generation and publishing of the plugin documentation so that a new version of the docs is generated/published whenever a new version of a plugin is released. [moved to a #9048]

[dedemorton]

UPDATED 4/20 to use patch versions instead of minor versions.

One solution for navigation would be to provide multiple versions of the plugin docs in a single book called something like "Logstash Plugin Reference". So the Doc landing page would have something like:

`````
Logstash: Collect, Enrich, and Transport

Logstash Reference [5.2] — other versions 
Logstash Plugin Reference

`````
We'd still provide plugin docs in the Logstash Reference that reflect the version of the plugin that was available when Logstash shipped (more about that later). I probably need to put a bit more thought on helping users understand why the plugin docs appear in two places.

Let's say that users go to the doc landing page and click the Logstash Plugin Reference link, they will go to a book with a nav that looks something like this:
`````
Logstash Plugin Reference
This guide contains the documentation for Logstash plugins organized by plugin version.

Input plugins
beats
cloudwatch
couchdb_changes
...
Output plugins
boundary
circus
cloud watch
...
Filter plugins
aggregate
alter
anonymize
….
If they expand a plugin in the nav, they will see the list of plugin versions
beats
beats 3.1
beats 3.0
...
Let's say that `beats` in the nav links to a page called beats-docs-by-version.asciidoc. The contents of beats-docs-by-version.asciidoc would contain something like the following, with each version being an active link.
beats
To see which version of the plugin you have installed, run bin/logstash-plugin list --verbose.

Documentation by Version
beats 3.1 Nov 30, 2015
beats 3.0 Oct 15, 2015
`````

In the Logstash Reference, each plugin doc page would have an "Other Doc Versions" link that points to the overview page for that plugin. For example, the beats plugin docs in the Logstash Reference would have an "Other Doc Versions" link that points to beats-docs-by-version.asciidoc. This would work well in both the Logstash Reference, where the link would take you to the book with all versions, and within the Lostash Plugin Reference, where the link would just pop you up to the container page that has the list of all versions.

Each page would also probably need to have a link back to the Logstash Reference. Need to think more about that....

[dedemorton]

I've updated the description above to reflect our ongoing discussions about how this content should be structured.

[dedemorton]

One thing I didn't think about before: if we include all versions the docs for a specific plugin in the same book, we'll need to add something to the asciidoc source files so that we get unique IDs for each version. This is going to be a bit tricky because the links that reference the IDs will also need to be updated, and that's gonna get weird for the links between plugins, especially if we update the plugin docs individually. I'm also not sure how we'll deal with links that use the cross link syntax <<link>> to point to topics in the LS Reference. We'll need to do more thinking about this.... [added above]

[dedemorton]

@andrewvc How do we get this work added to the roadmap? Right now, there is no easy way for users to access the documentation for the specific version of the plugin that they have installed.

[dedemorton]

@jsvd Just an FYI that this is something I've been wanting to do for awhile. We are not providing our users with an optimal experience with the plugin docs. My hope is that once we have versioned plugin docs in place, we'll be able to say more about compatibility with core.

[dedemorton]

Closing because the versioned plugin reference is published! Created a separate issue for the remaining cleanup work: #9048