Kibana: [discuss] Future of Developer Documentation

[joshdover]

This issue was brought up about a year ago (warning: contents may be hot! 🔥). I'd like to propose a long-term plan for the future of Kibana developer documentation that incorporates many of the ideas discussed previously along with new concerns and things we've learned in the past year.

Goals

Before jumping into the details, I think it's important to state the goals of developer documentation. Docs should be...

1. Easy to find for both internal and external consumers
2. Easy to use and navigate
3. Easy to write and contribute to
4. Up-to-date and accurate

If we fail to meet any one of these goals sufficiently, it's reasonable to assume that our docs will go unused. Docs that go unused fall behind, wither and die quite quickly. Whatever solution we converge on should meet all 4 of these in order to be successful.

One litmus test we could use to determine if we're meeting these goals: are engineers on the Kibana team actually finding these useful?

Context

Broadly speaking, there appears to be 3 types of documentation that we have or need for Kibana:

1. User documentation (95% of what currently exists at elastic.co)
2. Plugin developer documentation
3. Kibana developer documentation

This proposal addresses (2) and (3) and does not propose any substantial changes to (1).

Plugin developer documentation

These docs should aid both internal and external developers in building full-fledged Kibana plugins, covering both the APIs and tools available, as well as best practices and cookbooks.

• What is a Kibana plugin? What can it do?
• Basic tutorials on how to create a new plugin
• Detailed documentation for Core TypeScript APIs
• Detailed documentation for Plugin TypeScript APIs (eg. data plugin)
• Detailed documentation for stable RESTful APIs (eg. Saved Objects)
• Cookbooks on how to do common customizations (custom viz, embeddables, etc.)
• Conventions broadly useful to plugin developers
• How to test plugins in general
• How to use Saved Objects & Migrations
• How to migrate to the New Platform

Kibana contributor documentation

These mostly serve to augment the plugin developer docs with information about contributing to our repository directly:

• Considerations for Core development (this is becoming less important as Core's API surface shrinks)
• Core's internal architecture
• How to get a pull request merged (dev process)
• How to run Kibana's tests
• How to contribute updates to Localization
• CLI options available in development

Where should dev docs live?

TL;DR: We should invest time into getting our docs to work with the existing elastic/docs system in order to publish on elastic.co/guide as a Asciidoc "book" separate from our user guides.

Beats has separated their docs into the reference guide and the developer guide and I believe we should do the same.

At a high level, I propose we:

• Add a new "Kibana Developer Guide" book that is separate from our existing Kibana Reference (user guides). This new book would contain both the Plugin and Core developer docs.
• Move all content from the Contributing to Kibana section to this new book.
• Create tooling to generate docs from code that output to compatible Asciidoc used to populate the Kibana Developer Guide (more on this later).
• Cross-link between the Kibana Reference and Kibana Developer Guide where appropriate to bridge the gap and overlap between these to books.
• Explore linking to richer outputs in the future (eg. OpenAPI)

The advantages of this approach:

• All Elastic docs continue to have a single home (✅ Goal 1)
• Having a separate book for dev docs gives us more room in the page hierarchy (✅ Goal 2)
• Docs "source code" can be near (or in) the actual code, should help keep docs fresh (✅ Goal 3, ✅ Goal 4)
• Asciidocs are easily explorable within Github (✅ Goal 2)
• We leverage all the existing work of elastic/docs
• Any improvements we make to elastic/docs can be shared across the entire company

Custom tooling

In order to make this dream a reality, we will need to invest in some custom tooling, though I don't expect it to be much.

TSDocs will need to be extracted and output to Asciidoc. While there is no existing output in the open source community, Microsoft's API Extractor tool provides examples and code we can use to do this. A rough glance at the source code points to this not being a large effort. Building our own tooling gives us a full control on the format and output of what we want these docs to look like. We should be able leverage the features of Asciidoc to make these generated docs as interactive and cross-linked similar to other tools.

We are also exploring adding OpenAPI specs to public RESTful APIs. If we adopt this, then there is already existing tooling in the community for generating Asciidoc for these specs. If needed, we could also fork this to gain more control. This is something the Elasticsearch team could leverage as well as they move to OpenAPI.

Freshness

One complaint about using the existing Asciidoc system is that dev docs are likely to get out of date quickly because developers forget to update the corresponding documentation files when updating code. I think there are 3 things we can do to fix this:

1. Generate API docs from code (as mentioned above)
2. Add a mechanism to import asciidoc files from outside the docs/ file tree into the book. For example, a high-level asciidoc file describing a plugin in the source code should be usable in the external documentation book.
3. Eliminate other documentation where practical. We currently have a "Contributing to Kibana" section in the user guide, a CONTRIBUTING.md file in the repo, MIGRATION.md guide in the repo, and internal Google Docs floating around. We should consolidate all of these into this new Asciidoc book.

Richer outputs

Many of the tools we're using (or planning to use) have rich documentation outputs (eg. OpenAPI) which we should leverage if they offer real benefits or features that we cannot implement or add easily to the existing Asciidoc system.

I don't have a grand plan here, but I don't see why we couldn't generate these outputs and link to them from the Asciidoc book, all as part of the regular elastic/docs pipeline.

I think this is something we explore in the future one we're in a good place with our existing docs. The reality is that we have a lot of work to do just to make ANY dev docs actually exist and usable. The lack of docs is a glaring hole in both our internal onboarding and external plugin dev community. Making any docs exist is more important than how rich the interface is.

Miscellaneous

• You'll notice that there is some overlap between these areas, and I believe this is unavoidable. For example, RESTful APIs are also useful to users who are want to script their usage of Kibana but not build actual plugins. We can solve these problems by cross-linking across documentation books.

Points for discussion

• Do these goals feel correct?
• Are there any fundamental flaws with this proposal to meet the state goals?
• Are there limitations to the elastic/docs tooling that I am unaware of that would make any of this impossible?

[elasticmachine]

Pinging @elastic/kibana-docs

[elasticmachine]

Pinging @elastic/kibana-platform

[elasticmachine]

Pinging @elastic/kibana-app-arch

[joshdover]

cc @epixa @cjcenizal @stacey-gammon @timroes @tylersmalley

[cjcenizal]

👏 Well done, Josh! The problem is well-defined and the solution is well-reasoned. I support this proposal.

Add a mechanism to import asciidoc files from outside the docs/ file tree into the book. For example, a high-level asciidoc file describing a plugin in the source code should be usable in the external documentation book.

If we can get this part working, I think we'll have a great DX. Or should I say WX (writer's experience).

[epixa]

+1

[lukeelmers]

This is great and solves an entire category of problems we are facing in terms of developer experience. (Are my answers in the Migration guide? The Contributing guide? Styleguide? elastic/docs? Random markdown files in the source? Some other internal Google Doc? etc)

It also feels like something that we could accomplish with a reasonable amount of effort since we aren't completely starting from scratch.

I'll be especially pumped if TSDocs extraction becomes a reality ❤️

[jinmu03]

I'd like to include "How to write functional test?" in Kibana contributor documentation
and "a sample plugin" to Plugin developer documentation

[stacey-gammon]

I think these are all great efforts, but I'd like to bring up an additional option that I feel addresses two main aspects.

• One aspect is accuracy and even though you mention that the source code being near the actual code can help with this (I agree), it still doesn't ensure it.

• The other aspect is that some developers can learn more from interacting with a thing, and seeing/exploring real, working examples, as opposed to just text and screenshots (screenshots also are a pain to keep up to date).

I have been working on an idea of living documentation inside Kibana. This is what I've done with embeddables and more recently, search services. I've received really good feedback from developers, using these embeddable examples to write their own. I also think the documentation and examples could be way better, but even just having it inside Kibana, tested so we always know it's working, seems to be really helpful to developers.

One downside with this model today is discoverability. Everyone comes to me to ask about how embeddables work, and I point them to yarn start --plugin-path test/plugin_functional/plugins/kbn_tp_embeddable_explorer. I think having a single "Living documentation" app, that is itself extensible, would help this (of course, still with a special flag, this wouldn't be anything included in a production build, but that flag could be --livingDocumentation). That could be the first place a developer goes to learn about some service or another.

I'm not sure how helpful this is, but I also expose the actual code snippets that are backing the examples, which guarantees that the code samples I'm showing are up to date and working. Someone updates the code, the code snippet is automatically updated.

I am bringing this up on this issue because I'm struggling with the thought of having yet another duplicate source of information in the textual parts of this code. For example, the data services entry:

I copied from the readme

Which made me think, well I could just paste the README's directly in there, maybe even using an EUI markdown widget. However the links likely wouldn't work then bc the links in github READMEs are relative, here they'd need to be absolute. I could probably find a workaround to add the prefix for any relative paths I suppose.

Another cool aspect I could implement in this "developer documentation mode" of Kibana -- having an action on embeddables that gives you the code snippet you would need to embed any given embeddable in your own application. @smith was just talking about how it took a lot of time to create a visualization to embed in a solution, and how much easier it was to use Kibana to build that visualization - all we need then is the "export code snippet" step that shows exactly what you'd need to paste into an app to render that embeddable.

So, my questions are:

• how could/should these two systems could work together? Which one links to which? Do they inter-link? We could even have a hosted Kibana with this plugin turned on we could link to, for any developer to view, without having to actually run Kibana themselves.
  
  
  
  • what kind of value do people find in the "in-kibana" living documentation? I'd actually love to hear from @nreese or @spong or @crob611 , @peteharverson , or anyone else who may have used the Embeddable Explorer as a reference (or is in the process). Personally I feel like working, tested, examples, are the best way to learn.
  
  

[stacey-gammon]

Developer audiences

I don't think we can break down our developer audience into just "Plugin developers" vs "Kibana developers". I think it's more fine grained:

• Developers who are developing a plugin,
  
  
  
  • who write code directly to our repo,
    
    
    
    
    
    • who don't work at Elastic.
    
    
    
    • who work at Elastic.
    
    
    
  
  
  • who write code to an external repo.
  
  
• Developers who write code that is not in a plugin (e.g. "core" or "operational" - always in the Kibana repo)
  
  
  
  • who don't work at Elastic.
  
  
  • who work at Elastic.
  
  

To make it easier to refer to these different audiences, I'm going to create shorter names for them:

| Name | Description |
|------|----|
| Internal plugin contributors | Developers who write code in a plugin, directly to our repo, who work at elastic |
| External plugin contributors | Developers who write code in a plugin, directly to our repo, who don't work at elastic |
| External plugin developers | Developers who write code in a plugin that is outside our repo |
| External core contributors | Developers who write code that is not in a plugin who don't work at Elastic|
| Internal core contributors | Developers who write code that is not in a plugin who work at Elastic|

• Contributor terminology is used when any developer is writing code directly to our repo.
• Internal vs External terminology is used to indicate whether the developer works at Elastic or not (it doesn't indicate where the code resides).

The questions we need clear answers to

In order to solve the goals Josh listed above, we need clear answers to the following four questions (which is more fine grained than "Where should dev docs live?"):

1. What language should this documentation be written in.
2. What language should this documentation end up as.
3. Where should this documentation be written?
4. Where do you go to learn about something?

What language should this documentation be written in?

What language should this documentation end up as?

Where should this documentation be written?

Where should you to go learn about something?

Linking sources of truth from different locations

What do we need to do to get to the above vision?

• Move content in kibana/docs/developer into kibana/dev_docs
  
  
  
  • More discoverable for folks browsing the git repo, clearer separation of end user facing docs and developer facing docs.
  
  
• Link to kibana/dev_docs from kibana/docs/index.asciidoc instead of kibana/docs/developer
• Link from kibana/dev_docs to asciidoc files for specific plugins
• Convert plugin documentation from markdown to asciidoc (like READMEs)
• Move autogenerated kibana/docs/development to kibana/api_docs
• Build tooling to convert auto generated API documentation from markdown to asciidoc

Long term, we can still use this setup to transition to an improved online system for our api documentation. Short term, we'll still have this content hosted online.

[stacey-gammon]

Condensed version of the above:

Summary

Where should I write information?

| Documentation description | language | location |
|---|---|---|
| Low level documentation for specific APIs, functions, or services | tsdoc | comments in the files, along side the code. |
| Tutorial-like documentation that includes code snippets, examples and recipes | typescript | Add tested code in kibana/examples to ensure code snippets are up-to-date |
| High level documentation for a specific plugins APIs, functions, or services | asciidoc | in the plugin directory. kibana/src/{plugin}/README.asciidoc or kibana/src/{plugin}/docs/MORE_INFO.asciidoc |
| High level documentation that goes across plugin boundaries. Examples: Embeddables vs Visualization extension points. Query languages across Kibana. KQL vs EQL. Kibana architecture overview | asciidoc | kibana/docs/developer/architecture |
| Process information related to contributing. Examples: how to use github, how to label issues, how to create a pull request. | asciidoc | kibana/docs/developer/contributing |
| Process information related to contributing and external developer. Examples: best practices | asciidoc | kibana/docs/developer/best_practices |
| Information only relevant to internal Kibana developers. Examples: team structure, onboarding, slack links | markdown | kibana-team |

Where do I find, or learn about this information?

If you work for Elastic, start in the kibana-team repo. If you are external, start with our hosted developer guide: https://www.elastic.co/guide/en/kibana/current/development.html

[cjcenizal]

@stacey-gammon Thanks for breaking this down! This looks great and makes sense to me. I have one suggestion. Can we present this information in a simpler way? In my experience, the simpler something seems, the easier it is to understand, and the more likely people are to use the information. For example, I'd suggest making these changes to simplify this info:

• I think we can remove the kibana-team row, since hopefully its purpose is made clear to the team from the repo's README and it's not a public repo.
• I'd consider moving the examples directory into kibana/dev_docs, so that all out-of-code documentation lives in one place.
• Group this information into two main buckets: in-source docs and out-of-source docs.
• Is "language" a useful dimension here? I assume that from looking at each location, I'd be able to infer the language I should use, so perhaps we don't need to surface when talking about where docs should live.

Roughly, here's what I'm thinking the result would look like. WDYT?

In-source docs

Inline comments

When documenting low-level implementation details and intentions, use inline comments alongside the code being documented. For example, specific functions, type definitions, and algorithms.

Root-level READMEs

High-level documentation for a plugin or its modules should go in a root-level README in the directory that encapsulates the plugin or module being documented.

Out-of-source docs

kibana/dev_docs/examples

These examples consist of tutorial-like documentation that includes working code snippets and recipes.

kibana/dev_docs/architecture

This is high-level documentation that crosses plugin boundaries. For example, embeddables vs visualization extension points, Kibana query languages such as KQL vs EQL, and a Kibana architecture overview.

kibana/dev_docs/contributing

This is process information related to contributing. For example, how to use GitHub, how to label issues, and how to create a pull request.

kibana/dev_docs/best_practices

This is process information related to contributing and external developers.

[peterschretlen]

@stacey-gammon ++, what I like about this is we organize our existing doc mechanisms into a consistent whole, and make it easy to navigate and understand. I think there is lot of mileage we can get from just doing that.

@cjcenizal agree on simplicity, I think with a few iterations we can get to "easy to navigate and understand"

I'd distill the above comments into. If you are writing, think about the level of abstraction and your audience, and choose one of these 4 places, from lowest to highest level:

1. code (tsdoc)
2. plugin or module (readme or docs subfolder)
3. product ( dev_docs, or examples )
4. team/internal elastic (team repo)

The underlying idea is the best place to store documentation is on the documented thing itself, or as close as possible. Each of these might have sub-locations and specific guidance in terms of doc language and process, and that is OK it lets us optimize for each location.

While I agree with @cjcenizal that maybe we don't mention the team repo in public docs, I think the team location should be top of mind. I'd argue that our most important audience at the moment are all the Kibana contributors at Elastic that aren't on the Kibana team, and it happens to be an internal audience. Knowing things like "who do I talk to" for example.

[stacey-gammon]

In source docs > Root level READMEs

I don't think we should over qualify this. Some plugin-level documentation is better to break up into multiple docs, rather than keep in a single README. Example: https://github.com/elastic/kibana/tree/master/src/plugins/embeddable/docs

Out-of-source docs > Examples

I don't think we should add examples inside dev_docs because those are actual working plugins. I'd consider them "in-source" docs.

I'd distill the above comments into....

I like this simple 4 list breakdown! Only one thing, 3., "product" makes me think of the user facing docs, which go in kibana/docs.

Is "language" a useful dimension here?

I actually do think it is important to call out the language and specify asciidoc. It will lead us towards consistency, remove a point of confusion, and it's necessary to create an online, searchable, holistic view from our online hosted documentation.

I think it's fine to get there incrementally, but specifying the end goal/vision, is helpful, imo. For example, I kept going back and forth on a PR I made converting CONTRIBUTING.md to asciidoc because I was worried we'd end up choosing Markdown as the language of choice and I'd have to undo all those changes. Having a clarifying statement, that this kind of documentation should be in asciidoc, will unblock me.

[stacey-gammon]

I'll leave this issue open till the end of the week for any more comments. If nothing big comes up, the plan forward is:

• Migrate contributing.md content into asciidoc, move it into our hosted developer guide, clean up the current content, and outline the directory structure. (https://github.com/elastic/kibana/pull/67764)
• Create kibana/dev_docs and move kibana/docs/developer content into it.
• Create a section that codifies our documentation process in our documentation.
• Create a link in the kibana-team repo to this documentation.

[lcawl]

TL;DR: We should invest time into getting our docs to work with the existing elastic/docs system in order to publish on elastic.co/guide as a Asciidoc "book" separate from our user guides.

I think from the latest comments in this issue a separate "book" is no longer the strategy. Just in case it is, however, please note that we're actively working on an extensive documentation redesign that moves away from our current book-focused product-specific docs. Instead, we're organizing content around our 3+1 solutions and then further grouping it based on the user-role and content type. We don't have any external links yet, but I'm happy to chat further if you have any concerns about how this affects the trajectory of the developer documentation.

[stacey-gammon]

I think, with the recommendations we are proposing, we could do either (a separate book, or inlined)?

Short term, I was just going to inline them, to keep the Developer Guide where it is hosted now, even if I change where it lives in the Kibana rep (e.g. inside docs/index.asciidoc, instead of include::developer/index.asciidoc[], it'd be include::../dev_docs/index.asciidoc[])

We may have to refactor the number of =s used for each title I suppose, if the guide was moved to a new top level book, vs sub section.

Does asciidoc have a way to use that as a variable? so you could do something like:

{dev_guide}= Contributing

And {dev_guide} could be either an empty string, if it's a top level book, or =, if it's a subsection in an existing book?

Maybe there are other concerns/issues though that I am not aware of!

[lcawl]

There are indeed cool ways of manipulating the nesting of pages without altering the pages themselves: leveloffsets. I think this is something we're hoping to move to across the library in general (i.e. controlling the navigation/nesting at the highest level of the tree, not down in the leaves/pages). I've been using these offsets lots in my prototypes, so I'm happy to help if you want further implementation assistance.

[joshdover]

As part of the initial move to kibana/dev_docs I'd really like to see us prioritizing creating a simple index of links to all the existing documentation, ideally for all plugins. I'm thinking a table like:

| Component Name | Short Description | README | Client Reference | Server Reference |
|----------------|---------------------------------------------------------|---------------------------------------|---------------------------------------------|---------------------------------------------|
| Core | Platform APIs exposed to plugins | README | Client API | Server API |
| data Plugin | APIs for index patterns, filter, timepicker, and search | README | Client API | Server API |

Ideally we include every plugin in this table, even if there is no documentation. I think this helps with discoverability of what's available to developers.

[stacey-gammon]

+1 @joshdover! I could see that going under the kibana/dev_docs/architecture section.

[stacey-gammon]

Slight change, the directory for dev docs will remain where it is now at:

docs/developer

instead of dev_docs. Rather, the read of the user facing content will eventually move to docs/user so we will still have that top level distinction.

Otherwise, the plan remains the same and the initial PR to head towards it has been merged. Will go ahead and close this!