Lokomotive: Refactor config reference docs

[johananl]

The configuration reference docs currently contain lots of extra information and are missing some other information.

IMO, the critical information a config reference should describe is the configuration syntax and a list of all the knobs with an explanation about each one of them.

Looking at this for example, we should probably change the following:

• Remove the "Introduction" and "Prerequisites" sections. These are useful in a tutorial but have no real meaning in a reference guide AFAICT (e.g. prerequisites for what? For reading the document?).
• Add a section explaining the directory structure expected by lokoctl. For example, if we allow reading multiple .lokocfg files from the CWD, we should explicitly note it.
• Change the "Configuration" section to "Configuration Syntax" and explain the syntax. This means explaining the high-levle structure of a .lokocfg file and its various sections.
• Clean up the config sample. There is no value in including a variable block for each and every config knob. Rather, we should demonstrate the use of HCL variables in the "Configuration Syntax" section once, and then keep things simple in the rest of the document. Also, we should remove all the extra whitespace in the sample.
• Cover all of the config knobs. For example, right now there is no information regarding the backend block in the reference.
• Break down the attribute reference table. I find the section.attribute syntax confusing. I'd rather have a section for each config area. For example, worker_pool knobs should be described in their own table.
• Remove the "Applying" and "Destroying" sections. These don't belong in a config reference.

[invidian]

Break down the attribute reference table. I find the section.attribute syntax confusing. I'd rather have a section for each config area. For example, worker_pool knobs should be described in their own table.

I disagree with that. Helm charts README.md use this "dot" syntax and it works really well, especially for deeply nested structs. IMO way better than other kind of nesting. Example: https://github.com/helm/charts/tree/master/stable/prometheus-operator#prometheus-operator-1

Cover all of the config knobs. For example, right now there is no information regarding the backend block in the reference.

We do have this: https://github.com/kinvolk/lokomotive/blob/master/docs/configuration-reference/backend/local.md

• Add a section explaining the directory structure expected by .lokoctl. For example, if we allow reading multiple .lokocfg files from the CWD, we should explicitly note it.
  
  
  
  • Change the "Configuration" section to "Configuration Syntax" and explain the syntax. This means explaining the high-levle structure of a .lokocfg file and its various sections.
  
  
  • Clean up the config sample. There is no value in including a variable block for each and every config knob. Rather, we should demonstrate the use of HCL variables in the "Configuration Syntax" section once, and then keep things simple in the rest of the document. Also, we should remove all the extra whitespace in the sample.
  
  

I think for these, we could have a separate document, maybe in docs/concepts explaining the syntax we support?

[johananl]

I disagree with that. Helm charts README.md use this "dot" syntax and it works really well, especially for deeply nested structs. IMO way better than other kind of nesting. Example: https://github.com/helm/charts/tree/master/stable/prometheus-operator#prometheus-operator-1

Fine with me, as long as it's clear. I've looked at how this is done in Helm and I couldn't find a place which explains that a.b refers to "b as a member of a". I'd rather not leave this implicit.

We do have this: https://github.com/kinvolk/lokomotive/blob/master/docs/configuration-reference/backend/local.md

Great, then we should link to it. Or better yet - remove the backend block from the platform reference docs since it's covered in its ownd doc and isn't required for explaining e.g. cluster 'aws' {}. In any case, a reference guide should be, well... a reference. That means we should explain all the knobs we show. Think about this as if you were a user: you've come to this document to find out what a knob does.

I think for these, we could have a separate document, maybe in docs/concepts explaining the syntax we support?

What do you mean by "these"? The config syntax should definitely be in the reference guide. The syntax is inseparable from the knobs and their values.

[invidian]

What do you mean by "these"? The config syntax should definitely be in the reference guide. The syntax is inseparable from the knobs and their values.

I mean all aspects related purely to configuration syntax, so variables, functions and probably supported file extensions etc. This is all related to HCL configuration.

The syntax is inseparable from the knobs and their values.

For me, knobs are the specific content of the configuration. If we're going to describe the syntax in the same place where we describe the knobs, it will be duplicated content (as knobs are logically grouped, per platform, per component etc.).

[johananl]

As a user, a knob isn't of much use to me if I don't know where in the text file to put it. I don't see why we should be sending users to other docs just to understand how to write a cluster config file. The way I see it, a config reference should be one document which exhaustively describes everything there is to know about the configuration. If some information isn't included in it, it's no longer a reference.

Anyway @invidian, since it looks like you have a concrete idea in mind on how to address this - please feel free to open a PR.

[johananl]

Another thought: using tables for the knob descriptions might not be the best format. This leaves very little space for describing what a knob does, mentioning special cases, warning about pitfalls etc.

Some concrete examples from the AWS reference:

• The description of the controller_clc_snippets is currently "Controller Flatcar Container Linux Config snippets.". IMO the description gives almost the same information as the name of the knob. In order for the user to understand what this knob does, how to use it etc., I think we need a much longer and detailed description, which won't fit in a table.
• The description of the cluster_name knob contains a warning to the user regarding uniqueness. IMO we want to be able to write more detailed warnings and make them more visible. I think we can't do that when we have to squeeze the warning in the little description table cell.

Here is an example of a format which IMO is closer to what we are aiming for.

[invidian]

Calico use tables, which also looks fine to me https://docs.projectcalico.org/reference/resources/globalnetworkpolicy. But I'm fine with either format.

[johananl]

Right, but then they add notes at the bottom of the table which makes them out of context and harder to follow IMO. Example: https://docs.projectcalico.org/reference/resources/globalnetworkpolicy#entityrule

[johananl]

Another thought: maybe a paragraph-based format would be better for our config reference docs. Example: https://rancher.com/docs/rke/latest/en/config-options/

Rationale:

• It gives enough visual room for writing a full-text explanation of each knob.
• It allows adding NOTE sections and code snippets.
• It allows listing and explaining all valid values for a knob.
• It allows providing concrete values as sample values in context rather than in a "centralized" sample config snippet.

See https://github.com/kinvolk/lokomotive/pull/651#discussion_r443507149.

[marga-kinvolk]

I'm not sure how the rancher example differs from the envoyproxy one, but in any case, I'm very much in favor of dropping the table format. That table format is really hard to read, if we wanted for it to render correctly, we need to tweak it a lot. And being able to put warnings and explanations makes a lot of sense.

Apart from this, +1 to everything in the issue description. I found the config reference pretty hard to use and all those changes would make a significant improvement.

[alban]

Another thing to fix in the doc:

The configuration examples for each platform are duplicated and are out of sync. Example of packet:

• docs/configuration-reference/platforms/packet.md
• examples/packet-production/cluster.lokocfg

We should have one source. Maybe generate one from the the other. Or at least have the CI complain when the two sources are out of sync.

[johananl]

IMO config examples should not be a reference. Rather, config examples should provide "recipes" for common tasks (e.g. "deploy a minimal dev cluster on AWS", "deploy a production-ready cluster on Packet" etc.).

I agree that we should have one source for information which appears in multiple places. In the case of config reference vs. config examples, however, I don't think it's the same info because config references should be exhaustive whereas config examples should not.

I think we should have a very clear distinction between the "roles" of our various docs and config examples:

• Config references are exhaustive. They cover all knobs and explain everything about each knob.
• Quickstarts are quick. They show the user how to get up and running, and nothing but that.
• Config examples are useful as-is. They can be copy-pasted to get some specific, well-defined result.
• Howto guides show the user how to do a specific thing with minimum hassle.

Following that, for example, we should never try to cover all knobs in a config example, or in a quickstart. Similarly, a config reference should never include a "requirements" section because it's not a guide.