Quarkus: Improve generated configuration documentation (round 2)

[emmanuelbernard]

After https://github.com/quarkusio/quarkus/pull/3428 merge

This tracks further improvements requests.

• [ ] Make it possible to generate all properties in one file, a properties file per extension and properties per ConfigGroup (like general props, orm general, orm database, orm cache)
• [ ] Group properties per ConfigGroup and create separations (as header or full line in the table)
• [ ] try and merge Type and Default column to save horizontal space
• [ ] improve the logo lifecycle or fall back to text: it was not clear to me which was which. We could ask Red Hat's design team as well though). I did not see a legend description
• [ ] Consider adding the fist line of the description in the table, I feel it is a bit empty as it stands
• [ ] Rediscuss the Asciidoc vs JavaDoc conversation: how do people write the full multi paragraph description (with examples). I do not want to do it in normal Javadoc and I tend to disagree that the IDE support will be massively worse but we don't care as noone will use these classes in their IDE (user wise), it's about doc and application.properties.

Consider the following table structure

| property group |
| property name \n description |
| type | default | lifecycle |
| property name 2 \n description |
| type | default | lifecycle |

it would save on horizontal space but still be regular and parsable.
Maybe the full description could live in it but I don't know.

[emmanuelbernard]

CC @machi1990 @FroMage

[emmanuelbernard]

Let me know @machi1990 if you are willing to help for the second round.

[gsmet]

Make it possible to generate all properties in one file, a properties file per extension and properties per ConfigGroup (like general props, orm general, orm database, orm cache)

So for this one, I wasn't sure if we should do it automatically or manually including all of them.

At least, we should have a way to define a proper order of the extensions.

improve the logo lifecycle or fall back to text: it was not clear to me which was which. We could ask Red Hat's design team as well though). I did not see a legend description

Hmmm, the legend is at the top (see https://github.com/machi1990/quarkus-generated-extension-config-docs/blob/master/src/main/asciidoc/generated/quarkus-agroal.adoc). For now, it uses emoji so it's pure text. Note that it depends on the OS/browser.

Consider adding the fist line of the description in the table, I feel it is a bit empty as it stands

Yup, we had that in mind. We need to try to include things in the website first and see how much width we have. Also depends on the width of the other columns so we need to find a solution for the type.

Consider the following table structure

The issue is that it's very disturbing to have a variable height in a table. Some of the table will have it if keys are too long but as much as possible we should avoid it.

[emmanuelbernard]

The issue is that it's very disturbing to have a variable height in a table. Some of the table will have it if keys are too long but as much as possible we should avoid it.

If you only embed the first line of the descroption, I don't think the table will have variable heights. In fact this two lines per entry model is likely to reduce such encounters

[FroMage]

https://github.com/quarkusio/quarkus/pull/3809 has progress:

• Adjusted the column sizes
• Put the first javadoc sentence (not perfect yet in terms of length that can exceed the column width and end up on two lines, also considers i.e as a sentence end)
• Turn enum types into enum values
• Turn java.lang.Class<?> into class name
• Turn Java primitive wrapper types into primitives (nobody cares that they are Long or long)

See results:

Now, some remarks on the code that I saw, @machi1990:

• I added a FIXME because you appear to turn Optional<T> into T without checking that it's Optional (are there other cases?) Same for any type with two type arguments (that can't be right).
• It looks like we're generating docs in {root}/docs/src/main/asciidoc/generated for all extensions, but that can't be right for extensions outside our repo, right? We should probably default to not generate this file unless we're in the quarkus repo somehow, no?

@emmanuelbernard to be honest, the "don't use HTML, use only asciidoc" constraint is huge: I wasn't able to find how to add a "title" element to the icon, I wasn't able to find how to indent the first-sentence description or make it non-wrap.

Remaining todo that I noticed:

• For some enums, we get very long lines because there are lots of values, not sure how to solve this
• I saw values of type ElasticsearchVersion which turns out to be a MP-Config convention format because the type has an of method, so it's really a type String we should be documenting, possibly with a link to its javadoc (the ElasticsearchVersion.of method has javadoc, not sure if that's the norm). We can probably check if it's a known type and if not, if it has a static of method for that?
• We should put the enum values and class name type strings with special color or style to differentiate them from class names.
• I think we should drop javadoc for String types, I don't see what it buys us.

[emmanuelbernard]

@emmanuelbernard to be honest, the "don't use HTML, use only asciidoc" constraint is huge: I wasn't able to find how to add a "title" element to the icon, I wasn't able to find how to indent the first-sentence description or make it non-

I don’t understand. I am not asking for the template to be asciidoc. I am asking for the description source to be asciidoc (Taken en from the Javadoc)

[machi1990]

3809 has progress:

• Adjusted the column sizes
• Put the first javadoc sentence (not perfect yet in terms of length that can exceed the column width and end up on two lines, also considers i.e as a sentence end)
• Turn enum types into enum values
• Turn java.lang.Class<?> into class name
• Turn Java primitive wrapper types into primitives (nobody cares that they are Long or long)

👍 good work. I'd should be able check the PR tonight.

Now, some remarks on the code that I saw, @machi1990:

• I added a FIXME because you appear to turn Optional<T> into T without checking that it's Optional (are there other cases?) Same for any type with two type arguments (that can't be right).

Yes, there is only one case I could think of at the moment - List<T>.
So when it comes to generics, only Optional, List and Map are handle at the moment. I thought may be not skipping the type check could not hurt.

• It looks like we're generating docs in {root}/docs/src/main/asciidoc/generated for all extensions, but that can't be right for extensions outside our repo, right? We should probably default to not generate this file unless we're in the quarkus repo somehow, no?

We only generate this folder if we have found config items that are within are io.quarkus...... package.

[machi1990]

Let me know @machi1990 if you are willing to help for the second round.

I see that @FroMage has started working on it, but yeah sure I'll be able to help on this round too :-)

[FroMage]

We only generate this folder if we have found config items that are within are io.quarkus...... package.

I don't see that check, where is it?

[FroMage]

I don’t understand. I am not asking for the template to be asciidoc. I am asking for the description source to be asciidoc (Taken en from the Javadoc)

We're not talking about the same thing. I'm talking about the fact that we have to generate asciidoc without HTML in there. It makes for seriously hard formatting of the table.

[machi1990]

We only generate this folder if we have found config items that are within are io.quarkus...... package.

I don't see that check, where is it?

A config root is recorded only if its within a io.quarkus... package.
Here - https://github.com/quarkusio/quarkus/blob/master/core/processor/src/main/java/io/quarkus/annotation/processor/generate_doc/GenerateExtensionConfigurationDoc.java#L67-L79

If this list is empty, the generated config items will be empty upon arriving here - https://github.com/quarkusio/quarkus/blob/master/core/processor/src/main/java/io/quarkus/annotation/processor/generate_doc/GenerateExtensionConfigurationDoc.java#L147

[FroMage]

OK, thanks. But then it prevents other extensions from using our documentation generator. Well, we can get to that when someone asks for it.

[dmlloyd]

I would like to see us drop "type" and replace it with some kind of documentation derived from the converter in use. This would allow us to say things like "an integer between 0 and 65535, inclusive" or "any of the following values: xxx, yyy, zzz" and things like that, on an automated basis.

[dmlloyd]

Also we should probably state whether the property is required to have a value or not, now that this distinction is supported. With the next SmallRye Config update, optionality is implemented using a converter, so maybe that could be part of deriving value type from converter type.

[emmanuelbernard]

I see. I do have answer for some of your syntax questions. Hit me on Zulip
tomorrow.

On Tue 3 Sep 2019 at 10:09, Stéphane Épardaud [e-mail]
wrote:

I don’t understand. I am not asking for the template to be asciidoc. I am
asking for the description source to be asciidoc (Taken en from the Javadoc)

We're not talking about the same thing. I'm talking about the fact that we
have to generate asciidoc without HTML in there. It makes for seriously
hard formatting of the table.

—
You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub
https://github.com/quarkusio/quarkus/issues/3803?email_source=notifications&email_token=AACJNWCRVKM3TKFUYQ4AL6LQHYLTVA5CNFSM4IS35BG2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5XMJXI#issuecomment-527353053,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AACJNWHTM35YMIZX4UIUQC3QHYLTVANCNFSM4IS35BGQ
.

[machi1990]

• [ ] Consider adding the fist line of the description in the table, I feel it is a bit empty as it stands

addressed by https://github.com/quarkusio/quarkus/pull/3809 thanks to @FroMage

• [ ] Rediscuss the Asciidoc vs JavaDoc conversation: how do people write the full multi paragraph description (with examples). I do not want to do it in normal Javadoc and I tend to disagree that the IDE support will be massively worse but we don't care as noone will use these classes in their IDE (user wise), it's about doc and application.properties.

https://github.com/quarkusio/quarkus/pull/3887 will somehow address this with conditional asciidoc using the @asciidoclet block tag. Thanks to @gsmet

[machi1990]

• [ ] Make it possible to generate all properties in one file, a properties file per extension and properties per ConfigGroup (like general props, orm general, orm database, orm cache)

We have a work in progress for this TODO item here https://github.com/quarkusio/quarkus/pull/4050

[FroMage]

So the problem with generating a single page for all properties is that each extension only builds its own config documentation page. As a result there's no step where we run the doc generation for "all extensions".

We can fake this, by generating two files for each extension:

• $ext-config.adoc: contains the table title, headers and includes $ext-config-data.adoc
• $ext-config-data.adoc which contains only the rows

Then we hand-write a all-config.adoc file which has the title and table headers and includes every $ext-config-data.adoc file, thus placing each row one after the other.

The lines won't be alphabetically sorted, but they will be grouped by extensions. This will require adding each extension manually by adding an include.

It's this or we have to run the extension processor in the docs module, _somehow_, using aggregated extension data we store _somewhere_.

WDYT @emmanuelbernard @gsmet @machi1990 ?

[machi1990]

It's this or we have to run the extension processor in the docs module, _somehow_, using aggregated extension data we store _somewhere_.

I was thinking of doing this in the docs module, after all the extensions have been processed we have aggregate extension configuration data in a file called - target/asciidoc/generated/config/all-configuration-roots-generated-doc.properties.

[FroMage]

Ah, yes, that's also possible.

[FroMage]

So, currently https://github.com/quarkusio/quarkus/pull/4381 has a bunch of things, including the start of an "All config" page.

Except I don't have the name/description of the extensions in all-configuration-roots-generated-doc.properties so I can't have nice section titles for the extensions.

And search is slow, becuase the table is huge. I can probably make it faster, but we should open a new issue for this.

[FroMage]

OK, so the table is not "huge" by any Java measure: 570 rows, 166 of which are collapsible. BUT JavaScript DOM is indeed super slow, wich actions often taking longer than 2s.

I've removed jQuery usage to regular DOM to shave a few hundred ms, incorporated a delay when searching to not search three times when typing "abc", use a shadow DOM to work on modifications in an invisible tree copy (double-buffering, essentially).

Now the code is MUCH more verbose, but most actions take < 100ms. :man_shrugging: At least now it's usable.

[machi1990]

OK, so the table is not "huge" by any Java measure: 570 rows, 166 of which are collapsible. BUT JavaScript DOM is indeed super slow, wich actions often taking longer than 2s.

I've removed jQuery usage to regular DOM to shave a few hundred ms, incorporated a delay when searching to not search three times when typing "abc", use a shadow DOM to work on modifications in an invisible tree copy (double-buffering, essentially).

Now the code is MUCH more verbose, but most actions take < 100ms. 🤷‍♂ At least now it's usable.

Wooot 🔥 Is there a PR / branch somewhere so that I can take look also? @FroMage

[FroMage]

Not yet, I was hoping to be able to get the extension names first.

[FroMage]

Not sure I follow. I was wondering where to get the name of the extension as we don't have that info. I've used a workaround until we do.

Check out https://github.com/quarkusio/quarkus/pull/4522

[machi1990]

Not sure I follow. I was wondering where to get the name of the extension as we don't have that info. I've used a workaround until we do.

Check out #4522

Sorry my bad, I thought you wanted to get the extension names on the javascript's side. I only realised it was not the case (hence the deleted comment) after I connected to Zulip (was away the whole day).

Thanks for the PR, I'll check it out over the weekend.

[FroMage]

OK, I think I've got it with the extension names.

Except we're missing extension names for the following extensions (or sources of config):

• io.quarkus:quarkus-amazon-lambda-resteasy
• io.quarkus:quarkus-core
• io.quarkus:quarkus-elytron-security-jdbc
• io.quarkus:quarkus-infinispan-embedded
• io.quarkus:quarkus-jaeger
• io.quarkus:quarkus-security
• io.quarkus:quarkus-vertx-core
• io.quarkus:quarkus-vertx-http

@aloubyansky is this normal?

[FroMage]

Code at #4584

[aloubyansky]

It means they were missing from the extensions.json which was manually maintained. What I did was simply split it into pieces and put those pieces into the corresponding extensions. We need cover the missing ones of course.

[FroMage]

OK lemme add them, then

[FroMage]

I noticed we're documenting lists of values the same as their values, so string for a List<String>, which is confusing because it's really a comma-separated list.

I've changed it to (string)* but not sure if that's much clearer, WDYT?

I can document required keys, but how? I can add (required) next to their key, but, not sure.

[dmlloyd]

How about List of String? Syntactically this can be generalized to any generic or collection type.

[FroMage]

OK.

[FroMage]

For now I've put "REQUIRED" in the "default" column. Not sure this is the best way to visualise it, but it's better than nothing.

[FroMage]

Interestingly, on https://quarkus.io/guides/all-config we have an empty TOC which turns into an ugly blue bar floating at the right of the input search field :(

[gsmet]

@FroMage it probably shouldn't use the layout it uses.

[FroMage]

I'm not sure which layout it should use, and where I can override it.

This is the entire file:

= Quarkus - All configuration options

include::./attributes.adoc[]

include::{generated-dir}/config/all-config.adoc[opts=optional]

[gsmet]

The guides.html layout is the one used by default by the guides.

If you want another one, you need to define it with the appropriate header:

layout: guides-configuration-reference

Then create a _layouts/guides-configuration-reference.html similar to
_layouts/guides.html but without the TOC.

[FroMage]

@gsmet actually if I set the layout then I can't build the docs in the quarkus repo, because the layout is missing from there;

[ERROR] asciidoctor: ERROR: all-config.adoc: line 9: level 0 sections can only be used when doctype is book

[FroMage]

Ah I can ignore it with <skip-front-matter>true</skip-front-matter>

[FroMage]

FTR #4584 has been merged.

[gsmet]

I think we are good with this one now.