Ex_doc: Admonition text blocks support

[milmazz]

What is an admonition text block?

From the AsciiDoc documentation an admonition:

There are certain statements that you may want to draw attention to by taking
them out of the content’s flow and labeling them with a priority. These are
called admonitions. It’s rendered style is determined by the assigned label
(i.e., value)

Also, probably some of you have read the blog post announcing Distillery
2.0, in that blog post Paul mentioned some documentation improvements in
Distillery, specifically:

The docs have been significantly revamped using MkDocs. They are much easier
to navigate now, are more readable by extracting tips and warnings into
asides/callouts, and are fully searchable as well!

Behind the scenes MkDocs uses Python-MarkDown, which parses MarkDown
and support _admonitions_ via an extension.

So, I think it would be really nice to have support for something like this in ExDoc.

About the syntax

The _Admonition_ extension from Python-MarkDown follows almost the same syntax
as in reStructuredText:

.. note:: This is a note admonition.
   This is the second line of the first paragraph.

   - The note contains all indented body elements
     following.
   - It includes this bullet list.

But, they use !!! instead of ..:

!!! note
    You should note that the title will be automatically capitalized.

I also have seen these _callouts_ when I have used AsciiDoc, but the AsciiDoc syntax depends on the use case.

For a single paragraph it is like this:

WARNING: Wolpertingers are known to nest in server racks.
Enter at your own risk.

In this case the label must be uppercase and immediately followed by a colon. But the syntax is a little more complex for a _block of text_:

[IMPORTANT]
.Feeding the Werewolves
====
While werewolves are hardy community members, keep in mind the following dietary concerns:

. They are allergic to cinnamon.
. More than two glasses of orange juice in 24 hours makes them howl in harmony with alarms and sirens.
. Celery makes them sad.
====

Probably we should ask the Earmark team if they want to support this feature there, but I want to know your opinion before proceeding .

[josevalim]

HI @milmazz,

I believe this could be useful but there are some things to keep in mind. First of all, this is not supported by Markdown nor Commonmark. So it means people may try using this on their README and it won't work.

I personally find the Python-Markdown syntax quite foreign. I like the asciidoc syntax though. The best here would probably to come up with a syntax that would be nicely formatted on markdown when undetected but it could be detected and improved if desired.

For example:

[WARNING] This is a warning text.

And:

> ### [WARNING] Do not do this!
>
> This is the warning text.

So if Earmark sees a blockquote starting with a header followed by [ADMONITION], then it can render it in special content. As we can see, those are rendered just fine by Github:

[WARNING] This is a warning text.

And:

[WARNING] Do not do this!

This is the warning text.

[OvermindDL1]

\

Github handles restructuredtext fine, it's an old format and handles about everything in existence, including the above. I'd personally love to see ex_doc support ReST as well and things like this would come for free, as well as a log of other things, including (even optionally) parsed blocks of code for tests with even specific 'header' code that can be executed but not shown. It is the general 'Code Documentation Format', sphinx uses it as it's primary format as do many many other systems. All without needing custom markdown extensions.

[josevalim]

Unfortunately, having the community change as a whole to another format is way to complicated, so a markdown extension is the most accessible way to go. :)

[OvermindDL1]

Oh not change at all! I mean have ex_doc support multiple formats. :-)

There is precedence for this, 'most' programming language documentation systems support multiple languages, ReST tends to be the most common Default one in those, but multiples are supported. Even using an external tool like the ever-popular Pandoc you could support a few dozen formats and output them all to a single format that ex_doc would then handle exclusively (say, ReST due to it's abilities for code handling).

Could specify the document format on the @doc line somehow, perhaps like:

@doc format: :rest
@doc ""

# or perhaps

@doc format: ReST, do: """
"""

It could be started as a new alternative library to ex_doc, but that's starting to split abilities. What if ex_doc just had the ability to have pluggable input formats that output a native ex_doc format for output, kind of like how it has pluggable outputs? The above format: ... keyword argument could specify what format plugin to use for this @doc block with a default settable in the mix.exs or config.exs or so.

Even this Admonition support could just be tested as a new input support type.

_me has a side thought of 'programming' exr files that are just ReST with code and test blocks that become the actual code and the rest is documentation, some other languages have such identical libraries, hmm_

But really, ReST is so superior to markdown for a documentation format, tons of built-in features designed for documentation use and it is entirely pluggable with new features, all while still remaining readable as raw source like markdown.

Although admonition text block's could be added to the markdown processor for now, when does it stop with adding new features? Would you be amenable for ex_doc having pluggable input formats? The pluggable input format would even be useful for @tmbb and his project that keeps documentation 'outside' source files. ^.^

[josevalim]

The new documentation chunk does support custom formats (they are per module though) so this will eventually become supported but it is not our focus right now.

[tmbb]

ReST [...] is the general 'Code Documentation Format', sphinx uses it as it's primary format as do many many other systems.

ReST has its own limitation, which are not trivial (particularly regarding inline styles). In that aspect, I think asciidoc might have more features, although asciidoc's syntax is much more complex (IRC, ReST is context-sensitive but relatively simple).

Github handles restructuredtext fine, it's an old format and handles about everything in existence, including the above.

It is old, handles everything and has a total of one supporting implementation, which happens to require a VM which is not the BEAM. That's not a showstopper, but it's quite a big argument against.

But really, ReST is so superior to markdown for a documentation format, tons of built-in features designed for documentation use and it is entirely pluggable with new features, all while still remaining readable as raw source like markdown.

Indeed, but that comes at a cost of requiring something like 4 passes over the documentation. And requires python, whihc AFAIK is the only supported implementation.

What if ex_doc just had the ability to have pluggable input formats that output a native ex_doc format for output, kind of like how it has pluggable outputs?

It already does... Since my experiments with alternative syntax highlighters, ExDoc can support a custom markdown implementation. The thing is... It doesn't use the fact that it's a markdown implementation anywhere. It just assumes that's something that takes up text and outputs HTML. This means you can create your own "markdown implementation", which is actually a program that converts any format into HTML.

I almost suggested at the time that references to Markdown should be replaced in ExDoc in favor of something like "documentation format" or something like that, but then I had other stuff to do, and let it pass.

me has a side thought of 'programming' exr files that are just ReST with code and test blocks that become the actual code and the rest is documentation, some other languages have such identical libraries, hmm

You have my Guaxinim library (unfortunately not compatible with Elixir 1.7 because of some unannounced compatibility breaks which break the Amnesia package, one of which I can't quite pinpoint) which support inverse literate programming. You write normal Elixir files and the comments are converted into markdown prose.

[OvermindDL1]

I think asciidoc might have more features, although asciidoc's syntax is much more complex (IRC, ReST is context-sensitive but relatively simple).

Hmm, I use both and I'm unsure about that, ReST has been able to do things asciidoc hasn't, but I've not see anything asciidoc can do that ReST can't, though that is in my own experience.

And yep, ReST is context-sensitive, one reason why plugins are so much easier to write for it.

It is old, handles everything and has a total of one supporting implementation, which happens to require a VM which is not the BEAM. That's not a showstopper, but it's quite a big argument against.

What does? ReST has quite a few implementations, pandoc's for example is all built in haskell and compiled into pandoc itself.

Indeed, but that comes at a cost of requiring something like 4 passes over the documentation. And requires python, whihc AFAIK is the only supported implementation.

What about python? o.O?

It just assumes that's something that takes up text and outputs HTML.

Hence why I think it needs some intermediary format, something docbook-like but tuples/lists instead of xml perhaps.

This means you can create your own "markdown implementation", which is actually a program that converts any format into HTML.

What about converting to other formats, like epub, or pdf, or man pages, or docbook for integration into something larger, or to LaTeX, or to JSON for parsing in detail, all of which something like pandoc supports the output of.

Not saying to use Pandoc, but once you start adding non-standard extensions like admonitions into Markdown it becomes much harder to make use of such tools compared to using a language that already supports the features you need.

I wouldn't be opposed to asciidoc though, it's not bad, I use it as a templating language for the HTML on my maven servers for the users.

I almost suggested at the time that references to Markdown should be replaced in ExDoc in favor of something like "documentation format" or something like that, but then I had other stuff to do, and let it pass.

+1

You have my Guaxinim library (unfortunately not compatible with Elixir 1.7 because of some unannounced compatibility breaks which break the Amnesia package, one of which I can't quite pinpoint) which support inverse literate programming. You write normal Elixir files and the comments are converted into markdown prose.

Yeah your project is why that is on my mind again. ^.^

[josevalim]

Folks, the discussion has been completely side-tracked. Can we please move

the discussion about other formats elsewhere?

José Valimwww.plataformatec.com.br
http://www.plataformatec.com.br/Founder and Director of R&D

[eksperimental]

It gives me the creeps to start adding features that are only going to be supported by only one parser.

I agree with @josevalim 's comment.

EDIT: But the idea of these text blocks is good. I like it.

[josevalim]

@RobertDober is this something you would be interested in adding to Earmark? Is it doable? Or should we try to implement it as a pass or using some other extension point? Thank you for your work!

[RobertDober]

A first thought that springs into mind is to just use an IAL

> [WARNING] Do not do this!
>
> This is the warning text.
{:.admonition}

however who is going to provide the stylesheet, ex_doc? (pushing the work back to you guys 😉 )

A second thought is to write a plugin, here is (the only) plugin, as an example.

https://www.hex.pm/packages/earmark_tag_cloud
If this does not cut it I'll be happy to consider alternative approaches, but I am against emitting css in Earmark.

[josevalim]

@RobertDober the CSS should totally be up to us, we just need a way to identify those with CSS rules. So if the blockquote has any kind of annotation, we can address it.

[RobertDober]

so either you use IAL, or we transform

> ### [WARNING] Do not do this!
>
> This is the warning text.

into

<blockquote class="admoniton"><p>WARNING...

of course the first p could get helpful attributes too.

I am quite sure that @pragdave would not object to implementing this, nor would YHS :wink:

EDIT: I just do not know Markdown 🤣

[josevalim]

@RobertDober could it be <blockquote class="warning"> or <blockquote class="admoniton warning">? Also, I probably wouldn't change the text because that would violate the markdown standard somehow, so maybe <blockquote class="warning"><p>[WARNING]?

[pragdave]

I like the concept, but I'm not sure about the syntax. [xxx] is already related to links of various sorts, so having it as a special case seems not-quite-right.

Could we just use an IAL?

~~~

Do not do this! {.WARNING}

This is the warning text.
~~~

Github renders this as the fairly reasonable looking

Do not do this! {.WARNING}

This is the warning text.

The exdoc CSS could handle it as

~ css
blockquote:has(h3.WARNING) {
lots of fancy layout
}
~

Earmark would by then have stripped the IAL from the text, so you'd just see the heading and the rest of the body of the admonition.

[RobertDober]

All ideas are ok for me, of course @pragdave's is a NOP for Earmark and probably much better than my original idea of assigning an IAL to the whole blockquote.

I'll happily implement something in Earmark itself but please be aware that only using an IAL will be 100% backwards compatible, so whatever we might implement in Earmark, it should be enabled only by an option, say %Earmark.Options{admonition: true}.

[RobertDober]

@josevalim do you mean apart of the aforementioned IAL?

If not, this should be what you are looking for

iex(40)> x="*foo* {:title=foo}"
"*foo* {:title=foo}"
iex(41)> Earmark.as_html(x)
{:ok, "<p><em title=\"foo\">foo</em></p>\n", []}

[josevalim]

Perfect @RobertDober, thanks!

@milmazz would you like to give it a try? We should be able to fully achieve it with CSS, then we just need to add docs.

We can do:

### Do not try this at home {.WARNING}
### Do not try this at home {.ERROR}
### Do not try this at home {.INFO}

And similar for blockquotes:

> ### Do not try this at home {.WARNING}
>
> The examples here are done by professionals.

[RobertDober]

that would be

> ### Do not try this at home {:.WARNING}
>
> The examples here are done by professionals.

N.B. the colon in {:.WARNING}

[josevalim]

Ok, I have added this as design because the work is mostly around the layout and providing good CSS classes.

[milmazz]

@josevalim Sure, I'm not an expert on CSS, but I can give it a try in a few hours.

-------- Original Message --------
On Mar 18, 2019, 14:12, José Valim wrote:

Perfect @RobertDober, thanks!

@milmazz would you like to give it a try? We should be able to fully achieve it with CSS, then we just need to add docs.

We can do:

Do not try this at home {.WARNING}

Do not try this at home {.ERROR}

Do not try this at home {.INFO}

And similar for blockquotes:

Do not try this at home {.WARNING}

The examples here are done by professionals.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

[milmazz]

@josevalim you can find a prototype of this feature here, this is assuming that the following input in MarkDown:

> ### Do not do this! {.WARNING}
>
> This is the warning text.

will produce the following HTML:

  <blockquote>
    <h3 class="warning">Do not do this!</h3>
    <p>This is the warning text.</p>
  </blockquote>

The :has() CSS pseudo-class sadly cannot be used right now as @pragdave proposed, is an experimental feature that is not supported by any browser today, tomorrow I'll try to use another approach, but I'm open to suggestions. /cc @RobertDober

[RobertDober]

Let us try the IAL for the blockquote then

> ### Never open this door!
> Here lie dragons
> Here dragons speak the truth!
{:.warning}

That will give us

<blockquote class="warning"><h3>Never open this door!</h3>
<p>Here lie dragons
Here dragons speak the truth!</p>
</blockquote>

Which could be styled like

    blockquote.warning { ... }
    blockquote.warning > h1 { ... }
    blockquote.warning > h2 { ... }
    blockquote.warning > h3 { ... }
    /* etc. etc. */

[josevalim]

@milmazz in my opinion the example you posted is excellent. We will just need to change how blockquotes look overall in order to support it.

[RobertDober]

@josevalim unless I am misstaken, and I easily could be, if :has(...) is not supported there is no way to style the parent of our <h*> element 😢

That motivated my suggestion above, if any CSS guru can prove me wrong that would be great though.

Changing blockquotes overall, would mean that nobody can use them for other purposes anymore, not even talking about backwards incomps.

[josevalim]

@RobertDober my proposal is to have the blockquote CSS be general enough that it work with and without the special headers.

[RobertDober]

@josevalim are we in agreement that this approach only works iff

(i) we can select the parent of the <h*> tag in the CSS
or
(ii) we add the class to the <blockquote> element

?

Sorry for being stupid, but it is not my fault ;).

[josevalim]

If you look at the examples @milmazz provided here https://codepen.io/milmazz/pen/wZMrEB, He was able to provide nice styling without any of those. Unless I am missing something too!

[RobertDober]

Hopefully I am wrong, but IIANM he styles all blockquotes, correct @milmazz ?

That said it is most beautiful.

[josevalim]

Yes, he does. Hence my proposal to style all blockquotes in generic way that
supports both with and without admonition. :)

[RobertDober]

Ah ok if you are happy with this, why not, sorry that I assumed otherwise.

[josevalim]

@milmazz btw, let's ship with three modes right now: warning, error and info. Regarding the icons, let's pick them from IcoMoon, which we already we use: https://icomoon.io/app/#/select

Let me know your preferences and I will generate a new custom font and include it in the repo.

[josevalim]

Here is a suggestion on how to style the generic blockquote and also support admonition on top of what @milmazz did: https://codepen.io/anon/pen/VNeBrZ - not the most amazing thing in the world but OK?

[milmazz]

@josevalim Great! I'll create the PR tonight with those 3 modes and also using the suggested font.

@RobertDober Yeah, based on José's feedback this will be a generic style for the blockquote element, so, there's no need of using weird CSS pseudo-class here.

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Thursday, April 4, 2019 3:31 AM, José Valim [e-mail] wrote:

@milmazz btw, let's ship with three modes right now: warning, error and info. Regarding the icons, let's pick them from IcoMoon, which we already we use: https://icomoon.io/app/#/select

Let me know your preferences and I will generate a new custom font and include it in the repo.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.