Julia: Rendering of documentation in terminal could perhaps be better

[KristofferC]

For reference, below is a comparison between the documentation of a function as rendered in the terminal and in the documentation:

There are a few things I think are non ideal in the terminal output:

• There is no separation between different methods. Right now, it is hard to see where the doctest for one method end and the definition of the new method starts.
• The Example header is too prominent. We use single # for #Examples which gives this massive header. Only bold should be enough (as the HTML shows it). Perhaps changing either to just using **Examples** or ###### Examples to achieve this?
• The Example header is indented. I don't really see a reason for this, left alignment would look better to me.
• Final bonus, syntax highlighted doctests would be pretty sweet!!!

[KristofferC]

Just trying stuff out (playing with Highlights.jl):

[tkelman]

Cool. We should try to adjust the docsystem API's so it's more extensible and Markdown doesn't have to live in Base for much longer, and other packages can do more interesting things.

[StefanKarpinski]

Maybe just print headers as # Examples – i.e. bold with the correct number of # prefixes?

[StefanKarpinski]

U+bb would be a nice character for this too:

» Header 1
»» Header 2
»»» Header 3
»»»» Header 4
»»»»» Header 5

Should be fairly broadly supported since it's a very low code point.

[KristofferC]

I think just changing Examples to be standard bold is easier?

[StefanKarpinski]

Sure, but it seems like there should be a distinction between different levels of header, no?

[KristofferC]

Where? There is a distinction between headers, they use different underline. As I said we could use ##### Examples.

[StefanKarpinski]

That seems ugly. Am I missing something – why would we want to do that?

[KristofferC]

Maybe I'm not being clear...

Right now we use one # which means we get the top one. Ways to get less emphasized headers is by using more # or by just saying that it should be bold with **. The html documentation seems to ignore the number of # in docstrings and just puts <strong> for it no matter what.
My suggestion is just making the Example header not be a header and just print it in bold. This would look like:

[KristofferC]

Also other headers in the default documentation for examples uses bold:

[StefanKarpinski]

Perhaps, but I like that we use the header semantics correctly. The hard part is that some documents use all six levels of header and really need the top one to be A HEADER, but other documents only use a single level of header and really would be better off with just a subtle indication of headerness. Perhaps when outputting a markdown document, we should take the overall depth of headers into account and start at the lowest level and work our way up to more and more intense header levels as they are needed. Would require pre-scanning the document, but perhaps this could be done when the document is constructed – i.e. each section would have a header depth field so you know how bold to make your headers.

That dovetails nicely with an idea I had previously, which was that if you insert a chunk of markdown into a larger document, you really want the header levels in the chunk to be relative to the context into which its inserted. I.e. a level-one header in a chunk inserted into a book at a level-two section should become a level-three header under that section.

[KristofferC]

Perhaps, but I like that we use the header semantics correctly.

I don't know what the header semantics are, why shouldn't the markdown here be generated with the "largest" header when a # is specified? Because there should be some context saying that for this markdown block (the docstring) level 1 headers should actually start at a small size?

[fredrikekre]

xref #19914

[StefanKarpinski]

My point is that *Examples* on a line by itself doesn't semantically indicate that it's starting a section whereas # Examples does precisely that – we just don't like how it's being rendered in this case. My proposal was to fix the rendering issue without throwing all semantically meaning markup out with the bathwater. Using *Examples* is exactly like replacing <h1>Examples</h1> with <div><b>Examples<b></div> because we don't like how the h1 element is being rendered. That's some 1999 shit right there. Today everyone accepts that the right way to change this is to use CSS styling so that the h1 element renders as desired in the given context.

In this case, if there's only one level of header in some Markdown fragment, especially if we're showing it in the terminal, we typically want to output it only using the _least_ prominent header style instead of the most prominent one. If the fragment actually had six levels of header, then we'd want to use all six styles. If it was being inserted into another document, on the other hand, we'd want to use the next level of prominence, respecting the structure of the document. So the rule should probably be something like this:

1. When rendering a Markdown fragment with N levels of header by itself, use the N least prominent header styles for rendering headers.
2. When inserting a Markdown fragment with N levels of header into a Markdown document into a section that's already M levels of header deep, render the headers with prominence styles M+1 ... M+N.

This idea requires distinguishing Markdown fragments from Markdown documents, which I'm not sure we do, but that could be a distinction made at render time instead of in the data structure itself.

[KristofferC]

To experiment a bit, OhMyREPL (master) now does syntax highlighting of docstrings with the selected colorscheme:

The inline code blocks look a bit out of place now though...

[KristofferC]

I looked a bit into how we could better visually separate multiple docstrings.

This is how it looks right now:

We could either use a vertical separator:

Or a horizontal one:

Opinions?

[mortenpi]

I would vote for the vertical bar, but either way it is a nice improvement!

[StefanKarpinski]

How about the same kind of left-side [ braketting as we're using for logging messages now?

[KristofferC]

Yes, I tried that as well:

It would be good if there existed a L box character that has the horizontal part at the bottom / top. Right now, the box doesn't feel really enclosing.

[nalimilan]

It would be good if there existed a L box character that has the horizontal part at the bottom / top. Right now, the box doesn't feel really enclosing.

Not sure what you mean exactly, but maybe it would look good if the horizontal line was a few characters wide, on the line before the text starts?

[KristofferC]

You mean adding a blank line on the top and bottom like:

┌
│ blabla
│ blabla
└

?

[nalimilan]

I mean that if you put the horizontal line on its own line you can make it wider, e.g.:

┌──
│ blabla
│ blabla

[StefanKarpinski]

That does give some more visual separation without being quite as heavy as the full hline.

[KristofferC]

So here is an example of that:

Opinions?

[StefanKarpinski]

What about no horizontal line at the bottom so it's just the border on the top and left of each section? Example:

┌──
│ Here is one set of methods
│ And some stuff about them

┌──
│ Here is another set of methods
│ And a great explanation of what they do

Not sure if the left border should extend below the text for its section or not.

[vtjnash]

I was going to suggest (or at least demonstrate for comparison) a similar option to Stefan:

┌── HEADER
│ Here is one set of methods
│ And some stuff about them
│
├── HEADER
│ Here is another set of methods
│ And a great explanation of what they do
│
└── 

[KristofferC]

Hmm, to me, that would look like the second HEADER is a subheader to the first one. I think it is better to completely "reset" between methods.

[KristofferC]

Think this is pretty ok now.

[stevengj]

Note that by using a header-1 for what is logically a subheading we also get odd rendering in IJulia, because the notebook uses <h1> for this: JuliaLang/IJulia.jl#766

Can't we at least use ## Examples ?

[KristofferC]

This was quite extensively discussed https://github.com/JuliaLang/julia/issues/22870#issuecomment-316484339 and following. I suggested just using bold to fix this straight away instead of a header at all, but, in my opinion, some overly complex solution was suggested instead and two years later we are still left with sucky example headers.

[StefanKarpinski]

I still like my "overly complex solution" fwiw 😝. When you splice a bit of markdown into another bit of markdown, it seems that the hierarchy of the text should be respected, i.e. an h1 in the subtext spliced into an h2 block should become an h3 in the combined text.

[KristofferC]

See you in two years ;)