Jetpack: Feature: Markdown support as a Markdown-specific Gutenberg Block

[nuzzio]

The reasoning behind this feature addition is to add the ability to save and edit Markdown in posts when using Gutenberg. This would restore the spirit of the classic editor Markdown implementation.

Additional reasons are:

By @MichaelArestad

There are times (like in p2s) where it’s much more complex to go back and edit a post than to initially create it. This is because p2 removes the markdown formatting replacing it with html. I think there is value in preserving the original content. It would also allow one to compose elsewhere and just paste in valid markdown.

By @mtias

People who prefer to write markdown want to retain the text as markdown.

Implementation Approach:

__First Iteration__
Implement a Markdown block as a simple field with no preview functionality.

__Second Iteration__
Implement a live preview functionality like in Calypso.

__Third Iteration__
Optimize UX by implementing formatting buttons which can wrap text elements in the chosen markup, i.e. bold to bold

I believe this one will require some discussion of the interactions with the toolbar since some of them may be unexpected in current patterns e.g.

1) Click to bold > bold stays selected
vs
2) Click to bold > bold Markdown bounds selection and bold toolbar button remains unselected

Those considerations are further down the road but worth noting.

Tasks for First Iteration

Sub-tasks to be added as specific changes are determined

• [x] Create a markdown-specific block that preserves markdown content.

TBD

• [x] Convert the Markdown to HTML on the server at the time of rendering.

[jeherve]

Related: #7786

[nuzzio]

Added mockups for each iteration in main issue.

[nb]

What do you think about exposing basic formatting in edit mode without the need to preview. Maybe only for headings, bold, italic, links, code. No need to do it for more complex constructs like tables. Here is what the in-editor formatting could look like:

# Example

## Sub-example

Hello, **surrounded by double asterisks* text is bold, __by underscores__ or **single asterisk* is italic, `code in backquotes` is monospace.

If we decide to include it, how does it affect your current iteration plan?

[nb]

One more question – why converting to markdown on the server and not on the client? What are the trade-offs?

[MichaelArestad]

What do you think about exposing basic formatting in edit mode without the need to preview.

@nb That is something I was thinking about. While I don't think we should highlight them as a preview, I do think we should highlight them to make it clear what is being formatted. For example, Ulysses doesn't show a preview, but does highlight it in a way that is preview-ish. This way, the user gets a good idea of how their markdown is working. We also wouldn't be jumping between font sizes. I do think we'll eventually want a full preview that hides the markdown characters and renders it the way the rest of the standard blocks look. Here's a screenshot from Ulysses:

[nuzzio]

@nb

What do you think about exposing basic formatting without the need to preview. Maybe only for headings, bold, italic, links, code. No need to do it for more complex constructs like tables.

I think this is an interesting idea. My initial thoughts are:

• This is an enhancement of the editing pane.
• This is not really a "Preview" but rather a markup functionality hint.
• We still need the preview pane for more complex structures.
• It does add a certain clarity to longer posts.

It would be interesting to test it.

If we decide to include it, how does it affect your current iteration plan?

In an additive way, it seems like a separate enhancement task which can be done anytime after the the first iteration.

why converting to markdown on the server and not on the client? What are the trade-offs?

This is a great question.

I was given some guidance by @mtias, it included:

• Gutenberg core shouldn’t be affected by what Jetpack chooses to do.
• True markdown support is only going to be plugin territory.
• Gutenberg will attempt to do this as much as possible: https://twitter.com/matias_ventura/status/953239042786766848

My current thoughts on this are as follows:

• Markdown needs to be saved on the server for editing purposes.
• Jetpack has an established workflow of rendering markdown on save and persisting it in the post_content_filtered column of the wp_posts table.
• The classic editor relies on the established markdown workflow with server side rendering.
• We still need the server side rendering after implementing a client side solution.
• Calypso has an implementation of client-side Markdown rendering.
• Gutenberg has on-the-fly helpers which translates some basic Markdown markup e.g. lists, headline.
• Adding rendering on the client will involve adding a script which has to have functional parity with the server implementation.

When I first began to map out the development path for this I thought that client side Markdown rendering would be ideal. I searched candidate open source parsers, deconstructed GitHub's parser a little, and analysed Calypso's Markdown parser.

It seemed to me that, given the fact that server-side Markdown rendering is not going away, there were two options for the Preview functionality:

1) Server Side - Make an Ajax call to render Markdown in Jetpack on Preview click.
2) Client Side- Implement a client side rendering script in the Markdown block, hopefully using as much from the Calypso implementation as possible.

I must say that, I would prefer that this be a client concern only and that we share a single dependency with Calypso for client side Markdown rendering. When choosing a development path, it seems to me, using server side rendering seems like a prudent first pass. I see client side rendering as another separate enhancement task to the previewing functionality. It does not fundamentally change the user experience just provides a little speed to see the preview.

I have not researched the exact steps to execute each option. I will have to do it by the second iteration.

What are the trade-offs?

__Server Side trade-offs__

• May take longer to render on long posts and slow connections.
• Will have to implement a loading indicator.
• Will be consistent with the output produced by the classic functionality.
• Single source of functionality.
• Parsing functionality already exists, but exposing the functionality via Ajax will have to be implemented.

__Client Side trade-offs__

• Potentially a better experience for the user previewing the Markdown.
• Introduces the possibility of a power user view which has side by side code/preview.
• Heavier script load and more processing required client side.
• Potential edge cases, e.g. character encoding, browser quirks, parsing inconsistent with server-side.
• Multiple sources of the same functionality.

[nuzzio]

@MichaelArestad I agree with you and also think that it is worth differentiating, it is not really a "Preview" but rather a markup functionality hint.

We also wouldn't be jumping between font sizes 👍

[nb]

Thanks for the answers, @nuzzio and @MichaelArestad.

I totally agree with both of you that the exposing the formatting is just a hint and should independent from the full preview, which is both a helpful and a must-have feature.

[nb]

As for client side vs. server side, there is another option that I didn't see above – can we keep it 100% client-side by storing the rendered HTML in the serialized block in post_content and the markdown somewhere else (either a block attribute or post_content_filtered). It may require some backwards compatibility wrangling, but seems more in line with Gutenberg’s philosophy to avoid dynamic blocks when we can help it.

P.S. I so much wish GitHub had threaded conversations…

[nb]

One more question – how do we reconcile Jetpack’s per-post Markdown option and the per-block markdown rendering discussed here?

[nuzzio]

@nb

As for client side vs. server side, there is another option that I didn't see above – can we keep it 100% client-side by storing the rendered HTML in the serialized block in post_content and the markdown somewhere else (either a block attribute or post_content_filtered). It may require some backwards compatibility wrangling, but seems more in line with Gutenberg’s philosophy to avoid dynamic blocks when we can help it.

I do think that this is the ultimate solution, I would like to avoid the dynamic block. I also agree that storing the rendered HTML in the serialized block in post_content is the way to go. My client side approach above had the client doing all the markdown parsing, and my naive assumption was to use the post_content_filtered storage as before.

I think the backwards compatibility wrangling will be interesting. I need to get some more background on the intended usage of post_content_filtered. Hmm I think I also need to look at how other blocks are storing data (block attribute) that is not intended to be on the rendered post.

Are there any examples of this that you would suggest looking at?

Your second question:

how do we reconcile Jetpack’s per-post Markdown option and the per-block markdown rendering discussed here?

I think the user contract of the classic implementation had two key parts:

1) We will persist your Markdown parts so that the Markdown parts will be there when you edit.

2) People viewing your post will see the rendered HTML version of your Markdown text, formatted as you specified in Markdown.

I think we can stay true to those two things within the block metaphor. We could make the changes clear to the user in the settings text:

Write posts or pages in plain-text Markdown syntax

to something like

Enable plain-text Markdown syntax in posts and pages. Enable the Gutenberg Markdown block.

What do you think?

[brentlogan]

One potential issue I see with the per-block vs. per-post issue: footnotes. With a per-post perspective, the footnotes end up at the bottom of the post. With a Gutenberg Markdown block, I assume the footnotes should still end up at the bottom of the post, but is that a given?

[nuzzio]

That is a good point. The current footnote functionality exists in the Jetpack Markdown module in PHP. If it becomes 100% client side, there will be edge cases like this that will require parity.

[nuzzio]

I will be continuing comments on the pull request: #9357