Tiptap: Markdown input and output

[SebastianEberlein]

In my use case, I can only load and save Markdown. It is similar to the Prosemirror Markdown example, but instead of the Prosemirror editor I want to use TipTap.

What’s the best way to integrate the Prosemirror Markdown package into TipTap?

[philippkuehn]

Hey @SebastianEberlein. At the moment it's not possible to add this functionality with extensions. That's something I have to add to the core.

[erickwilder]

@philippkuehn this could be a good candidate for 1.x roadmap

[philippkuehn]

Yeah I see definitely see this after 1.0 – maybe for 1.1!

[antonioribeiro]

This package looks really nice, but Markdown is a huge thing today. I'm using markdown to write this very message, and I'm having to find an alternative to your package, only because I cannot use something different to store text. So I don't know how far you are from 1.0 and 1.1, but I hope you can make time to get to that ASAP.

[robguthrie]

Hi there, your project looks great. I'd love to contribute to it's development so we can use it within our project. Can you give any overview or hints about what might be required to support markdown input and output?

[philippkuehn]

hey @robguthrie! that would be great! there is an official prosemirror example and a plugin for that. I think we have to extend the schema for each of our nodes and marks. something like this:

export default class CodeBlock extends Node {

  get schema() {
    return {
      parseDOM: [
        { tag: 'pre', preserveWhitespace: 'full' },
      ],
      toDOM: () => ['pre', ['code', 0]],
      parseMarkdown: () => {
        // not sure about syntax
      },
      toMarkdown: () => {
        // not sure about syntax
      },
    }
  }

}

There is a MarkdownSerializer and MarkdownParser Class (using markdownit) in prosemirror-markdown but I'm not sure if we should use them or should write our own with a more simplistic syntax 🤷‍♂️ It is also interesting how to deal with custom nodes which are not defined in the default markdown spec (like mentions or whatever).

[petrbrzek]

Probably possible via https://github.com/mdx-js/mdx. But not sure if it's still valuable.

[philippkuehn]

BTW: I'm working on it right now! 🤓

[philippkuehn]

Ok, I'm kind of stuck. I decided to go with prosemirror-markdown and its MarkdownSerializer. This is working fine. You can define in every node how to render it as a markdown string. The prosemirror package has a great collection of functions to support all types of nodes I think.

// heading example

get schema() {
  return {
    toMarkdown: (state, node) => {
      state.write(`${state.repeat('#', node.attrs.level)} `)
      state.renderInline(node)
      state.closeBlock(node)
    },
  }
}

But there are some problems:

1. No official table support

One downside of using prosemirror-markdown is that tables aren't support (and won't be). I could write that part on my own but that means we have to drop some advanced functionality for tables like multiple lines, nested tables, using lists inside of tables etc. Maybe we should provide a Table node _with_ markdown support and a ExtendedTable _without_ markdown support?

2. Choosing a markdown parser

After that I tried to implement a parseMarkdown() function and this is way more complicated. There is a MarkdownParser in prosemirror-markdown which is using markdown-it under the hood. This is fine but means that we're limited by that parser. We have to extend the parser rules for custom nodes such as mentions, todo lists, etc. And extending means a lot of code in our config files. I also investigated to use remark but it seems to be even more complicated to extend. I always thought there are markdown parsers out there with a really simple api for extending but I guess I was wrong. :(

Any thoughts?

[qyloxe]

Ok, I'm kind of stuck. I decided to go with prosemirror-markdown and its MarkdownSerializer.

1. No official table support

Any thoughts?

Well, I always thought of tiptap as "data islands" manager. So basically, you have a "markdown island" here, and "xml island" there and "latex island" and "python island" and "my specialized beloved best documentish island". In terms of architecture, I would go with the option to integraty many "island" parsers/renderes and concentrate on smart protocol of communication between them. Which is what you basically do all the time. So, in this matter, we could have "simple but fast markdown without tables support island" and "full markdown support island" or even "my own markdown because I dont like others, island". Tiptap is not an 1:1 mapping of prose features but it is a manager of many features on different levels where every feature needs: serialization, representation, communication, parsing, rendering, events, etc.
Thats my $0.02.

[robguthrie]

I'm so grateful that you've done all this so far!

I was expecting that if we used markdown we would not get the same level of functionality as the html option. In my case, lack of tables support is acceptable for comments. And I imagine that things could improve over time.

Have you reviewed marked, It's been reasonably extensible for our needs.

[eighthday]

So happy you are working on this !

1) no table support is fine for me - markdown editors tend to more minimal than wysiwg, so I would be surprised if this is real need for this at all.

[mutoe]

@philippkuehn I saw your progress on other branches these days.
I want to experience the output Markdown in advance. How can I install the development version?

[philippkuehn]

@mutoe pull the repo. yarn install. yarn start.

[bdrtsky]

I am also vote for no tables!
For tables you should have separate component with data in most cases anyway.

[robguthrie]

I think that tables would be really wonderful to have, as they are part of GFM and other popular markdown specs. However if that's a show stopper right now, I am willing to leave it and consider adding it later.

[MarkGStacey]

I am going to be controversial and say that the table support in markdown should be done fully and properly upfront as it will be much more challenging to shoehorn in later if it isn't considered early on. It's also a huge feature in terms of uptake

[matthew-inamdar]

An early release without table support would be fantastic, with table support added in a later release. This is an amazing package, but this feature is a dealbreaker for us!

[matthew-inamdar]

As a hack 🤢... we have been using a package to convert from markdown to HTML and vice versa which is used to proxy input to and from the WYSIWYM. Extra overhead for the client, but luckily it's an internal app.

[jdrydn]

It's also worth noting that tables in Markdown are an afterthought too, courtesy of GFM.

From daring fireball:

Markdown is not a replacement for HTML, or even close to it. Its syntax is very small, corresponding only to a very small subset of HTML tags. The idea is not to create a syntax that makes it easier to insert HTML tags. In my opinion, HTML tags are already easy to insert. The idea for Markdown is to make it easy to read, write, and edit prose. HTML is a publishing format; Markdown is a writing format. Thus, Markdown’s formatting syntax only addresses issues that can be conveyed in plain text.

For example, to add an HTML table to a Markdown article:

This is a regular paragraph.

<table>
    <tr>
        <td>Foo</td>
    </tr>
</table>

This is another regular paragraph.

Note that Markdown formatting syntax is not processed within block-level HTML tags. E.g., you can’t use Markdown-style emphasis inside an HTML block.

So perhaps it would be worth releasing a version without tables (where tables are inserted as HTML as per the original Markdown spec), and later release a version with { gfm: true } as some sort of config option to format tables into GFM (or whichever spec you decide)?

[nsrosenqvist]

nextcloud/text now seem to integrate markdown in TipTap, perhaps that can be contributed back upstream?

[philippkuehn]

Nextcloud is using prosemirror-markdown for it. But they are using only a few nodes of tiptap (the simple ones – no tables, mentions, …).

Here are some more thoughts about the difficulty of adding markdown support: https://github.com/directus/app/issues/1597#issuecomment-490486735

[juliushaertl]

Hi, Nextcloud Text developer here :wink:

First of all a huge thanks for your work on tiptap, it was really a pleasure to build on top of that :clap:

I was facing the same issues @philippkuehn described above regarding the markdown parser, so for now we use markdownit to render html as an in-between step, which is then passed to tiptap. Of course that is far from an optimal solution but turned out to work quite well for now. I hope to find some time to look into improving that in the future.

extending markdown is a pain

• there are different flavors

Commonmark and GFM which is based on commonmark seem to be the ones that are at least somehow properly specified, so it might be

• for some features there is no standard (tables, mentions, ...)
• for custom nodes you have to come up with custom syntax
  
  
  
  • extending a markdown parser for custom syntax is infinite pain
  
  

Maybe it makes sense to provide markdown-supporting nodes/marks as a separate package, where we can reuse the tiptap ones but also can provide a more simplified tables implementation that fits to the GFM tables extension That way the feature set of tiptap-extensions would not be affected by formatting that is not supported in markdown.

[Subwaytime]

Hey there, any Information about this Topic? 😉

[dougalg]

@philippkuehn how hard would it be to make this pluggable so that the end user can specify their own markdown parser and serializer? That way, tiptap can not have to care about these details, and just provide an interface. Then on the side, you can provide an example implementation using, for example, prosemirror-markdown which will be good enough for most cases, and users can figure out what works for them in the other cases.

Then:

• You don't need to worry about adding lots of code by default
• It's 100% customizable

[douglasg14b]

Markdown support is the only thing stopping me from using tiptap, I sure hope this is added eventually...

[ericnograles]

While we wait on official support in tiptap, here's a Codesandbox of converting tiptap HTML to Markdown (and back) for anyone who wants to experiment: https://codesandbox.io/s/tiptap-markdown-to-html-ssbus

It uses these two libraries:

• Marked: Markdown => HTML
• turndown: HTML => Markdown

I haven't shook out what gets mangled in the process of translating Markdown back to HTML, but it appears to work for basic elements like headers, code blocks, ticks, links, lists. Obviously, this isn't ideal, but it may work in a pinch.

[nonoesp]

Hey, everyone!

I just came across tiptap and it's incredibly nice. It'd be great to use it as a Markdown-only editor. If I had to vote for what Markdown standard to use, I'd go with CommonMark (commonmark/commonmark.js).

[fmym]

GitHub flavored?

[samudranb]

While we wait on official support in tiptap, here's a Codesandbox of converting tiptap HTML to Markdown (and back) for anyone who wants to experiment: https://codesandbox.io/s/tiptap-markdown-to-html-ssbus

It uses these two libraries:

• Marked: Markdown => HTML
• turndown: HTML => Markdown

I haven't shook out what gets mangled in the process of translating Markdown back to HTML, but it appears to work for basic elements like headers, code blocks, ticks, links, lists. Obviously, this isn't ideal, but it may work in a pinch.

This example seems to be broken? Would you please be kind enough to fix it? Thanks in advance!

[samudranb]

Ok, I'm kind of stuck. I decided to go with prosemirror-markdown and its MarkdownSerializer. This is working fine. You can define in every node how to render it as a markdown string. The prosemirror package has a great collection of functions to support all types of nodes I think.

// heading example

get schema() {
  return {
    toMarkdown: (state, node) => {
      state.write(`${state.repeat('#', node.attrs.level)} `)
      state.renderInline(node)
      state.closeBlock(node)
    },
  }
}

But there are some problems:

1. No official table support

One downside of using prosemirror-markdown is that tables aren't support (and won't be). I could write that part on my own but that means we have to drop some advanced functionality for tables like multiple lines, nested tables, using lists inside of tables etc. Maybe we should provide a Table node _with_ markdown support and a ExtendedTable _without_ markdown support?

2. Choosing a markdown parser

After that I tried to implement a parseMarkdown() function and this is way more complicated. There is a MarkdownParser in prosemirror-markdown which is using markdown-it under the hood. This is fine but means that we're limited by that parser. We have to extend the parser rules for custom nodes such as mentions, todo lists, etc. And extending means a lot of code in our config files. I also investigated to use remark but it seems to be even more complicated to extend. I always thought there are markdown parsers out there with a really simple api for extending but I guess I was wrong. :(

Any thoughts?

Echoing others above, I would rather have something that I can export as Markdown, rather than wait for the perfect solution.

From your message above, it sounds like you almost have it working for most of the basic nodes? I understand some nodes (e.g. tables) might not work, but we can consider that as a known limitation and address it via documentation?

I would love to use TipTap for a project that needs to save the output in markdown, and limit the users to only the basic nodes:

• Bold, italic, underline, strikethrough etc.
• H1, H2, H3
• Ordered and unordered lists
• Images
• Quotes

Everything else is icing!

Happy to help with documentation if needed.

[tre-dev]

Even bold and italic would be enough for me tbh. Everything else, if supported, could be added later on.

[hanspagel]

I don’t see us adding this to the core, it’s just too different. So I’m closing this here for now. Consider using CodeMirror, if you really want to work with Markdown.

Feel free to comment though.