Nextcloud-notes: Replace RxMarkdown with Markwon

[quthla]

Some tasks left, but i think we are already on a good way

• [x] Syntax Highlighting (needs another lib i think)
• [x] Different Header levels do not work properly yet in edit mode
• [x] Fix coloring on search
• [ ] Check if we can prevent selection of tasks when toggling checkboxes
• [x] Check if a checkbox can be toggled when clicking on the checkbox directly (not the text)
• [x] Check external links, especially fix or remove links in checkboxes (affects maybe only edit mode?)
• [x] Make sure that internal links work
• [x] Styling (should be as close as possible to the webapp)
  
  
  
  • [x] Codeblocks should always use monospace font
  
  
  • [x] Closing fence of cosing blocks are different from opening blocks
  
  
• [x] Extract plugins where possible to make them reusable
• [x] Centralize the configurations
  
  
  
  • [x] Respect monospace font setting
  
  
  • [x] Respect font size setting
  
  
• [x] Use Markwon on all places where previously RxMarkdown has been used (e. g. widgets)

[stefan-niedermann]

Looks interesting. Does it support inline-editing?

[tnyeanderson]

Keep an eye on this, maybe we can get the same platform for mobile and the webapp:

https://github.com/nextcloud/notes/issues/204

[tnyeanderson]

I don't think Markwon will work with EditText. It's not built for it. We do need to find a replacement though. This issue should be retitled and used to track finding a new markdown library.

[tnyeanderson]

Here's one to look at: https://github.com/crescentflare/SimpleMarkdownParser

[stefan-niedermann]

Unfortunately only supports

headers (#), text styles (italics and bold), strike through text, lists and links

which is way less then the current lib.

[tnyeanderson]

Definitely, it's a newer library (says more support is coming.... who knows when). It's surprisingly difficult to find renderers that are actively maintained and allow inline editing :(

What else would need to be included? Those styles are supported by nextcloud/notes using SimpleMDE, and only supports the following in addition:

• Quotes
• Images
• Tables
• Code

We should also include tasks functionality (with clickable checkboxes)... but to get SimpleMarkdownParser up to par might not be as bad as we think...

What other syntax would be needed other than those listed?

[tnyeanderson]

https://github.com/vsch/flexmark-java

This might actually be better. It's a fork of the core library used by Markwon, designed for EditText. This might actually be usable :)

[quthla]

https://github.com/gsantner/markor

This project has a proper inline editor.

[stefan-niedermann]

Seems they have not published their markdown logic to a separate lib, so it would be necessairy to extract and keep it up to date :(

[tnyeanderson]

Hate to say the same thing twice, but I looked into a lot of these libraries and https://github.com/vsch/flexmark-java seems to meet the needs here.

Thanks for all the great work @stefan-niedermann !!

[stefan-niedermann]

@tnyeanderson i meant markor, i didn't have a look at flexmark-java yet, but i sure will do that :)

[stefan-niedermann]

So i read the REAMDE twice, but i have no idea how to get this marriaged with an EditText. All i can see is a Parser which creates some kind of abstract model, but the default rendering seems to beHTML.

Maybe you can provide some sample of how to use flexmark-java? Does not have to bee a comolete Pull Request, but some inspiration of how to use this in combination with an EditText. It seems to be required to implement the connection between the EditText and the AST?

[quthla]

It's not meant for EditText by itself but the sample you're asking for is basically markor.

https://github.com/gsantner/markor/blob/master/app/build.gradle

[tnyeanderson]

@stefan-niedermann It took me a while (been a while since I took a look), but I FINALLY found where I got that information from.......... this Markwon issue: https://github.com/noties/Markwon/issues/97#issuecomment-460990096 ..... I think it may be talking about this plugin: https://github.com/vsch/idea-multimarkdown

This is why I need to check myself before I repeat things I said long ago...... I have no idea how to actually get these things working, and that commenter could have been misinformed.... I have no clue.

That said.... I really like Markor. Great find @quthla !

Hope this helps

[stefan-niedermann]

Okay, i will check this and maybe (depends on license) we can even reuse some of Markors code.

However, i am currently very busy with implementing Single-Sign-on and Multi-Account-support for the Notes app. This comes with huge changes in the whole app, therefore i would be glad if we could move this topic to after the PR has been merged :)

[MisterY]

Flexmark states Android compatibility to be added, and I believe that Markor is using commonmark/commonmark-java Markdown parser.
Ref. Github-flavoured Markdown.

[lenzls]

I just skimmed through this thread and wonder what is meant by inline editing and why it is a requirement for a new library.

Is the plan to merge edit and preview mode?

---

I really like the way the nextcloud notes webapp handles this.

An edit mode that immediately interprets markdown elements but displays them half transparent.

And a read only preview mode that does not even include the markdown elements. Just renders the document nicely.

I think this is quite elegant and underlines markdown as a markup language which should be functioning and visually pleasing also in plain text view.

I wonder if you meant something like this or more of a preview mode with inline editing. The second I would see critically. More often than not these do not what I want and I do all my edits in edit-only mode in the end.

[stefan-niedermann]

The first one is meant - ideally just like the webapp, but native without webview :)

[MisterY]

Just a brief update. I recently installed Markor again and notice that the editor has improved significantly. It now does support Github-flavored Markdown, including to-dos, strikethrough, etc. Looks and acts very decent. Edit mode with visual Markdown indication and a toolbar for formatting, Preview mode. Check it out.

[quthla]

@stefan-niedermann may I ask what the plan is? Current editor is broken in many ways unfortunately

[stefan-niedermann]

Not sure yet. It will be very complex one way or the other. I even thought about forking RxMarkdown and fix it.

[quthla]

Not sure if that makes sense. Markor is just way better and has more features.

Maybe you can extract the Markor markdown handling code into a module and PR that to Markor. Then there's no big maintenance burden anymore.

[beatbrot]

By the way, Markwon in the meantime has gained editor support in release 4.2.0

[stefan-niedermann]

I agree that Markwon looks more and more interesting as time passes. I will have another look at it.

[stefan-niedermann]

I created a very rough proof of concept (download apk).

Before we really put everything into this basket, i need to know first, whether we can add support for

• Toggling checkboxes in preview mode
• Enable live-editing of Header level 2-6 (i only got level 1 working in this short time)

I noticed a few bugs in the edit mode, e. g. links (see below), but i think we can live with them or even disable linking in edit mode at the beginning.

That's indeed all.

I would appreciate each helping hand, with those two issues, but especially with the toggle-checkbox-thingy. this has to work. :rocket:

[stefan-niedermann]

Since the official Nextcloud (files) app also bets on Markwon, it just makes sense to set our focus on that library.

[stefan-niedermann]

Just for reference:

• Hint about toggling checkboxes in preview mode
• Hint about how ti fix the broken links in lists

Really bracing how nice a maintained lib with support feels 😄

[beatbrot]

I did some further investigtion on Markown:

• It seems like the suggested method for clickable checkboxes only works if we remove android:textIsSelectable="true" from fragment_note_preview. I am not yet sure what the implications of that are.
• If we then enable the clickable checkboxes, we get the problem that the line gets selected as soon as we try to trigger the checkmark. This doesn't look very nice.

[stefan-niedermann]

@beatbrot can you push your changes to the markwon-branch? It is just a proof of concept to check if everything will work.

When we are sure that everything we need is technically possible, i will open a new branch from scratch with a clean implementation.

You don't need to watch code style or stuff, it's justa playground.

[stefan-niedermann]

@beatbrot just for info, i have a proof of concept that checkboxes are clickable with enabled textIsSelectable="true" attribute. It's not great, but it works with a similar workaround we used previously with RxMarkdown. Would be sufficient for me.

[beatbrot]

@stefan-niedermann And I added a PoC version for different header levels :)

[stefan-niedermann]

moved task list to issue description

[beatbrot]

I tried adding support for code-blocks in the markwon-branch.

Unfortunately, the opening fence is black while the closing fence is gray. I couldn't get rid of this unfortunately.

[stefan-niedermann]

What exactly tried you to achieve?

[beatbrot]

I wanted to achieve, that the text in code-blocks is monospace and has that gray-ish background.

P.S.: I forgot to push so the commit just went up a few seconds ago :P

[stefan-niedermann]

aah, okay, nice though, the closing fence is no show-stopper for this. I will add it as a bullet point at the very top of this issue, but as nice to have.

[beatbrot]

• [ ] Respect monospace font setting
  
  
  
  • [ ] Respect font size setting
  
  

Implemented those two things :)

[beatbrot]

Btw, is there a reason why

[ ] Different Header levels do not work properly yet in edit mode

is not yet checked? Do you still have any issues with it?

[manuelkamp]

Does this add the correct view of tables, like the nextcloud web app shows them?

[stefan-niedermann]

This is one of the goals, yes.

[Baldomo]

Since Markwon does everyting with Spannables, would it be possible to implement a kind of dynamic separation between checked and unchecked items (a la Google Keep)? Would it be easy or would it take considerable effort? Just asking because I haven't used the library myself

[stefan-niedermann]

We'll see as soon as we have restored the current behavior and features with the library. It's taking longer than expected and we are still at trial stage.

[larena1]

We'll see as soon as we have restored the current behavior and features with the library. It's taking longer than expected and we are still at trial stage.

Does this mean there's been no further progress in contrast to what was done more than 3 months ago already?
Can you elaborate on the obstacles you're facing while integrating Markwon?

[stefan-niedermann]

@larena1 as you can see i did in the past 3 months more work than in the whole time of the project together.

While multiple features could be accomplished, there was not much progress at this particular issue.

Can you elaborate on the obstacles you're facing while integrating Markwon?

Main issues are

• Markwon Editor is quite new
• In opposite to RxMarkdown nearly every feature like Headings etc. have to be implemented by us. This is no simple exchange
• Markdown is used in various parts of the UI. The project grew in the past and i would like to try to split the new editor into a separate module for easier exchanges in the future
• We need an editor and a viewer, which work completely different (believe it or not)

Does this mean there's been no further progress in contrast to what was done more than 3 months ago already?

Please keep in mind that i work on this project for fun in my personal free time. I consider this kind of questions disrespectful and they really do not motivate me nor other contributors.
You are welcome to contribute more than asking "Why haven't you implemented this for free yet?".

I would appreciate if we could keep the issue clean and only comment if we have new information to the actual topic.

[larena1]

Did you try raising an issue in the Markwon repo, state your plans and discuss your concerns there? The author might possibly be interested in helping out with easing the process of integrating Markwon features into 3rd party apps.

[stefan-niedermann]

Yes i contacted them for various topics like checkboxes and got good responses so far.
If you checked out the PR you will notice that basic editing is already possible.

Major issue remains my free time which i have to split for various open source projects (like the Deck Android app). Therefore again: You will need to stay patient or contribute some code (remaining topics are documented at the top of this issue) 🙂

[fabianski7]

The checkboxes are not being rendered if they are in a sublist, will these proposed changes in this issue also solve this or should I create another issue?

The result for the same text here at github:

- Let's do something
    - [ ] step 1
    - [ ] step 2

• Let's do something
  
  
  
  • [ ] step 1
  
  
  • [ ] step 2
  
  

[stefan-niedermann]

Hi @fabianski7

this depends on the new lib (Markwon or Nextcloud Text - not yet finally decided). So no, a separate issue does not make sense at the moment. In case the new lib does not support it oit of the box, we can open an issue upstream after implementing it.

[fabianski7]

Hi @fabianski7

this depends on the new lib (Markwon or Nextcloud Text - not yet finally decided). So no, a separate issue does not make sense at the moment. In case the new lib does not support it oit of the box, we can open an issue upstream after implementing it.

Thanks for the answer. Let's wait then.

[larena1]

Any news here @stefan-niedermann?

[stefan-niedermann]

I am currently working on a modernization of the architecture (e. g. migrating from SQLiteOpenHelper to Room) to clean up the code and make maintenance easier.

When i am working on the editor, i will post updates here, and if i don't work on the editor, i obviously won't post anything. Everyone is always welcome to join the development and work on the branch. I will happily support where i can if someone wants to pick up this.

I will lock the conversation for now, since the issue is getting quite unreadable with all the comments which don't provide any news compared to the checklist on the very top.

[stefan-niedermann]

Some updates for everyone who subscribed to this issue:

Today we started rolling out the Deck Android app 1.13.2 which is kind of a big deal for the Notes app in general and especially this issue. Why? Well, the new Deck version features a new markdown editor for the description field.

This editor is based on Markwon and we used the Deck app as a testing environment here: The new editor can improve over the time and as soon as it reaches the same function level as the Notes app has today, we can easily reuse it for Notes.

Today we made a very big step because the basic editoe is working, it already has auto continuation for lists and checkboxes and we are currently working on the context based formatting which the Notes app also has today. As soon as this has been implemented, we can pull the editor over to the Notes app. For the viewer we are still missing support for togglable checkboxes as well as proper link handling. But also here we will implement and test the Markwon based viewer in the Deck app and can pull it over as soon as it is feature complete.

Thank you for the patience. Everyone who is interested in contributing some code: Just ping me in the linked pull request or in the linked blog post of the help forum.

[stefan-niedermann]

Hey there :wave:

I have just published v2.18.0 at the Play Store beta channel. This version features the new Markwon based editor. Basically it should be way more reliable (especially when handling lists) than the old editor.
It also should be available on F-Droid in a few days. I am very grateful for each feedback to the new editor of this new version (Just open separate new issues).

We are continuing to work on the viewer which is basically only missing "togglable checkboxes" and then can replace the old RxMarkdown based viewer.

[jakobroehrl]

Thanks for your work! 👍
Would it be possible to add some shortcuts for checkbox, headlines, lists, ...?

Example from Ticktick:

---

Example from Notion:

---

Is it necessary to have write and view modus? Is a WYSIWYG view maybe possible?

Could it be possible to beginn the next line with a new list item/checkbox if the previous was a list/checkbox item?

[stefan-niedermann]

We use context based formatting to avoid distractions while writing. This is not "Word on Android" and another scope.

You have the shortcuts available in a context, e.g.

• when you select some text, you can make it bold, italic etc.
• when you hit the selector thumb without selected text, you can will have actions in the context of the current line, like making it a checkbox

The context makes sense to only show the actions that make sense for the current context. For screenshot see here: https://github.com/stefan-niedermann/nextcloud-deck/pull/760

(And yes: We will extend this system further in the future and might add toggles for headlines etc)

[jakobroehrl]

* when you select some text, you can make it bold, italic etc.

* when you hit the selector thumb without selected text, you can will have actions in the context of the current line, like making it a checkbox

This is very good, but I missed it, maybe you could give a hint on the start or so?

[stefan-niedermann]

Yes, i think i will implement some kind of hint for the first run. It is already mentioned in the "❓ Formatting" entry at the sidebar. If you want you can open a separate issue for that, it doesn't belong into this issue 🙂

[stefan-niedermann]

Quite a long time, but finally i merged the new Markwon based preview mode to the master branch and released v2.19.0. It will be available on the Play Store beta channel and on F-Droid in a few days.

Thank you all for your patience, i hope you will do excessive testing and report any issues here, so we can finally push it to the Play Store release channel soon. :slightly_smiling_face:

:eyes: Screenshots

| 1 | 2 | 3 |
| --- | --- | --- |
| | | |