Even if markdown is not correctly interpreted, seeing .md files as text within zim would be an advantage
as we could integrate markdown pages within the zim tree.
I'm all in favor of this - but only with a reasonable robust parsing and writing of markdown code. Good to start with a subset of features of markdown.
Probably also requires disabling some features in zim when source is markdown (e.g. insert of rich objects).
The Python CommonMark parser can generate an abstract syntax tree from Markdown/CommonMark text.
I would say not to stop at Markdown. AsciiDoc format would be great too
@takacsot I guess another format request should go to another issue.
+1
Is there any progress on this? Will it be easy to implement on the codebase or is the Zim wiki syntax too tightly coupled with the application?
@gkbrk there is some coupling between features in the editor and features in the wiki format, but nothing preventing adding formats. Would be intermediate difficulty I think.
I am not against editing markdown directly until the html output will be fine.
Could it be simpler to have dual screen editor like https://stackedit.io/editor ?
By this the editor could be plain text editor and have an automatic preview screen too. In case of display only mode only preview screen could be visible. I could imagine that such a "mode" could simplify the integration of other markup languages too.
What do you think?
FWIW, here is a ready-made repo with some BASIC TEST DATA (no images, etc).
Any traction on this one?
This would make zim very useful for storing a local notefile that you could also read in a private github repo in a browser.
Note that there exists a maybe outdated (code blocks for example seem not to be done right) zim2markdown / markdown2zim conversion Python script: https://github.com/Xunius/markdown2zim
Maybe this can help.
Actually I personally am very interested in ZIM handling markdown transparently for the following reason: With https://github.com/aaren/notedown it would be easy to convert such Markdown pages into Jupyter Notebooks which I work with a lot. That would allow me to write lectures or prepare talks perfectly organized in ZIM, converting them easily to Jupyter Notebooks when needed and run the code cells resulting from ZIMs "verbatim blocks" in Jupyter.
Vice versa it would be possible with notedown to convert existing Jupyter Notebooks to ZIM pages. This would very much help me organize all the experiments I frequently do in Jupyter notebooks, make them versionable with git (via ZIMs versioning plugin) and readable and editable in an organized way without starting a Jupyter notebook server.
@jaap-karssenberg - nudge
My interest is similar: I generate many *.md pages out of R via rmarkdown and I could organize them within zim, along with my notes, which are in zim.
I really like @takacsot 's stackedit.io and AsciiDoc's idea. Besides of dual panel like stackedit.io, we can also consider listen to the key event. Once the special block (for example, latex, mermaid, code...) is ended, then the app render and override the screen. If any mouse or keyboard event is caught, the rendered image is gone. Of course, this approach is more complicated. Just in case if the author has more time ;)
In the most remote and disconnected sense, I think that two panels would be a killer feature (#267), but that is in expanding zim's functionality and effectiveness. I would literally rather move back to editing plain text than give up on WYSIWYG, or fall back to a two-stage edit/preview workflow, or have aweful studdering jumpy bits from constantly reloading pages.
Any news? I agree that this would be a killer feature and also could be used as Nextclouds note editor and alternative to QownNote
or perhaps QownNote is the note editor alternative to Zim...
Agus
On Sun, Sep 9, 2018 at 11:33 AM TheNomad11 notifications@github.com wrote:
Any news? I agree that this would be a killer feature and also could be
used as Nextclouds note editor and alternative to QownNote—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/jaap-karssenberg/zim-desktop-wiki/issues/26#issuecomment-419702743,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AFyuxhU-Z5Ck347CSx8NvvWmykxB4wy_ks5uZOBsgaJpZM4NWSs6
.
--
Agustin Lobo
[email protected]
I would like to point your attention to a new project: jupytext
It claims to be able to convert between
and comes with a lot of tests guaranteeing that (apart from Jupyter notebook output) nothing gets lost inbetween conversions. Furthermore it has a "pair mode" where you can work on a Jupyter notebook and then the associated markdown file is automatically updated (and vice versa).
The point why I am mentioning it here is that once one has a working parsing/conversion between ZIM pages and only one of the above formats, e.g. .md, one gets the other formats for free from jupytext. Furthermore, having a look on the source code of jupytext, if we had a working ZIM<->md conversion, one could directly integrate ZIM as just another format in jupytext.
This way
I currently considering putting some work into this, but I would like to here your opinion first. What problems do you see?
Zim is incredible. So are tools like typora and nextcloud. Markdown is also pretty ubiquitous now. I would love to know how I can contribute to this. If anyone wants to reach out with info, please do.
Key steps to integrate markdown would be:
2nd phase would be user interface improvements:
Biggest hurdle will be how to handle inserted objects. Either use a
markdown extension for these, or disable them in the editor for markdown
pages.
Regards,
Jaap
On Fri, Mar 15, 2019 at 12:49 PM dpalmigiano notifications@github.com
wrote:
Zim is incredible. So are tools like typora and nextcloud. Markdown is
also pretty ubiquitous now. I would love to know how I can contribute to
this. If anyone wants to reach out with info, please do.—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/zim-desktop-wiki/zim-desktop-wiki/issues/26#issuecomment-473256821,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABMMHnT3KOwhkhG90A6RBCrYq_tnpYzNks5vW4jkgaJpZM4NWSs6
.
Would love to see this happen, but I don't have the time to add it. If somebody can land a PR for this I'd be happy to send a donation their way (tag me on pr).
A couple things: If we can get markdown tightly integrated the result is basically numerous mobile and cloud hosted apps already built. 1) This solves the lack of a native android app. 2) This allows cross platform collaboration easily -- gf doesnt wanna install zim on her mac and to be fair it is kinda sluggish. 3) as a result of the former 2, efforts can be focused on things that really matter: linux dev.
Also found this. Might make our job a bit easier. https://github.com/Xunius/markdown2zim
I think markdown note taking is becoming a necessity nowadays. Zim is the best note taking app for produtivity in Desktop. I use it at production level for a couple of years now. However, nowadays I'm practically obliged to use typora or boostnote for singlefile note taking and Zim to manage multiple files, tag, crossrefs, etc. It is annoying, let me say.
Markdown and coding blocks are so widely spread (including coding notebook like Jupyter). Thus, I think the only (or most recommended) direction to take in a near future would be to integrate a markdown viewer/editor and slowly moving from .txt. gui-based editing to .md gui-based editing. A possibly easier way to do it might be to use another markdown open-source editor as a backen, like Boostnote.
I would love to support this project.
Production note-taking app Utopia for me
Are there any updates on this. I think this should be the current priority, dont you guys?
The easiest way i can see this being implemented, at least temporarily, is to hijack the code that opens a file and the code that saves the file, pipe both through https://github.com/Xunius/markdown2zim and add additional code to perform the extension change from .txt to .md
As far as zim is concerned its seeing the wiki syntax its familiar with, and the filesystem sees only markdown.
@BeatLink I know how to fix this, but for me this is lower priority than a number of other things. If you feel like working on this, we can discuss the exact implementation.
All: there is no need to add comments just to ourselves how important this is. The only thing that will pull this in is actual code being submitted. If it is up to me to fix it please be very patient. Do realize that I'm doing this on my own outside of my actual working hours and as I'm already fulltime employed there is no way I can spent more time on zim.
So please submit your own code - stable code with a test suite - or be patient.
Its pretty hacky but i managed to get markdown support to work here https://github.com/BeatLink/zim-desktop-wiki/tree/markdown-support. It uses zims built in markdown export format in addition with the markdown2zim code above to parse. Change the default extension and boom it works, kinda. Ill see if i can use the markdown2zim dumper as well for a consistent parsing and writing.
Update, it now uses markdown2zim code both ways, for parsing and saving. Basic functionality works but its pretty buggy.
@jaap-karssenberg Would your implementation solve all the bugs? If so, how would you go about implementing this?
@beatlink I think a good next step is to create test cases that ensure that
all markdown features are at least handled gracefully - either they work,
or at least they "pass through" in a compatible way. I suggest going
through this spec step by step and make sure your implementation is
consistent: https://github.github.com/gfm/
Then the other challenge is to make sure that all zim features are
handled correctly. So what happens if a user inserts a tag or an object,
does that work? What happens for images with width and height attributes
set? And is that handled in a way compatible with other markdown
implementations? A first step could be to disable features that do not
work for markdown pages. To do that the pageview widget needs to know the
format in which a page is stored and make a bunch of features inactive.
Regards and good luck!
Jaap
On Tue, Jul 23, 2019 at 11:42 PM BeatLink notifications@github.com wrote:
@jaap-karssenberg https://github.com/jaap-karssenberg Would your
implementation solve all the bugs? If so, how would you go about
implementing this?—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/zim-desktop-wiki/zim-desktop-wiki/issues/26?email_source=notifications&email_token=AAJQYHTMTTPJKHBFBKP7PJLQA53LHA5CNFSM4DKZFM5KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2UQWNA#issuecomment-514394932,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAJQYHQ5O7ZUM7M44HP3ALTQA53LHANCNFSM4DKZFM5A
.
Thanks! I'll get on it!
Perhaps another alternative to consider would be a zim2 package that would
display the correct format of markdown pages only and the txt pages without
formatting. Then a separate program that would convert current zim
formatted txt files to md. Zim2 would progressively replace zim. I imagine
that keeping both formattings in zim will be cumbersome.
Agus
On Wed, Jul 24, 2019, 22:14 BeatLink notifications@github.com wrote:
Thanks! I'll get on it!
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/zim-desktop-wiki/zim-desktop-wiki/issues/26?email_source=notifications&email_token=ABOK5RW5AI7CAXCA2RCZKW3QBCZZLA5CNFSM4DKZFM5KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2XPLKI#issuecomment-514782633,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABOK5RSK66FOLSYVI6MCBD3QBCZZLANCNFSM4DKZFM5A
.
@aloboa no, splitting off a separate code base for the markdown formatting
actually will make it worse, because now you got to maintain two programs -
this only works assuming you have two teams of people - zim has all the
infrastructure in place to support multiple formats
On Thu, Jul 25, 2019 at 8:33 AM aloboa notifications@github.com wrote:
Perhaps another alternative to consider would be a zim2 package that would
display the correct format of markdown pages only and the txt pages without
formatting. Then a separate program that would convert current zim
formatted txt files to md. Zim2 would progressively replace zim. I imagine
that keeping both formattings in zim will be cumbersome.
AgusOn Wed, Jul 24, 2019, 22:14 BeatLink notifications@github.com wrote:
Thanks! I'll get on it!
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<
https://github.com/zim-desktop-wiki/zim-desktop-wiki/issues/26?email_source=notifications&email_token=ABOK5RW5AI7CAXCA2RCZKW3QBCZZLA5CNFSM4DKZFM5KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2XPLKI#issuecomment-514782633
,
or mute the thread
<
https://github.com/notifications/unsubscribe-auth/ABOK5RSK66FOLSYVI6MCBD3QBCZZLANCNFSM4DKZFM5A.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/zim-desktop-wiki/zim-desktop-wiki/issues/26?email_source=notifications&email_token=AAJQYHSAUBKPTLERUKRUCXDQBFCKXA5CNFSM4DKZFM5KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2YP7LA#issuecomment-514916268,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAJQYHXOFEZLPHACB3ITOMLQBFCKXANCNFSM4DKZFM5A
.
Unfortunately, looking at the code, I'm not sure i'd be able to take it on, as i do not have the time myself. However, i will be closely tracking the progress on this, and my branch will still remain up, if for nothing more than a proof of concept or potential starting point. Hope you find the time soon
Update: after spending the last 24 hours searching for markdown apps and even trying to code my own, i now realize that it would be less effort to implement this in zim
Update to the Update: I switched to atom. Sorry guys.
Just for info, I updated markdown2zim to Python 3.7.
Also, for inspiration, there IS an opensource alternative to Typora, which started around the same time (but is not as much developed since it's not meant to be a commercial app): http://abricotine.brrd.fr/
Is this markdown support still under consideration?
I am not against editing markdown directly until the html output will be fine.
Could it be simpler to have dual screen editor like https://stackedit.io/editor ?
By this the editor could be plain text editor and have an automatic preview screen too. In case of display only mode only preview screen could be visible. I could imagine that such a "mode" could simplify the integration of other markup languages too.
What do you think?
Although what I argue below is true for me, and may not be true for others, I thought it would be helpful to provide my views on this subject nonetheless, so, here it goes.
Dual screen is too distracting in my opinion. I have used (and still use, depending on the task at hand) Asciidoctor with Guard or Epiphany to auto-reload when I save a file, and place the editor and the browser side by side, but (for me, at least) it generates a good deal of cognitive burden.
Having to look back and forth between editing and preview windows takes time and energy, and causes distraction as compared to just having an _inline preview_. One has to constantly search for the precise place in the preview window that was updated due to the changes in the editing window. Zim and Typora excel in this area. You just type and format and see the result right then and there. No distraction, no cognitive burden.
The main difference between Typora and Zim in terms of markup editing, is that Zim generally opens a dialog where one edits links, images, etc, and in some cases we open the entire source code markup .txt file and edit it from an external editor, and Typora displays the underlying markup when the cursor enters the area, like an inline code block or image. I have, as of now, no formed opinion on which would be best or more advisable on a given situation, but I think both are fine.
I agree that a live editor with inline preview is better than a dual window. Anyway there should be no issue on that front, since Zim already has a GUI that supports inline preview. The only thing missing is a modular architecture separating the language parsing and other functionalities of the software (live previewing, document/links management, etc) so we could swap the Zim language for any other language we would want.
Hi, I just came across Zim and would love to use it if markdown support was available.
Its pretty hacky but i managed to get markdown support to work here https://github.com/BeatLink/zim-desktop-wiki/tree/markdown-support. It uses zims built in markdown export format in addition with the markdown2zim code above to parse. Change the default extension and boom it works, kinda. Ill see if i can use the markdown2zim dumper as well for a consistent parsing and writing.
The link is broken.
Its pretty hacky but i managed to get markdown support to work here https://github.com/BeatLink/zim-desktop-wiki/tree/markdown-support. It uses zims built in markdown export format in addition with the markdown2zim code above to parse. Change the default extension and boom it works, kinda. Ill see if i can use the markdown2zim dumper as well for a consistent parsing and writing.
The link is broken.
Must have accidentally deleted the repo during a cleanup. I may try to help develop this in a month or two but right now im just too busy with school :(
FYI I've put a bounty up for this. Markdown support would be a huge plus for me personally
Update: I've bumped up the bounty
@jaap-karssenberg since there is now a bounty, will you be willing to implement it? This is one of the most asked for and second oldest issue in the repo. Allowing markdown usage will instantly make zim compatible with thousands of other markdown apps on desktop, web and especially mobile. You could kill so many birds with this one feature
@Beatlink please see: https://lists.launchpad.net/zim-wiki/msg04895.html
On Mon, Jun 8, 2020 at 9:47 AM BeatLink notifications@github.com wrote:
@jaap-karssenberg https://github.com/jaap-karssenberg since there is
now a bounty, will you be willing to implement it? This is one of the most
asked for and second oldest issue in the repo. Allowing markdown usage will
instantly make zim compatible with thousands of other markdown apps on
desktop, web and especially mobile. You could kill so many birds with this
one feature—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/zim-desktop-wiki/zim-desktop-wiki/issues/26#issuecomment-640429635,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAJQYHQJUQATYAVJIJ5XG4DRVSJRNANCNFSM4DKZFM5A
.
@jaap-karssenberg since there is now a bounty, will you be willing to implement it? This is one of the most asked for and second oldest issue in the repo.
@beatlink Is there any hope you recover the markdown-support branch you once shared?
Just an opinion but I think we just need Zim to be in a state where the Zim markup is decoupled from the rest of the app (particularly the display and GUI). Making a parser for another language would then be easily doable for an external dev (I would be willing to try but no promises) and also allow to continue the development on the Android app (it needed the Zim to HTML parser to reach the next milestone).
I didn't read the new codebase since the major rewrite for gtk3, maybe now the markup is more modularized?
It is also high on my personal wishlist, and I'm using a markdown editor
myself on Android, so I'll get around to fix it eventually.
However, it takes time. And one thing I simply don't have at the moment is
lots of time.
Op ma 8 jun. 2020 20:20 schreef Maykel Moya notifications@github.com:
@jaap-karssenberg https://github.com/jaap-karssenberg since there is
now a bounty, will you be willing to implement it? This is one of the most
asked for and second oldest issue in the repo.Is there any hope you recover the markdown-support branch you once shared?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/zim-desktop-wiki/zim-desktop-wiki/issues/26#issuecomment-640792631,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAJQYHVLMQRLTNSW2O3W3UDRVUTVRANCNFSM4DKZFM5A
.
Sorry to be blunt, but probably it is better to first have a look at the
code base before offering opinions.
I hope you read the mail I linked above and realize the amount of stress it
causes me when there is all this discussion of "zim really ought to..."
without anyone putting in the effort to understand the work.
So if you want to help please start helping.
Op ma 8 jun. 2020 22:31 schreef Stephen Karl Larroque <
[email protected]>:
Just an opinion but I think we just need Zim to be in a state where the
Zim markup is decoupled from the rest of the app (particularly the display
and GUI). Making a parser for another language would then be easily doable
for an external dev (I would be willing to try but no promises) and also
allow to continue the development on the Android app (it needed the Zim to
HTML parser to reach the next milestone).I didn't read the new codebase since the major rewrite for gtk3, maybe now
the markup is more modularized?—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/zim-desktop-wiki/zim-desktop-wiki/issues/26#issuecomment-640869501,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAJQYHX774SNZVW46MVQPQ3RVVDBDANCNFSM4DKZFM5A
.
P.s. the latest release actually contains important steroid towards
markdown support. One is the support for overlapping formatting styles. The
other is the url parsing which is more compatible. So there are actually
concrete steps towards making it easy to "just" add the parser.
Op di 9 jun. 2020 07:11 schreef Jaap Karssenberg
:
It is also high on my personal wishlist, and I'm using a markdown editor
myself on Android, so I'll get around to fix it eventually.However, it takes time. And one thing I simply don't have at the moment is
lots of time.Op ma 8 jun. 2020 20:20 schreef Maykel Moya notifications@github.com:
@jaap-karssenberg https://github.com/jaap-karssenberg since there is
now a bounty, will you be willing to implement it? This is one of the most
asked for and second oldest issue in the repo.Is there any hope you recover the markdown-support branch you once shared?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/zim-desktop-wiki/zim-desktop-wiki/issues/26#issuecomment-640792631,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAJQYHVLMQRLTNSW2O3W3UDRVUTVRANCNFSM4DKZFM5A
.
Sorry Jaap that my message was stressful, it was not my intention but
rather the opposite, to try to propose a lesser amount of work for you and
on which others such as me can build upon.
Yes i saw these progresses and that's awesome. I tried to look at the
codebase in the past when i worked on the Android draft, but Zim has become
so complex it's difficult for an external dev to pick it up without a
technical documentation. This only demonstrates the marvelous job you did,
and i can only commend you and thank you for that. I only wish i could help
more with PRs.
Can you just tell me a quick rundown of the python files where the zim
markup is handled and I'll give it a try?
Le mar. 9 juin 2020 Ã 07:31, Jaap Karssenberg notifications@github.com a
écrit :
P.s. the latest release actually contains important steroid towards
markdown support. One is the support for overlapping formatting styles. The
other is the url parsing which is more compatible. So there are actually
concrete steps towards making it easy to "just" add the parser.Op di 9 jun. 2020 07:11 schreef Jaap Karssenberg <
jaap.[email protected]:
It is also high on my personal wishlist, and I'm using a markdown editor
myself on Android, so I'll get around to fix it eventually.However, it takes time. And one thing I simply don't have at the moment
is
lots of time.Op ma 8 jun. 2020 20:20 schreef Maykel Moya notifications@github.com:
@jaap-karssenberg https://github.com/jaap-karssenberg since there is
now a bounty, will you be willing to implement it? This is one of the
most
asked for and second oldest issue in the repo.Is there any hope you recover the markdown-support branch you once
shared?—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<
https://github.com/zim-desktop-wiki/zim-desktop-wiki/issues/26#issuecomment-640792631
,
or unsubscribe
<
https://github.com/notifications/unsubscribe-auth/AAJQYHVLMQRLTNSW2O3W3UDRVUTVRANCNFSM4DKZFM5A.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/zim-desktop-wiki/zim-desktop-wiki/issues/26#issuecomment-641038581,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAIRFXVGFJJHTUMHGJ4H4QDRVXCM3ANCNFSM4DKZFM5A
.
@Stephen thanks for you support - sorry to get stressed out a bit at times
The key file to add the markdown parsing is "zim.formats.mardown" it should
contain a "Parser" and a "Dumper" class that transform markdown from text
to zim internal structure and back. Have a look at "zim.formats.wiki" for
example code.
The main challenge I see is that parsing markdown is less straightforward
than it looks - have a look at https://github.github.com/gfm/
Also the features of the markdown format are not 1-to-1 compatible with the
features supported by the zim editor.
I uploaded a quick rundown of my notes here:
https://github.com/zim-desktop-wiki/zim-desktop-wiki/wiki/BlueprintMarkdown
On my side I still have to make up on the promise to hook up the config to
allow people to play around (step 1. in the plan at above link).
Regards,
Jaap
On Tue, Jun 9, 2020 at 10:17 AM Stephen Karl Larroque <
[email protected]> wrote:
Sorry Jaap that my message was stressful, it was not my intention but
rather the opposite, to try to propose a lesser amount of work for you and
on which others such as me can build upon.Yes i saw these progresses and that's awesome. I tried to look at the
codebase in the past when i worked on the Android draft, but Zim has become
so complex it's difficult for an external dev to pick it up without a
technical documentation. This only demonstrates the marvelous job you did,
and i can only commend you and thank you for that. I only wish i could help
more with PRs.Can you just tell me a quick rundown of the python files where the zim
markup is handled and I'll give it a try?Le mar. 9 juin 2020 Ã 07:31, Jaap Karssenberg notifications@github.com a
écrit :P.s. the latest release actually contains important steroid towards
markdown support. One is the support for overlapping formatting styles.
The
other is the url parsing which is more compatible. So there are actually
concrete steps towards making it easy to "just" add the parser.Op di 9 jun. 2020 07:11 schreef Jaap Karssenberg <
jaap.[email protected]:
It is also high on my personal wishlist, and I'm using a markdown
editor
myself on Android, so I'll get around to fix it eventually.However, it takes time. And one thing I simply don't have at the moment
is
lots of time.Op ma 8 jun. 2020 20:20 schreef Maykel Moya <[email protected]
:@jaap-karssenberg https://github.com/jaap-karssenberg since there
is
now a bounty, will you be willing to implement it? This is one of the
most
asked for and second oldest issue in the repo.Is there any hope you recover the markdown-support branch you once
shared?—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<https://github.com/zim-desktop-wiki/zim-desktop-wiki/issues/26#issuecomment-640792631
,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAJQYHVLMQRLTNSW2O3W3UDRVUTVRANCNFSM4DKZFM5A
>.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<
https://github.com/zim-desktop-wiki/zim-desktop-wiki/issues/26#issuecomment-641038581
,
or unsubscribe
<
https://github.com/notifications/unsubscribe-auth/AAIRFXVGFJJHTUMHGJ4H4QDRVXCM3ANCNFSM4DKZFM5A.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/zim-desktop-wiki/zim-desktop-wiki/issues/26#issuecomment-641115234,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAJQYHVDBL5KPS2CBNFAV4DRVXVYRANCNFSM4DKZFM5A
.
Thank you very much Jaap! I'll have a look and see what I can do!
No worries, I understand your situation, I have my share of
opensource projects I voluntary and freely maintain, Zim being so old it's
a marvelous job you do all those years! Plus the current world situation
adds to the regular stress and lack of time :-/
About the difficulties of writing a markdown parser, yes I am aware but
luckily in the past few years multiple very reliable editors and parsers
were made, so we may be able to build onto them instead of reinventing the
wheel (and having to deal with parsing issues ;-) ). My goal is to make the
basis for a partial syntax support, then I'm sure other developers will be
happy to jump on boat :-)
Take care and have a great week!
Best regards,
Stephen
Le mar. 9 juin 2020 Ã 21:27, Jaap Karssenberg notifications@github.com a
écrit :
@Stephen thanks for you support - sorry to get stressed out a bit at times
The key file to add the markdown parsing is "zim.formats.mardown" it should
contain a "Parser" and a "Dumper" class that transform markdown from text
to zim internal structure and back. Have a look at "zim.formats.wiki" for
example code.The main challenge I see is that parsing markdown is less straightforward
than it looks - have a look at https://github.github.com/gfm/Also the features of the markdown format are not 1-to-1 compatible with the
features supported by the zim editor.I uploaded a quick rundown of my notes here:
https://github.com/zim-desktop-wiki/zim-desktop-wiki/wiki/BlueprintMarkdownOn my side I still have to make up on the promise to hook up the config to
allow people to play around (step 1. in the plan at above link).Regards,
Jaap
On Tue, Jun 9, 2020 at 10:17 AM Stephen Karl Larroque <
[email protected]> wrote:Sorry Jaap that my message was stressful, it was not my intention but
rather the opposite, to try to propose a lesser amount of work for you
and
on which others such as me can build upon.Yes i saw these progresses and that's awesome. I tried to look at the
codebase in the past when i worked on the Android draft, but Zim has
become
so complex it's difficult for an external dev to pick it up without a
technical documentation. This only demonstrates the marvelous job you
did,
and i can only commend you and thank you for that. I only wish i could
help
more with PRs.Can you just tell me a quick rundown of the python files where the zim
markup is handled and I'll give it a try?Le mar. 9 juin 2020 Ã 07:31, Jaap Karssenberg notifications@github.com
a
écrit :P.s. the latest release actually contains important steroid towards
markdown support. One is the support for overlapping formatting styles.
The
other is the url parsing which is more compatible. So there are
actually
concrete steps towards making it easy to "just" add the parser.Op di 9 jun. 2020 07:11 schreef Jaap Karssenberg <
jaap.[email protected]:
It is also high on my personal wishlist, and I'm using a markdown
editor
myself on Android, so I'll get around to fix it eventually.However, it takes time. And one thing I simply don't have at the
moment
is
lots of time.Op ma 8 jun. 2020 20:20 schreef Maykel Moya <
[email protected]
:@jaap-karssenberg https://github.com/jaap-karssenberg since there
is
now a bounty, will you be willing to implement it? This is one of
the
most
asked for and second oldest issue in the repo.Is there any hope you recover the markdown-support branch you once
shared?—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<https://github.com/zim-desktop-wiki/zim-desktop-wiki/issues/26#issuecomment-640792631
,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAJQYHVLMQRLTNSW2O3W3UDRVUTVRANCNFSM4DKZFM5A
>
.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<https://github.com/zim-desktop-wiki/zim-desktop-wiki/issues/26#issuecomment-641038581
,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAIRFXVGFJJHTUMHGJ4H4QDRVXCM3ANCNFSM4DKZFM5A
>.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<
https://github.com/zim-desktop-wiki/zim-desktop-wiki/issues/26#issuecomment-641115234
,
or unsubscribe
<
https://github.com/notifications/unsubscribe-auth/AAJQYHVDBL5KPS2CBNFAV4DRVXVYRANCNFSM4DKZFM5A.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/zim-desktop-wiki/zim-desktop-wiki/issues/26#issuecomment-641523163,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAIRFXVUJPYQYHNKZ7CK2DTRV2ELZANCNFSM4DKZFM5A
.
From https://github.com/zim-desktop-wiki/zim-desktop-wiki/wiki/BlueprintMarkdown :
- What format to use for page links? Zim's internal link structure is well aligned with the rest of the application but will not be recognized by other MarkDown parsers. If converting to file links, some hint is needed on whether it was a relative or floating link. If we do it zim specific might prefix with e.g. "zim-page://" but this get's ugly quite quickly.
With all the talks about "https://github.github.com/gfm/", it makes sense to follow github-flavorness regarding wiki extensions too: https://docs.github.com/en/github/building-a-strong-community/editing-wiki-content . Which uses classic wiki [[PageName]] syntax, just the same as Zim.
A problem may pop up with hierarchical pages. Would be a good opportunity to think what makes more sense - [[Foo:Bar]] or [[Foo/Bar]]. (Github wiki doesn't "like" the latter (in a sense that it patches slash with a hyphen). Though it's classic MoinMoin/MediaWiki style. But it's more important how easy it will be to patch various mobile markdown viewers/editors.)
As many other folks, I'd like to see Zim to support Markdown. And I'd like to integrate "industrial grade" Markdown parser, but I'm currently effectively at the exploratory stage with Zim, and afraid to invest too much effort into it. At the same time, as I collect my stuff into Zim notebook (effectively plaintext) and format it lightly, I don't want "wrong" markup to be used for header or emphasis.
So, the idea to just patch up existing Zim parser/dumper comes along. It's actually pretty simple, and I have a strong suspicion that I'm missing somebody already have done that. Still, this ticket is the best resource on the matter I've found, and here people go to extravagant measures like on-the-fly integrating 3rd-party markup converters with Zim, and that's roughly a hack squared comparing to just patching Zim's own stuff.
I should emphasize there're 2 facets of "Markdown support":
p.1 is rather trivial, as experimenting shows. p.2 is much more complex, and rightfully leads to concerns from @jaap-karssenberg, as may lead to formatting loss, and worst-case data loss in existing content, and users will eat him in support requests.
But again, if you just want to be able to browse, and maybe create (simple) notes on your mobile device with existing Markdown viewers/editors (and sync with a desktop), it's all much more simpler, and can fast-track around the "full" support.
As a final note, while just unconditionally switching format to Markdown-like is indeed trivial and literally can be done in 15mins, that's far too cowboyish even for the kind of usage advocated here. Even if disregarding problems of people who have existing notebooks in Zim format, the builtin manual/help is in Zim format, and reading it in pretty way is helpful. So, supporting both Markdown'ish and classic format is a must even for a prototype.
I quickly hacked together all this - basics are working. Would need to actually test interoperability with existing Markdown tools and more advanced features.
Implementing zim-flavored-markdown may be a good chance to deal with #hashtag syntax for tags (https://github.com/zim-desktop-wiki/zim-desktop-wiki/issues/417).
Ok, just to trace my steps. Initially, I considered it beneficial to convert to markdown page by page, i.e., to allow same notebook to contains pages both in zim-wiki and markdown formats. So, my initial quick-hack implementation added a level of indirection inside existing zim.formats.wiki, which dispatched to either existing syntax parser/dumper, or to markdown'ish parser/dumper, which themselves were forks of existing parser/dumper.
As I mentioned, I'm just starting with Zim, so my notebook is light, and by now I converted all the pages to Markdown (and again, I'm just starting, so syntax subset I use is light, it's effectively just text-formatting markup). And looking at @jaap-karssenberg's commit linked above, https://github.com/zim-desktop-wiki/zim-desktop-wiki/commit/be4d741dc3e0d0777c3e1a7593a1d9a0d469495a , it contains everything needed to dispatch on notebook-global "format". And I already relied on it to handle .md extension instead of .txt.
So, that's how I refactored my patchings now. Just copied existing zim/formats/wiki.py under new format name, I chose zim_markdown.py for now (because once again, this is not even remotely like "full" or "compliant" markdown). Once initial, unchanged version was captured, I re-applied my previous patches. So yes, it's just one new file, an explicit copy of another one, to be able to pull in any changes from the original.
Then to use this new format, need to add following to notebook's notebook.zim file:
default_file_format=zim_markdown
default_file_extension=.md
My stuff is available at https://github.com/pfalcon/zim-desktop-wiki (branch pfalcon, set as default). Needless to say, it's all heavy WIP, not guaranteed to work as you expect, etc., etc. (The branch is also rebased, so use git pull --rebase to update.)
So, now with basics working, time to look at bigger and more subtle problems.
And the first problem is proverbial "line-based vs paragraph-based text" dichotomy. Let me remind that the whole purpose of Markdown so the files remained readable as plaintext. And plaintext is line-based. That means that you need to wrap your lines [manually] at suitable margin, and add blank lines between paragraphs (lists, etc.)
But Zim is effectively paragraph-based format, even if, I have to admit, plays really well (almost transparent) with line-based text. At least, if you have line-based .txt and drop it into your notebook, how it looks in Zim won't suck. But again, Zim is paragraph-oriented format, based solely on the fact that it's WYSIWYG-based: you type your text, it will be wrapped based on the width of editing pane, and when you hit Enter, you ended a paragraph and started a new one.
But remember that Markdown requires blank lines between block-level (paragraph-level) elements: you should have empty line after heading, between paragraph, before and after list, etc.
That means that currently, you need to add such extra lines in Zim. And that's effectively leaking of underlying format peculiarities into WYSIWYG nature of Zim. And what happens if you don't? Well, that's where the fun begins. Because authors of some Markdown parsers figured that humans beings love to err, and don't insert blank lines everywhere. So, they use a forgiving parsing method. While authors of other Markdown parsers are from another planet, not forgiving to us mere humans, and thus may not render the file like you expect. One of the drivers for this work was interoperability with existing Android Markdown readers, one prominent is Markor (https://github.com/jaap-karssenberg/zim-android-mockapp/issues/6), and it exactly seems to use non-forgiving parser.
Let's also come back to line wrapping - when I use Markdown, I expect my lines to be wrapped. Having a single 1000-char line for a paragraph is not acceptable. But that's exactly what Zim produces now, unless you wrap lines manually (which is again, weird thing to do in a WYSIWYG editor).
There're more subtle things. E.g., when you attach and insert a picture in Zim page, it's referenced as {{ pix.png }}. But it's stored actually in PageName/pix.png. Treating PageName/ as implicit may be ok for a magic proprietary format like zim-wiki, but not acceptable for open, tool-independent format like Markdown. It should be .
So, I don't know how to deal with all that, but it has to be dealt with, if Zim is to become a tool which deals with open-format user data, instead of magic-silo for user data.
FWIW, here is a ready-made repo with some BASIC TEST DATA (no images, etc).
https://github.com/rikai/OpenYourMouth
Another possible test corpus: https://github.com/daryllxd/lifelong-learning . Dunno how basic it is (seems to be).
Again, the whole idea behind Markdown support is to stop people dumping Zim because it can't work with standard formats. A typical case: https://animesh.blog/near-perfect-note-taking-software/
Zim
This is a very mature program and is available in apt. It ticked every checkbox except markdown format.
QOwnNotes
I did not like its webpage design then and do not like it now.
Guess what the guy selected? Of course QOwnNotes.
So again, need to actually test how Zim works on real-world Markdown dumps out there.
Again, the whole idea behind Markdown support is to stop people dumping Zim because it can't work with standard formats. A typical case: https://animesh.blog/near-perfect-note-taking-software/
Totally agree with you! Zim is such a marvellous piece of software, but the lack of markdown support is discouraging people to use (and actively contribute to) it.
@jaap-karssenberg wrote in https://github.com/zim-desktop-wiki/zim-desktop-wiki/issues/26#issuecomment-300757202
Probably also requires disabling some features in zim when source is markdown (e.g. insert of rich objects).
There's no need to disable anything. Just as currently Zim-wiki format uses image markup (with extra "type" query param) for objects, just the same Markdown could use image markup to store them, e.g.:
{{./diagram.png?type=diagram}} => .
Then 3rd-party Markdown viewers will just use pre-rendered image. (There could be a couple more steps to make sure that actually works with various viewers, but hopefully everyone agrees that the idea is simple and flexible.)
However, https://github.com/zim-desktop-wiki/zim-desktop-wiki/issues/841 puts the current smart, flexible, extensible, interoperable scheme under risk. Hopefully, it will be reconsidered.
So, with basic Markdown formatting working well, I'm looking to deal with images and links.
With all the talks about "https://github.github.com/gfm/", it makes sense to follow github-flavorness regarding wiki extensions too: https://docs.github.com/en/github/building-a-strong-community/editing-wiki-content . Which uses classic wiki [[PageName]] syntax, just the same as Zim.
Thinking about it, I guess that's not the right approach. While some (very few?) Markdown applications support [[PageName]] syntax, majority certainly don't, as it's not part of the standard Markdown. For example, I doubt that any Android app supports them, and interoperability on mobile is major driver for Markdown support in the first place.
So, the only right solution is to use standard Markdown links, [PageName](PageName.md). There're number of issues with it, and skipping purely technical ones (format parser/dumper need to have access to notebook Layout object), there's bigger conceptual issue with multitude of link types as supported by Zim: https://zim-wiki.org/manual/Help/Links.html#relative_links .
Of particular interest a concept of "dancing relative links". Quoting the above page:
For example you have a page "Zim:Examples:Linking:Relative" now you just use "Absolute" to link to "Zim:Examples:Linking:Absolute"
That is, if "Zim:Examples:Linking:Absolute" exists, because otherwise it can link to "Zim:Examples:Absolute", "Zim:Absolute", "Absolute", as the manual above explains.
So, to produce correct Markdown links, one needs to know not just static notebook layout rules (as encoded in the Layout object), but the actual structure of the notebook, which pages exist and which don't.
Even more interesting is that native Zim links need to be reconstructed from unfancy Markdown links on load.
So far, I don't know how to achieve all that, but will keep looking. (I noticed that for export, Zim has a Linker class which is supposedly deals with a problem of rendering static links from "dancing" links. But Linker is not available for normal save (aka "_store") page operation).
@pfalcon I skimmed your code and found one potential issue if the aim is compatibility with mobile Markdown editors.
Conventionally in Markdown, check lists are represented like this:
- [x] done
- [ ] not done
The same thing without quoting here on GitHub as an example:
This also affects Nextcloud Notes which is what I'm looking forward to being able to use in combination with Zim.
Of course, most of those tools won't support alternative signed for incomplete/moved, so you might still get things like:
Maybe just using a custom syntax and seeing this is overall better?
[*] done
[>] migrated
[ ] not done
(I see at least GitHub doesn't interpret this as a single paragraph, but I suspect some tools would)
OTOH, using the "standard" markdown syntax (prefixing with -) would let people that mostly limit themselves to checked/unchecked would get nicer interoperability. I'm leaning towards this being better for using it in conjunction with Nextcloud Notes, which is why I'm putting this out there.
@pfalcon I skimmed your code and found one potential issue
Thanks for looking! Well, at this time, there're many more issues actually. As I mentioned in my initial comments, I go slowly over Markdown features as I use them, and add support for them to my code. Per the worklog above, what's currently supported is basic inline/block-level formatting, and I'm currently working on polishing intrapage link handling (the code isn't even pushed fully to my branch, as it's too dirty).
Regarding checkbox lists, I'm aware of the issue, and going to tackle it when its turn comes, as I'm already using Zim Task plugin, and find it quite neat.
So, the only right solution is to use standard Markdown links, PageName.
Ok, so I pushed more or less brushed up code for that to my branch (https://github.com/pfalcon/zim-desktop-wiki/ ). The code is definitely not suitable for upstream. The main problem is that Zim current architecture has its markup parser/dumper interface is "context independent", i.e. its not aware of a particular page hosting a markup. But for a case of Markdown it's important, because links must be resolved and generated relative to the page.
So, I'm working around that by passing a Page object to parser/dumper. But then I also need notebook Layout object to perform actual link resolution. And some functionality lives in the Notebook object. So, obvious way is to just cross-reference a Page object to a god object of Notebook (which also contains Layout), and call it done. But turns out that Indexer passes to parser not a full Page, but merely a Path (which is a page Path, and not what you may have thought). And Indexer has access only to Layout, not a full Notebook. So, I need to pass multiple "stuffs" thru multiple layers. I also trying to minimize textual patches (so it was easier to maintain/less conflicts over time), so don't patch constructors, and just set object attributes. As you guessed it's a bit of messy, and while Zim definitely has an architecture, it's not the cleanest architecture out there either.
All in all, I'm using that code for more than week now, and it works for me. And I did't for example tested how links to Markdown documents outside a notebook would work.
These days also mark one month use of Zim to me. Good news is that I didn't give up on it ;-). Bad news is that novelty factor has largely worn off, and my hacking on it slows down. My next big task is migrating my old personal wiki from Trac to this new Zim-markdown (and further elaboration of Markdown support would be driven by that).
I skimmed your code and found one potential issue
I've created a ticket to track missing Markdown features: https://github.com/pfalcon/zim-desktop-wiki/issues/2 . Surprisingly, out of top of my today's head, I added only a very few items there. That's of course why I created it - because I'm getting out of that context and forgetting things. Additions are welcome.
Context: https://github.com/zim-desktop-wiki/zim-desktop-wiki/issues/26#issuecomment-699752776
OTOH, using the "standard" markdown syntax (prefixing with -)
Note about this: for me, "standard" markdown syntax is one which uses * as a bullet ;-). At least, that's what I use ;-). It should be possible to support - for parsing (no ambiguity for something else), but then it will be converted to * on saving. Heck, it actually should be possible to actually preserve the actual bullet character, as Zim AST supports arbitrary attributes on nodes, but who would do all that? ;-)
Markdown, check lists are represented like this:
- [x] done
- [ ] not done
[*] done
[>] migrated
[ ] not done
I just noticed that for Markdown, done is [x], while for Zim, [*], and for Zim, [x] is "cancelled" or whatever (red X-cross in UI). So, sadly it's not possible to use the same set of checkbox syntaxes for zim_markdown, effectively supporting superset of Markdown checkbox states, due to this ambiguity with [x]. Thus, I think that it makes sense to recognize only [ ] and [x] as checkboxes, and treat anything else literally (kinda like Github).
Expressiveness wise, I don't think it's a big loss. If an item is cancelled, then it's natural to just strike it thru (and mark as done or remove checkbox). For migrated/moved, it's natural to well, just move it elsewhere, or strike thru similarly and add (moved). And that's of course not talking about option of keep writing it like * [>] which will be rendered as a normal bullet with literal [>].
(Semi-related note, I'm not happy how monospace aka literally-quoted text is rendered in Zim. I guess, I got spoiled by Github and its grey background. Just noticed that there now seem to be round corners in this grey highlight. Bastards, I don't like that :-F.)
I just noticed that for Markdown, done is [x], while for Zim, [*], and for Zim, [x] is "cancelled"
Sadly, that also means that "automigrate" approach (parse both Zim and Markdown syntax, dump in Markdown) won't work. Of course, it would be possible to make it work, but why bother to write code which will be removed soon anyway, if I have only one pageful of checkboxed tasks.
More general migration handling would need to be worked out (by somewhom, somewhen).
So, sadly it's not possible to use the same set of checkbox syntaxes for zim_markdown, effectively supporting superset of Markdown checkbox states
So, down with doubts, I just render "migrated" as * [>] (like before), and "X-cross" as * [#] (Also had ideas of [-], [~], [c], but IMHO [#] conveys the idea that it's stroked in and out better). Other Markdown tools should just display them literally, so it's up to use to whether use them in Zim or not (I found I used X-cross on one occasion).
With the above notes in mind, the * [ ] Markdown checkbox list syntax is now supported in my branch, https://github.com/pfalcon/zim-desktop-wiki
Paul,
Thanks a lot for your contribution. It's a long time ago ( May 10, 2017) that I started this issue!
In your opinion, is your branch something that a simple user could use/test? If so, how should we install it?
Can we try it along with the "official" version? Should we try on a different directory tree?
Best,
Agus
Thanks.
In your opinion, is your branch something that a simple user could use/test?
Test, yes. I'd appreciate any feedback, it may help to direct/plan/allocate resources for further work (but I can't promise to do anything beyond my own needs).
Use - not on any production (i.e. important to you) data. (Unless you're ready to maintain the code further, as I'm doing that at a pace and to an extent suitable to me.)
If so, how should we install it?
I'm on Linux, so I just git clone and run "zim.py".
Can we try it along with the "official" version?
I never "installed" official version, so don't know if there can be any "conflicts" between the two. (I wouldn't think there would be much, but as I didn't try, that's what I'm saying).
Should we try on a different directory tree?
Definitely, my branch should be used with a separate notebook, which is already in Markdown format. And so far it was tested with just my personal notebook.
May be do it by special plugin - not everyone uses this format ?
May be do it by special plugin - not everyone uses this format ?
Everyone uses this format, that's the point. That's why I use it for example, if everyone used some other format, I'd use it instead either.
And yes, as soon as @jaap-karssenberg will support wiki formats as plugins, this stuff can be converted to it. So far, my implementation relies on a commit from dev branch for more or less clean implementation at all (and stuff in dev branch isn't really perfect).
Happy New Year! Just a usual reality check that I'm still using my Markdown branch. By now Zim almost completely merged into my background - I use some pages in my notebook daily, and forgot to update others (like daily journal), the usual personal wiki things.
As it's now "background" thing, I didn't do much hacking on it, which is clear sign that Markdown support works reasonably well (bunch of small things are missing, just the same as Zim itself misses tons of small things). I just pushed a rebase which picks up recent bugfixes from master branch, and committed 2 cosmetic changes:
#section). Like, make the thing to not poo on them completely, but otherwise support for them is far from acceptable, on the Zim side of things. I hope https://github.com/zim-desktop-wiki/zim-desktop-wiki/pull/1293 will tackle that.The branch rebased on master (0.73.5).
"By popular demand", I added detailed instructions to the fork's README on how to create a Markdown notebook: https://github.com/pfalcon/zim-desktop-wiki/blob/pfalcon/README.md . I'd like to emphasize again that support for native Zim format is fully retained, which format to use is per-notebook setting. Well, to an extent. For example, the system-default page template was switched to Markdown syntax (I by now don't remember if there can be per-notebook page templates; if not, it may be worth to add support for them).
The only "pressing" issue is that multi-level lists use tab-indentation (as the original Zim format), whereas Markdown supports space-indentation (so, if you have existing Markdown files with multi-level lists, they might not render properly). I'll get to fixing that one of these months. Beyond that, there may be a bunch of little things too of course (which don't affect me so far, so I don't care much).
Most helpful comment
A couple things: If we can get markdown tightly integrated the result is basically numerous mobile and cloud hosted apps already built. 1) This solves the lack of a native android app. 2) This allows cross platform collaboration easily -- gf doesnt wanna install zim on her mac and to be fair it is kinda sluggish. 3) as a result of the former 2, efforts can be focused on things that really matter: linux dev.
Also found this. Might make our job a bit easier. https://github.com/Xunius/markdown2zim