Hugo: ENH: custom markdown processor

[sglyon]

I love how easy Hugo makes it to organize and display my content.

I am used to using pandoc to convert my markdown to various formats and would love to use it within Hugo. Is there any way we could shell out to a custom markdown processor within Hugo?

I realize that this might defeat a lot of the speed gains we get by using Hugo, but it would also add some flexibility for those who are willing to sacrifice a little performance.

[spf13]

I’m sure it’s possible, but would require some code changes to do it. Someone contributed some preliminary support for rst via an external mechanism. It would probably be valuable to others to generalize this somehow. Not sure what makes sense here. 

Hugo is already written in such a way to add support for a bunch of different formats, but the only processor go has that I could find was markdown.

Best,
Steve

-- 
Steve Francia
http://stevefrancia.com
http://spf13.com
http://twitter.com/spf13

From: Spencer Lyon [email protected]
Reply: spf13/hugo [email protected]
Date: March 20, 2014 at 3:13:27 PM
To: spf13/hugo [email protected]
Subject:  [hugo] ENH: custom markdown processor (#234)

I love how easy Hugo makes it to organize and display my content.

I am used to using pandoc to convert my markdown to various formats and would love to use it within Hugo. Is there any way we could shell out to a custom markdown processor within Hugo?

I realize that this might defeat a lot of the speed gains we get by using Hugo, but it would also add some flexibility for those who are willing to sacrifice a little performance.

—
Reply to this email directly or view it on GitHub.

[abh]

I don't think it'd be sacrificing performance so much -- the input could be cached. I've been using pandoc to convert old content to markdown for hugo use (though not deployed yet; there were some bugs in pandoc dealing with some of my textile content).

We could cache the pandoc output and only re-generate it if the source changes. It's a bit ugly but there's a lot done in pandoc that I don't see anyone having enough interest to re-implement in Go.

[zackp30]

It would be great to see Pandoc (and others) supported in Hugo!

[OJFord]

This would be fantastic; I think it's worth pointing out that markdown isn't any one Markdown^TM - there must be hundreds of variations that build on the basic Gruber syntax.

Pandoc is probably the most versatile (?) implementation in both the senses that 'pandoc markdown' permits a larger syntax (code, tables, amsmath) than anything else I've seen - but also that it's completely customisable. You can start with markdown_strict (as it calls it) and add features to it.

Anything else isn't saying so much "you can create content in markdown" as it is "you can create content in this markdown".

I think a really nice solution would be to have a config file something like:

[renderers]
    mymarkdown = pandoc --options
    latex = pandoc --different-options
    myownlanguage = sometooliwrote

and then being able to run hugo --use-mymarkdown et al.

[OJFord]

Also in Pandoc's favour - the workaround for using _ in math in Hugo's markdown (to have it not begin emphasis) is absurd, and should be considered a bug rather than just the way it is.

Wrap my $$ in <div> so it's rendered verbatim? Pandoc doesn't need that, it knows what to do if tex_math_dollars are enabled (they are in default pandoc markdown, or else it's --from=markdown_strict+tex_math_dollars).

[novelistparty]

👍

[Vonng]

It's a pity that hugo needs workaround on handling math blocks.
<div> and `` doesn't needed on other markdown editors (Typora, MWeb,....) or render(pandoc). Lots of math content broke and needs another converter before working around with hugo.

If hugo can't handle this properly, at least there's could be a switch that disable _ \ interpretation during markdown processing.

[lsaravia]

So it is no easy way to add pandoc? could you point out where have to make the changes? because it seems that the only solution to use heavy math is by using pandoc.

[sjcjoosten]

Seems that #4060 is about to fix this issue in 'a couple of days'.

[kaushalmodi]

@OJFord @Vonng @lsaravia Can you provide example heavy math content? I am the maintainer of Org to Hugo exporter for Emacs (ox-hugo), and would like to see if those math examples get exported fine using the Org flow.

• Currently tested math support

[lsaravia]

I don't remember exactly because I switched to Jekyll, but I guess that this markdown is a good example https://github.com/lsaravia/MultifractalsInR/blob/master/Curso3.md

[kaushalmodi]

@lsaravia Thanks! That example post worked just fine with my flow :)

Org source -> Exported Hugo-compatible Markdown -> Hugo generated HTML

[lsaravia]

mmmm is not working very well, the symbol (R) should not be in the equations :(

[Vonng]

@kaushalmodi Heres an example of rich math content: Neural Network Notes

Actually I solved this by forking a copy of blackfriday and add MathJax support to it. Commit

And corresponding patch to Hugo is Here

Now it work well for me.

[kaushalmodi]

@lsaravia Well the smartypants Blackfriday extension is over smart (https://github.com/russross/blackfriday/issues/411). I should be able to put a hack in ox-hugo to outsmart that.

@Vonng You seem to be a serious Go coder (I haven't yet coded in Go). Have you considered submitting a PR for better MathJax support to Blackfriday?

[Vonng]

@kaushalmodi PR created.

• Add Math support (MathJaX) to blackfriday.v2

[kaushalmodi]

@lsaravia First of all, why wouldn't you use (x) like normal folks and use (r) instead /sarcasm :)

In any case, if you ever use Hugo again, that too, with ox-hugo, this issue is now solved.

The trick was simple (even if you don't understand elisp, the comment there should explain the trick), that now transforms the earlier exported multifractals-in-ecology-using-r.md as follows (I also fixed a typo manually in there). As a result, now the (r) do not get converted to ® by Blackfriday [See].

(.. and a bonus test).

Let me know if you still see issues in there.

It's strange and wonderful, you helped fix a bug in something you don't even use :). Thank you.

[Vonng]

@kaushalmodi markdown feature won't effect in math block, so (x) (r) is ok in equations.
Besides, there's an option in blackfriday API that can disable smartypants (actually not that smart and necessary...)
hugo should have such options somewhere, too.

[kaushalmodi]

@Vonng

so (x) (r) is ok in equations.

Of course it should be OK (I had put a /sarcasm tag there :)). The problem is that Blackfriday infers the (r) even in equations.

there's an option in blackfriday API that can disable smartypants

I am aware of that, but I don't want to disable the whole of smartypants (it does a lot of things). I want to disable just smartParens component of smaryPants and only in equations. If you click the link I just referred, you'll see that there's no way to disable just smartParens.

hugo should have such options somewhere, too.

Hugo does have an option to disable things that can be disabled (like SMARTYPANT_FRACTIONS, SMARTYPANTS_DASHES, etc. see Blackfriday Options in Hugo documentation) because Blackfriday has provided an API for that.. but nothing similar exists on the Blackfriday side to disable just smartParens.

[lsaravia]

@kaushalmodi I don't like x-files so I never use x :), many people helped me selflessly so I try to contribute back...

[kaushalmodi]

@Vonng

Heres an example of rich math content: Neural Network Notes

Thanks! That was a great example! I got to fix few more things while making that example work.

The only hurdles I faced while porting your Markdown example to Org were:

• Org emphasis (bold) characters need to have a space before and a space after. So strings like 特性：*光滑可微*，形 had to be converted to 特性： *光滑可微* ，形.
• Just as with the emphasis characters, Org needs the inline equations to also have a space before and a space after.. So strings like 层第$k$号神 had to be converted to 层第 $k$ 号神.
• Org supports embedded LaTeX environments.. so blocks like \begin{align} ..\end{align} do not need to be (should not be) wrapped by $$.

That's all I needed after converting your Markdown to Org using pandoc. I believe that your example renders fine now.. see the Hugo generated HTML linked below.

Org source -> Exported Hugo-compatible Markdown -> Hugo generated HTML

Looking forward to comments :)

[bep]

Please use https://discourse.gohugo.io/ for discussions.