Pelican: => characters in a code block are rendered as =>

[gsauthof]

After upgrading from Pelican 3.7.1 to 4.0.1 an old markdown article is rendered incorrectly.

The markdown article contains a code block like this:

blah blah, i.e.:

    :::text
    147362896070 45458.422057 1000 / / 10 9 ^ / p
    => 3.241 GHz

more text

With 3.7.1 the resulting html for the code block is:

<div class="highlight"><pre><span></span>147362896070 45458.422057 1000 / / 10 9 ^ / p
=&gt; 3.241 GHz
</pre></div>

Now, with 4.0.1 it is:

<div class="highlight"><pre><span></span>147362896070 45458.422057 1000 / / 10 9 ^ / p
=&amp;gt; 3.241 GHz
</pre></div>

Somehow, pelican now html-lizes the =>two times. That means in a browser the code is displayed incorrectly like this:

147362896070 45458.422057 1000 / / 10 9 ^ / p
=> 3.241 GHz

Expected output:

147362896070 45458.422057 1000 / / 10 9 ^ / p
=> 3.241 GHz

How to fix/work-around this?

[avaris]

I don't remember any change in pelican code to affect this. Are your other packages same? Especially Markdown?

[gsauthof]

I upgraded Pelican after switching from Fedora 27 to 29 - i.e. this is a switch from Python 3.6 to Python 3.7.

On both systems, I installed Pelican in a virtual env, via pip.

With Pelican 3.7.1 I used:

Markdown-2.6.8
MarkupSafe-1.0

Now pip fetched:

Markdown-3.0.1
MarkupSafe-1.1.0

[gsauthof]

After downgrading Markdown with

pip install -U Markdown==2.6.8

the output is properly quoted, again. Thus, this is a valid work-around.

[justinmayer]

A known issue, and an annoying one. I fixed this in Travis via https://github.com/getpelican/pelican/commit/08ae2ef36128a7f719724cb595b53f328c3e60f3, but that doesn't solve things for end-user installation. Not sure there's anything to be done here until the next Python-Markdown release, which I've encouraged but not yet successfully elicited.

[dbrgn]

On pelican 3.7.1 with latest Markdown (3.0.1) I noticed that while the indent syntax results in double-escaped brackets:

    hello <world>

...this does not:

```
hello <world>
```

Might be a workaround for some in the meantime.

[ekaitz-zarraga]

Upgrading to Python-Markdown 3.1 solves the issue:

pip install -U markdown

[justinmayer]

I un-pinned Python-Markdown in requirements/test.pip which, as expected, causes the test_article_with_footnote test to fail because the default separator has been changed from a hyphen to a colon as of Python-Markdown 3.x. See related discussion: https://github.com/Python-Markdown/markdown/issues/723

I'd like to solicit input regarding how we should resolve this. Should we adjust the test output to use a colon? Or use the new configuration option in Python-Markdown 3.1 to use a hyphen as the footnote separator?

[avaris]

I'd prefer using the new config but it's slightly annoying. Markdown doesn't like extension configs it doesn't recognize. Below is the best I could come up to make it work in markdown 2 and 3:

     def test_article_with_footnote(self):
-        reader = readers.MarkdownReader(settings=get_settings())
+        settings = get_settings()
+        # Markdown 3.1 introduced SEPARATOR argument and
+        # you can only set a config if it is recognized
+        from markdown.extensions.footnotes import FootnoteExtension
+        FE = FootnoteExtension()
+        if 'SEPARATOR' in FE.config:
+            ec = settings['MARKDOWN']['extension_configs']
+            ec['markdown.extensions.footnotes'] = {'SEPARATOR': '-'}
+        reader = readers.MarkdownReader(settings)

[justinmayer]

@avaris: Nicely done. Would you be willing to implement a fix based on your work above? I agree Python-Markdown's fussiness here is a bit annoying, but for now your proposed solution is probably better than to perpetually pin it at < 3.0. 😉

[justinmayer]

Follow-up question... Should we pin Python-Markdown at >3.0 instead? Are there good reasons to support both 2.x and 3.1+ at this point?

[avaris]

I'm OK with making it 3.1+ only.

[justinmayer]

Me too. I've pinned markdown to >= 3.1 in requirements/test.pip. Not sure there's anything else to be done here except for perhaps updating the Pelican 4.0 release post, so I'm considering this issue closed.

[justinmayer]

Hehe. Just kidding. Of course the test output also needs to be changed, or we need to adjust the test to use the hyphen separator instead of the new default colon. 😅

[justinmayer]

I took a shot at adjusting the test based on the work by @avaris above. Many thanks, Deniz!