October: Footnote html isn't escaped in markdown rendering

[Hambern]

It seems like somethings wrong with the markdown editor. Footnotes aren't rendered correctly anymore. The html is created, but it's escaped and doesn't respond to adding |raw to the |md.

Expected behavior

I want to add a footnote

Actual behavior

It adds a buch of html that does not go away with adding |raw to the content|md .

Reproduce steps

Create a markdown text with a footnote.

[LukeTowers]

@Hambern are you able to dig into exactly where the issue occurs? Also, does adding content | raw | md work if you put it in that order instead?

[Hambern]

Can’t look into it right now. But i tried that also: no luck. Can check what’s Up tonight and maybe suggest a sullotion :)

[FlusherDock1]

Can't reproduce it in markdown editor In rainlab.blog. can you paste your example?

[Hambern]

Well it is for me... Strange... I've tried a lot of different stuff.

This is the blogg post:

# Quisque Eget Odio Ac Lectus Vestibulum

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nullam in dui mauris. Vivamus hendrerit arcu sed erat molestie vehicula. Sed auctor neque eu tellus rhoncus ut eleifend nibh porttitor. Ut in nulla enim. Phasellus molestie magna non est bibendum non venenatis nisl tempor. Suspendisse dictum feugiat nisl ut dapibus. Mauris iaculis porttitor posuere. Praesent id metus massa, ut blandit odio. Proin quis tortor orci.[^1] Etiam at risus et justo dignissim congue. Donec congue lacinia dui, a porttitor lectus condimentum laoreet. Nunc eu ullamcorper orci. Quisque eget odio ac lectus vestibulum faucibus eget in metus. In pellentesque faucibus vestibulum. Nulla at nulla justo, eget luctus tortor. Nulla facilisi. Duis aliquet egestas purus in blandit. Curabitur vulputate, ligula lacinia scelerisque tempor, lacus lacus ornare ante, ac egestas.[^2]

[^1]: http://www.testingalink.com
[^2]: This is an ordinary note

Then I've tried all of these:

{{ post.content|md|raw }}
{{ post.content|md }}
{{ post.content_html }}

They all produce the same output:

Quisque Eget Odio Ac Lectus Vestibulum
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nullam in dui mauris. Vivamus hendrerit arcu sed erat molestie vehicula. Sed auctor neque eu tellus rhoncus ut eleifend nibh porttitor. Ut in nulla enim. Phasellus molestie magna non est bibendum non venenatis nisl tempor. Suspendisse dictum feugiat nisl ut dapibus. Mauris iaculis porttitor posuere. Praesent id metus massa, ut blandit odio. Proin quis tortor orci.1 Etiam at risus et justo dignissim congue. Donec congue lacinia dui, a porttitor lectus condimentum laoreet. Nunc eu ullamcorper orci. Quisque eget odio ac lectus vestibulum faucibus eget in metus. In pellentesque faucibus vestibulum. Nulla at nulla justo, eget luctus tortor. Nulla facilisi. Duis aliquet egestas purus in blandit. Curabitur vulputate, ligula lacinia scelerisque tempor, lacus lacus ornare ante, ac egestas.2

1. <p><a href="http://www.testingalink.com">http://www.testingalink.com</a>&#160;<a href="#fnref1:1" rev="footnote" class="footnote-backref">&#8617;</a></p>
2. <p>This is an ordinary note&#160;<a href="#fnref1:2" rev="footnote" class="footnote-backref">&#8617;</a></p>

[FlusherDock1]

this is what I get via |raw

<h1>Quisque Eget Odio Ac Lectus Vestibulum</h1>
<p>Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nullam in dui mauris. Vivamus hendrerit arcu sed erat molestie vehicula. Sed auctor neque eu tellus rhoncus ut eleifend nibh porttitor. Ut in nulla enim. Phasellus molestie magna non est bibendum non venenatis nisl tempor. Suspendisse dictum feugiat nisl ut dapibus. Mauris iaculis porttitor posuere. Praesent id metus massa, ut blandit odio. Proin quis tortor orci.<sup id="fnref1:1"><a href="#fn:1" class="footnote-ref">1</a></sup> Etiam at risus et justo dignissim congue. Donec congue lacinia dui, a porttitor lectus condimentum laoreet. Nunc eu ullamcorper orci. Quisque eget odio ac lectus vestibulum faucibus eget in metus. In pellentesque faucibus vestibulum. Nulla at nulla justo, eget luctus tortor. Nulla facilisi. Duis aliquet egestas purus in blandit. Curabitur vulputate, ligula lacinia scelerisque tempor, lacus lacus ornare ante, ac egestas.<sup id="fnref1:2"><a href="#fn:2" class="footnote-ref">2</a></sup></p>
<div class="footnotes">
<hr>
<ol>
<li id="fn:1">
<p><a href="http://www.testingalink.com">http://www.testingalink.com</a>&nbsp;<a href="#fnref1:1" rev="footnote" class="footnote-backref">↩</a></p>
</li>
<li id="fn:2">
<p>This is an ordinary note<a href="#fnref1:2" rev="footnote" class="footnote-backref">↩</a></p>
</li>
</ol>

[Hambern]

{{ content|raw }}

...results in...

# Quisque Eget Odio Ac Lectus Vestibulum

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nullam in dui mauris. Vivamus hendrerit arcu sed erat molestie vehicula. Sed auctor neque eu tellus rhoncus ut eleifend nibh porttitor. Ut in nulla enim. Phasellus molestie magna non est bibendum non venenatis nisl tempor. Suspendisse dictum feugiat nisl ut dapibus. Mauris iaculis porttitor posuere. Praesent id metus massa, ut blandit odio. Proin quis tortor orci.[^1] Etiam at risus et justo dignissim congue. Donec congue lacinia dui, a porttitor lectus condimentum laoreet. Nunc eu ullamcorper orci. Quisque eget odio ac lectus vestibulum faucibus eget in metus. In pellentesque faucibus vestibulum. Nulla at nulla justo, eget luctus tortor. Nulla facilisi. Duis aliquet egestas purus in blandit. Curabitur vulputate, ligula lacinia scelerisque tempor, lacus lacus ornare ante, ac egestas.[^2]

[^1]: http://www.testingalink.com
[^2]: This is an ordinary note

but...

{{ content|md|raw }}

...results in...

<h1>Quisque Eget Odio Ac Lectus Vestibulum</h1>
<p>Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nullam in dui mauris. Vivamus hendrerit arcu sed erat molestie vehicula. Sed auctor neque eu tellus rhoncus ut eleifend nibh porttitor. Ut in nulla enim. Phasellus molestie magna non est bibendum non venenatis nisl tempor. Suspendisse dictum feugiat nisl ut dapibus. Mauris iaculis porttitor posuere. Praesent id metus massa, ut blandit odio. Proin quis tortor orci.<sup id="fnref1:1"><a href="#fn:1" class="footnote-ref">1</a></sup> Etiam at risus et justo dignissim congue. Donec congue lacinia dui, a porttitor lectus condimentum laoreet. Nunc eu ullamcorper orci. Quisque eget odio ac lectus vestibulum faucibus eget in metus. In pellentesque faucibus vestibulum. Nulla at nulla justo, eget luctus tortor. Nulla facilisi. Duis aliquet egestas purus in blandit. Curabitur vulputate, ligula lacinia scelerisque tempor, lacus lacus ornare ante, ac egestas.<sup id="fnref1:2"><a href="#fn:2" class="footnote-ref">2</a></sup></p>
<div class="footnotes">
<hr>
<ol>
<li id="fn:1">
&lt;p&gt;&lt;a href="http://www.testingalink.com"&gt;http://www.testingalink.com&lt;/a&gt;&amp;#160;&lt;a href="#fnref1:1" rev="footnote" class="footnote-backref"&gt;&amp;#8617;&lt;/a&gt;&lt;/p&gt;
</li>
<li id="fn:2">
&lt;p&gt;This is an ordinary note&amp;#160;&lt;a href="#fnref1:2" rev="footnote" class="footnote-backref"&gt;&amp;#8617;&lt;/a&gt;&lt;/p&gt;
</li>
</ol>
</div>

As you can see... the paragraph and link tags are escaped...

[Hambern]

I'm using build 436 and blog version 1.2.19

[Hambern]

I also use it in a partial if that's any kind of help

[LukeTowers]

@Hambern is this still an issue?

[Hambern]

Yes, but havent had the time to look into it further

[vosco88]

Still an issue, I just tried it.

[nathanlesage]

I can confirm that with multiple sites this happens to me as well! It's definitely an issue within the parser, because using the raw Markdown and then parsing it using the builtin Markdown parser yields the same as directly using the content_html: escpaed footnote reference HTML.

It definitely doesn't have to do anything with whether or not it's a partial or not, because the auto-generated HTML (content_html) is generated directly in the Post-Controller, which means there is definitely something going on with the Markdown parser.

[LukeTowers]

@nathanlesage could you look into it further and see what's causing it?

[nathanlesage]

@LukeTowers Okay, OctoberCMS can ease up, it's not inside the source code of October.

The issue arose in March (!) on the repository of Parsedown, currently there are several open issues pertaining that bug:

• https://github.com/erusev/parsedown/issues/654 (closed by now, tracked in the 118 of parsedown-extra)
• https://github.com/erusev/parsedown-extra/issues/118

Unfortunately, they still haven't fixed this thing, because they are currently working on an AST, as far as I understand. This means: Still have to wait. They have committed a suitable bug fix on July 24th, but since then I can't make out any progress.

Yet, for all who need this to work _now_, here's the more-than-ugly monkey patch:

1. Find all files, where your posts are displayed.
2. Add this to the Code section:

function onEnd() {    
    // Ugliest monkey patching ever
    $this->post->content = \Markdown::parse($this->post->content);
    $cnt = explode('<div class="footnotes">', $this->post->content);
    $cnt[1] = html_entity_decode($cnt[1]);
    $this->post->content = implode('<div class="footnotes">', $cnt);
}

1. In your file use the raw filter: {{ post.content|raw }}
2. Hope there's no malicious code in your footnote area and pray to whatever you believe in.

[github-actions[bot]]

This issue will be closed and archived in 3 days, as there has been no activity in the last 30 days. If this issue is still relevant or you would like to see action on it, please respond and we will get the ball rolling.

[nathanlesage]

To prevent this from closing, I had another look at the corresponding issue (erusev/parsedown-extra#118). Apparently, they're close to solving this. There's a PR on the repository (https://github.com/erusev/parsedown/pull/708) that seems to also include a fix to this problem, so at least from the looks of it it seems to be coming to a close!

[LukeTowers]

@w20k I took the In progress label off so that the bot will revisit this in 30 days and we can check in on the status then.

[w20k]

Good idea @LukeTowers !

[github-actions[bot]]

This issue will be closed and archived in 3 days, as there has been no activity in the last 30 days. If this issue is still relevant or you would like to see action on it, please respond and we will get the ball rolling.

[bennothommo]

Looks like it is still in progress.

[github-actions[bot]]

This issue will be closed and archived in 3 days, as there has been no activity in the last 30 days. If this issue is still relevant or you would like to see it actioned, please respond and we will re-open this issue.

[aidantwoods]

Hi, I'm the author of the PR referenced earlier (https://github.com/erusev/parsedown/pull/708). The main reason this hasn't been merged is, despite it being left open for a while and referencing it in quite a few issues as a proposal to solve some of them, I'm struggling to get any feedback on it. It represents a substantial set of changes to a fairly popular library (which I didn't myself originally author), so I'm hesitant to just push the changes through without any discussion.

If anyone here wants to help, I'd really appreciate a review of that PR (even from something as simple as a usability perspective).

[bennothommo]

@aidantwoods Thanks for getting in touch. I will strive to take a look at your PR within the next couple of days and offer some feedback if I have any. :)

[github-actions[bot]]

This issue will be closed and archived in 3 days, as there has been no activity in the last 30 days.
If this issue is still relevant or you would like to see it actioned, please respond and we will re-open this issue.
If this issue is critical to your business, consider joining the Premium Support Program where a Service Level Agreement is offered.

[github-actions[bot]]

This issue will be closed and archived in 3 days, as there has been no activity in the last 30 days.

• If this issue is still relevant or you would like to see it actioned, please respond and we will re-open this issue. - If this issue is critical to your business, consider joining the Premium Support Program where a Service Level Agreement is offered.

[github-actions[bot]]

This issue will be closed and archived in 3 days, as there has been no activity in the last 30 days.
If this issue is still relevant or you would like to see it actioned, please respond and we will re-open this issue.
If this issue is critical to your business, consider joining the Premium Support Program where a Service Level Agreement is offered.

[github-actions[bot]]

This issue will be closed and archived in 3 days, as there has been no activity in the last 30 days.
If this issue is still relevant or you would like to see it actioned, please respond and we will re-open this issue.
If this issue is critical to your business, consider joining the Premium Support Program where a Service Level Agreement is offered.

[nathanlesage]

Just pinged @aidantwoods and asked how this issue is proceeding on his side!

[bennothommo]

@aidantwoods I sincerely apologise for the lack of movement on me giving your 2.0.0 PR a review. I've had a brief look at what you've done so far - it looks great, but given how substantial the change is, I want to look into it more before I offer any feedback. I will also be testing this with our usage of the library in the Markdown Editor widget in October CMS and see what ramifications (if any) it has on that. I will be in touch.

[github-actions[bot]]

This issue will be closed and archived in 3 days, as there has been no activity in the last 60 days.
If this issue is still relevant or you would like to see it actioned, please respond and we will re-open this issue.
If this issue is critical to your business, consider joining the Premium Support Program where a Service Level Agreement is offered.

[github-actions[bot]]

This issue will be closed and archived in 3 days, as there has been no activity in the last 60 days.
If this issue is still relevant or you would like to see it actioned, please respond and we will re-open this issue.
If this issue is critical to your business, consider joining the Premium Support Program where a Service Level Agreement is offered.

[github-actions[bot]]

This issue will be closed and archived in 3 days, as there has been no activity in the last 60 days.
If this issue is still relevant or you would like to see it actioned, please respond and we will re-open this issue.
If this issue is critical to your business, consider joining the Premium Support Program where a Service Level Agreement is offered.

[github-actions[bot]]

This issue will be closed and archived in 3 days, as there has been no activity in the last 60 days.
If this issue is still relevant or you would like to see it actioned, please respond and we will re-open this issue.
If this issue is critical to your business, consider joining the Premium Support Program where a Service Level Agreement is offered.