Core: [subscriptions] Email content corrupted after a [code] block

Created on 7 Apr 2016 · 20Comments · Source: flarum/core

This bug was tested locally and also on the sandbox of discussion.flarum.org.

Simple to reproduce :

Follow a discussion
ask someone to post a message with :
[code]OK[/code]

or anything which contains a code block.

See example here : https://discuss.flarum.org/d/2451-lol/9

In the email received to inform that there was a new post, the text is correct until the end of the code block. Immediately after, there is a sequence of a js file (always the same sequence exactly, even after recompiling). The text following the code block is not present in the email.

In the example on the sandbox, here is the email I received :

Hey Averell!

datitisev made a post in a discussion you're following: Lol

To view the new activity, check out the following link:
https://discuss.flarum.org/d/2451/9

---

Okif(&quot;undefined&quot;!==typeof hljs)hljs._ha();else if(&quot;undefined&quot;===typeof hljsLoading){hljsLoading=1;var a=document.getElementsByTagName(&quot;head&quot;)[0],e=document.createElement(&quot;link&quot;);e.type=&quot;text/css&quot;;e.rel=&quot;stylesheet&quot;;e.href=&quot;//cdnjs.cloudflare.com/ajax/libs/highlight.js/9.0.0/styles/default.min.css&quot;;a.appendChild(e);e=document.createElement(&quot;script&quot;);e.type=&quot;text/javascript&quot;;e.onload=function(){var d={},f=0;hljs._hb=function(b){b.removeAttribute(&quot;data-hljs&quot;);var c=b.innerHTML;c in d?b.innerHTML=d[c]:(7

---

You won't receive any more notifications about this discussion until you're up-to-date.

The same thing happens if you create a code block by an indent (four spaces).

typbug

Source

Averell7

All 20 comments

The actual JavaScript isn't like that, because it is plain text, so the HTML hasn't been parsed.
This is the actual script, all I did was parse it:

if("undefined"!==typeof hljs)hljs._ha();else if("undefined"===typeof hljsLoading){hljsLoading=1;var a=document.getElementsByTagName("head")[0],e=document.createElement("link");e.type="text/css";e.rel="stylesheet";e.href="//cdnjs.cloudflare.com/ajax/libs/highlight.js/9.0.0/styles/default.min.css";a.appendChild(e);e=document.createElement("script");e.type="text/javascript";e.onload=function(){var d={},f=0;hljs._hb=function(b){b.removeAttribute("data-hljs");var c=b.innerHTML;c in d?b.innerHTML=d[c]:(7

If I use a beautify-er, the code now looks like this:

if ("undefined" !== typeof hljs) hljs._ha();
else if ("undefined" === typeof hljsLoading) {
    hljsLoading = 1;
    var a = document.getElementsByTagName("head")[0],
        e = document.createElement("link");
    e.type = "text/css";
    e.rel = "stylesheet";
    e.href = "//cdnjs.cloudflare.com/ajax/libs/highlight.js/9.0.0/styles/default.min.css";
    a.appendChild(e);
    e = document.createElement("script");
    e.type = "text/javascript";
    e.onload = function () {
        var d = {}, f = 0;
        hljs._hb = function (b) {
            b.removeAttribute("data-hljs");
            var c = b.innerHTML;
            c in d ? b.innerHTML = d[c] : (7

And we can see, that it added the code to highlight the code block :smiley:
You are welcome, guys! Saved a lot of time :wink:

datitisev on 7 Apr 2016

afaik, the source of the code is : vendor\s9e\text-formatter\src\Plugins\BBCodes\Configurator\repository.xml (line 53)
Then it is "compressed" in vendor\s9e\text-formatter\src\Bundles\Forum\Renderer.php (line 46)
and then in storage\formatter\Renderer_1bb0f654d83b8c65f588e83ef8e10662ef06d0a9.php

I didn't know exactly what was this code, but you are right, it is in the block <bbcode name="CODE">.

Averell7 on 8 Apr 2016

Why does it stop suddenly after the 7 ? Because of the < which follows ?

I wouldn't think so...
As you know, the emails are plain text...
I also wondered about it.
Hopefully @JoshyPHP can tell us more of s9e/TextFormatter about that script :|

datitisev on 8 Apr 2016

@Averell7 Correct on both points. The view uses strip_tags() on the post's HTML and strip_tags() removes any text that follows the <.

JoshyPHP on 8 Apr 2016

An interesting news ! I changed the code which extracts this data from : strip_tags($blueprint->post->contentHtml)
to
$blueprint->post->contentHtml
The result makes sense :

The text of the message was : toto3

The message now contains :

<pre data-hljs="" data-s9e-livepreview-postprocess="if('undefined'!==typeof hljs)hljs._hb(this)"><code class="">toto3</code></pre><script>if("undefined"!==typeof hljs)hljs._ha();else if("undefined"===typeof hljsLoading){hljsLoading=1;var a=document.getElementsByTagName("head")[0],e=document.createElement("link");e.type="text/css";e.rel="stylesheet";e.href="//cdnjs.cloudflare.com/ajax/libs/highlight.js/9.0.0/styles/default.min.css";a.appendChild(e);e=document.createElement("script");e.type="text/javascript";e.onload=function(){var d={},f=0;hljs._hb=function(b){b.removeAttribute("data-hljs");var c=b.innerHTML;c in d?b.innerHTML=d[c]:(7<++f&&(d={},f=0),hljs.highlightBlock(b.firstChild),d[c]=b.innerHTML)};hljs._ha=function(){for(var b=document.querySelectorAll("pre[data-hljs]"),c=b.length;0<c;)hljs._hb(b.item(--c))};hljs._ha()};e.async=!0;e.src="//cdnjs.cloudflare.com/ajax/libs/highlight.js/9.0.0/highlight.min.js";a.appendChild(e)}</script>

So it is the full CDATA block.

And if now I copy below this block, without the back sticks, you will just see the text (toto3) highlighted :

toto3

Averell7 on 8 Apr 2016

Well, this probably references #537, about the HTML emails an all that :wink:

datitisev on 8 Apr 2016

UPDATE : Yes, you are perfectly right. I understood suddenly what happens, see next post.

Averell7 on 8 Apr 2016

Now what happens is clear. The cause is the following code in newPost.blade.php (see issue #537 for details on this file) :

The content of the message is displayed by the following line :
{{ strip_tags($blueprint->post->contentHtml) }}
which php equivalent is :
htmlentities(strip_tags($blueprint->post->contentHtml))

This means :

message is extracted from the database
converted to html, using the normal process of Flarum, when it displays a message, and the result is stored in $blueprint->post->contentHtml
then striptags removes the html specific codes
and htmlentities converts all special characters so that they are no longer html code

It is just nonsense. Building html code to have a beautiful display, then removing all html codes for security reasons.

I think there is no security issue, because all html tags typed by the author of the post are already converted in html entities in the process used by Flarum. If you type <b> in a message, it will appear as is, it is no longer an html code. So, it is possible to use html directly in the email, provided the email is in html format ( #537 discusses this matter).

conclusion : To fix the present bug :

In html format, use : {!! $blueprint->post->contentHtml) !!}
in text format, use : {!! $blueprint->post->content !!}

This will be implemented when #537 will be fixed, and the present bug is only a consequence of #537

Note that in html format, formating (bold, italics, titles, lists, background for quotes, colors...) will be preserved and the text in the email will look veray similar to the text on the forum (provided html formatting is supported by the email program.

Averell7 on 8 Apr 2016

👍1

since #537 is still pending, this bug is still pending also.

Averell7 on 12 Apr 2016

@tobscure Shouldn't we use contentPlain instead of contentHtml for the plain text emails? After all, Markdown works nice as plain-text formatting... (And once we add HTML emails, most people will see the pretty emails anyway.)

franzliedke on 19 Jul 2016

And if we simply dump the HTML in those HTML emails, will the script tags cause any problems with email clients? Does anybody know that? I could imagine that could make the messages look suspicious to anti-spam tools.

Or is there an easy way to render the HTML without any JS, @JoshyPHP ?

franzliedke on 19 Jul 2016

This issue seems to be present also if a message is quoted in the forum and there is a code

PeopleInside on 11 Dec 2018

I had a fix for this lying around for a long time now... :see_no_evil:

@PeopleInside Please do me a favor and tell me whether this commit fixes the issue for you.

franzliedke on 11 Dec 2018

I think the same issue might be there in the mentions extension as well?

postMentioned.blade.php
userMentioned.blade.php

matteocontrini on 11 Dec 2018

@matteocontrini Yes, I am just waiting for confirmation (and also testing myself right now) before pushing the fixes there. Likely fixing #537 in the process. :)

franzliedke on 11 Dec 2018

For me seems not work but not sure.
I done the edit on my server than cleared cache and refreshed the page.

but quoted test still have issue on the forum and on the email if code is present.