Hugo: 0.60.0 goldmark: <html> indentation in partial/shortcode wrapped in `<pre><code>`

Created on 28 Nov 2019 · 25Comments · Source: gohugoio/hugo

What version of Hugo are you using (`hugo version`)?

$ hugo version
Hugo Static Site Generator v0.60.0/extended darwin/amd64 BuildDate: unknown

Does this issue reproduce with the latest release?

Yes. In lates 0.60.0 using goldmark

Following this discourse discussion, confirmed by @onedrawingperday , I fill a bug here
Using indentation in partial/shortcode (inline or classic) causes the HTML to be wrapped in code blocks
Option [markup.goldmark.renderer] unsafe = true has no effect.

NeedsInvestigation Upstream

Source

divinerites

👍6

Most helpful comment

@divinerites,
Your issue appears to be related to the use of {{ .Content | markdownify }} in layouts/_default/single.html.

moorereason on 29 Nov 2019

🎉1 👍1

All 25 comments

https://commonmark.org/help/tutorial/09-code.html

bep on 28 Nov 2019

Yes, I can understand that indenting with 4 spaces means a code block. No problem.

But I was not expecting that pure HTML/Go-template would be treated like this (it was not with blackFriday).

So, If i read between lines :-) this means that goldmark will treat every 4 spaces as a code block, even for HTML.

So a good prevention measure would be to change "tab->number of space" default in my editor (VSCode) to 3 (at least not 4).

Will look if it changes things. Thanks.

divinerites on 28 Nov 2019

Done some testing with 3 spaces.

If there is strictly 3 spaces as indentation, the result is OK : HTML is written as is.
At the moment where i do 2 indentations (2 x 3) => same problem than before

So I guess that at the moment any HTML code with several indentations (not equal to 3 exactly) will have a problem.
I'll keep testing.

divinerites on 28 Nov 2019

So, after testing a bit I can confirm that :

HTML in a shortcode (inline or called via a partial) is processed by goldmark as if it was markdown.
So if you have ONE indentation of 3 (less than 4) space, it works as HTML. As before.
If you have 4 or more spaces (for ex 3+3 or 3+3+3) it is processed by goldmark and identify and rendered as code block.

I understand that Bep was not surprised by this behaviour, which is fine by me if it is expected behaviour.
But I found this quite surprising because i would never expect HTML being impacted by indentation.

Workaround is to flat your HTML code if it could be called by shortcode. Not read friendly but efficient.

cc @onedrawingperday who helped me a lot on this diagnostic.

divinerites on 28 Nov 2019

👍2

OK, that seems to be a bug in Goldmark. I will create a bug report over there.

I understand that Bep was not surprised by this behaviour,

I didn't say I wasnt surprised, I just pointed to a possible explanation.

bep on 28 Nov 2019

👍1

OK; I had a look at this and I don't understand it. The example shows an inline shortcode using {{< which Goldmark should never see. I could understand it if the {{< was indented 4 chars, but that isn't the case. I tested some variants of this myself, and cannot reproduce.

bep on 28 Nov 2019

Hi @bep - I'm not sure this should be closed.
https://github.com/gohugoio/hugo/issues/6552

I made a screencast I think you might enjoy. Headless only works when the folder name does NOT match the name I'm calling it with 🤯

https://youtu.be/pRznYM-3Itk

jensamunch on 28 Nov 2019

Later today, I'll provide access to the repo and prepare 3 pages with 3 different shortcodes/partials with this problem. Stay tuned.

divinerites on 29 Nov 2019

👍1

Bep & MooreReason, I invited you to the repo.

Use "Migration-en-goldmark" branch.
post vintage.fr.md is working fine (partial is flat HTML)
post test1.fr.md has problem (partial is fully indented HTML)
post test2.fr.md has same problem but different layout because indented a bit differently

First : thanks for your help, and let me know if I can do/test other things.

You can see the 3 post with exactly same code except HTML indentation on the 3 last post here : https://migration-en-goldmark--lagouille.netlify.com/dernieres-nouveautes/

divinerites on 29 Nov 2019

👀1

And test1/test2.html have 3 space indentation, and still get the problem. Mmmmmmmm

divinerites on 29 Nov 2019

@divinerites,
Your issue appears to be related to the use of {{ .Content | markdownify }} in layouts/_default/single.html.

moorereason on 29 Nov 2019

🎉1 👍1

Ahhhh. Yes makes sense. Thanks a lot. I Will take this in account.

So as the behaviour was different with use of blackFriday in 0.59.x, will look forward to see if this is expected behaviour or not and then change my code accordingly to the right practice.
But anyhow i DO thanks you a lot for founding a "reasonable" cause & explanation :-). Thanks.

divinerites on 29 Nov 2019

I think this is the issue I'm hitting in Bootstrap too after updating Hugo.

This worked fine before but now renders invalid HTML:

<div class="bd-example">
  <table class="table">
    <thead>
      <tr>
        <th scope="col">Class</th>
        <th scope="col">Heading</th>
        <th scope="col">Heading</th>
      </tr>
    </thead>
    <tbody>
      <tr class="table-active">
        <th scope="row">Active</th>
        <td>Cell</td>
        <td>Cell</td>
      </tr>
      <tr>
        <th scope="row">Default</th>
        <td>Cell</td>
        <td>Cell</td>
      </tr>

      {{< table.inline >}}
      {{- range (index $.Site.Data "theme-colors") }}
      <tr class="table-{{ .name }}">
        <th scope="row">{{ .name | title }}</th>
        <td>Cell</td>
        <td>Cell</td>
      </tr>
      {{- end -}}
      {{< /table.inline >}}
    </tbody>
  </table>
</div>

XhmikosR on 2 Dec 2019

Also hitting this after updating to v0.60.0. If you use {% %} syntax while invoking shortcodes, HTML tags inside the shortcodes are stripped off.

For example, shortcodes/details.html:

<details>
<summary>{{with .Get "title" }}{{.}}{{else}}Click to expand{{end}}</summary>
{{.Inner}}
</details>

Usage:

{{% details title="foo" %}}
bar
{{% /details %}}

Generated HTML Output:

<p>foo</p>
<!-- raw HTML omitted -->
<p>hello world</p>
<!-- raw HTML omitted -->

ahmetb on 2 Dec 2019

@XhmikosR, how does that render?

@ahmetb, are you enabling unsafe in Goldmark?

moorereason on 2 Dec 2019

@moorereason <pre><code> inside the HTML with unclosed div tags. I'll be able to provide the exact snippet tomorrow since for now I just reduced indentation to work around this issue https://github.com/twbs/bootstrap/pull/29750/files#diff-dcbb1aeb2504a4b02de5c00ba0296ceb

XhmikosR on 2 Dec 2019

@ahmetb, are you enabling unsafe in Goldmark?

First time I'm hearing about Goldmark and the goldmark: configuration.

I don't have markdown: key in my config.yaml and actually things work fine when I downgrade to v0.59.1. Was I relying on undocumenting behavior previously?

ahmetb on 3 Dec 2019

...to add to my previous comment:

I did not realize Blackfriday was replaced with Goldmark.

My Blackfriday config also did not have any relevant keys with respect to unsafe html.

ahmetb on 3 Dec 2019

I tried this without the workaround and it renders to:

HTML

  <table class="table">
    <thead>
      <tr>
        <th scope="col">Class</th>
        <th scope="col">Heading</th>
        <th scope="col">Heading</th>
      </tr>
    </thead>
    <tbody>
      <tr class="table-active">
        <th scope="row">Active</th>
        <td>Cell</td>
        <td>Cell</td>
      </tr>
      <tr>
        <th scope="row">Default</th>
        <td>Cell</td>
        <td>Cell</td>
      </tr>
<pre><code>  
      <tr class="table-primary">
        <th scope="row">Primary</th>
        <td>Cell</td>
        <td>Cell</td>
      </tr>
      <tr class="table-secondary">
        <th scope="row">Secondary</th>
        <td>Cell</td>
        <td>Cell</td>
      </tr>
      <tr class="table-success">
        <th scope="row">Success</th>
        <td>Cell</td>
        <td>Cell</td>
      </tr>
      <tr class="table-danger">
        <th scope="row">Danger</th>
        <td>Cell</td>
        <td>Cell</td>
      </tr>
      <tr class="table-warning">
        <th scope="row">Warning</th>
        <td>Cell</td>
        <td>Cell</td>
      </tr>
      <tr class="table-info">
        <th scope="row">Info</th>
        <td>Cell</td>
        <td>Cell</td>
      </tr>
      <tr class="table-light">
        <th scope="row">Light</th>
        <td>Cell</td>
        <td>Cell</td>
      </tr>
      <tr class="table-dark">
        <th scope="row">Dark</th>
        <td>Cell</td>
        <td>Cell</td>
      </tr>
&lt;/tbody&gt;
</code></pre>
  </table>

XhmikosR on 11 Dec 2019

This is why we use a workaround :-)

Without it is render as pre/code block.

divinerites on 11 Dec 2019

I have quasy-same problem with v0.6x.

See my markup config:

[markup]
    defaultMarkdownHandler = "goldmark"
    [markup.blackFriday]
    angledQuotes = false
    footnoteAnchorPrefix = ""
    footnoteReturnLinkContents = ""
    fractions = true
    hrefTargetBlank = false
    latexDashes = true
    nofollowLinks = false
    noreferrerLinks = false
    plainIDAnchors = true
    skipHTML = false
    smartDashes = true
    smartypants = true
    smartypantsQuotesNBSP = false
    taskLists = true
    [markup.goldmark]
    [markup.goldmark.extensions]
      definitionList = true
      footnote = true
      linkify = true
      strikethrough = true
      table = true
      taskList = true
      typographer = true
    [markup.goldmark.parser]
      attribute = true
      autoHeadingID = true
    [markup.goldmark.renderer]
      hardWraps = false
      unsafe = true
      xHTML = false
    [markup.highlight]
    codeFences = true
    hl_Lines = ""
    lineNoStart = 1
    lineNos = false
    lineNumbersInTable = true
    noClasses = true
    style = "monokai"
    tabWidth = 4
    [markup.tableOfContents]
    endLevel = 3
    startLevel = 2

shortcode code: http://doc.huc.fr.eu.org/en/web/hugo/hugo-shortcodes/#code
shortcode file: http://doc.huc.fr.eu.org/en/web/hugo/hugo-shortcodes/#file
_(for this site's version, I use actually v0.59.1)_
RAW src shortcode code: https://framagit.org/sh-web/hugo/doc.huc.fr.eu.org/raw/master/layouts/shortcodes/code.html
RAW src shortcode file: https://framagit.org/sh-web/hugo/doc.huc.fr.eu.org/raw/master/layouts/shortcodes/file.html

With v0.6(0-4).*, for thoses shortcodes, HTML code disappear.

I push thoses images:
v0 6x_error-shortcode-code_2019-12-12_23-51-36
v0 6x_error-shortcode-file_2019-12-12_23-52-54

hucste on 13 Dec 2019

I confirm this bug always with 0.63.x, and 0.64.x too! :(

hucste on 19 Feb 2020

@hucste v0.65 improves your results.

スクリーンショット 2020-02-24 13 36 09

and tweaking a little, hugo works perfectly.

スクリーンショット 2020-02-24 13 40 07

I just set a valid language as second arg in this line

satotake on 24 Feb 2020

After struggling, I reached quite a simple solution for this issue in terms of commonmark's spec.
Remove blank lines if you want to use html tags inside commonmark.
As long as blank lines are not inserted into html part, you do not need to flat your partial html.

For example, in @XhmikosR 's case, this PR will fix it.

This behavior seems to be commonmark's spec rather than Hugo's issue. I created a minimal example with commonmark.js

satotake on 24 Feb 2020

🎉1

OK, I understand.

@satotake: With your explains, now it runs correctly.

And I test with oldiers 0.6x, it's OK, too.

Thanks for all! :D

PS: I think, you can close this ticket! ;)

hucste on 25 Feb 2020

👍1

Was this page helpful?

0 / 5 - 0 ratings

Related issues

Incorrectly handles safeHTML in RSS <title> tags

antifuchs · 3Comments

Can't specify sitemap variables via front matter

ianbrandt · 3Comments

Add arm64 .deb and binary builds

vielmetti · 3Comments

i18n template escapes HTML

VoidingWarranties · 3Comments

Investigate if we can build the snaps statically

bep · 3Comments

Hugo: 0.60.0 goldmark: <html> indentation in partial/shortcode wrapped in `<pre><code>`

What version of Hugo are you using (hugo version)?

Does this issue reproduce with the latest release?

Most helpful comment

All 25 comments

Related issues

What version of Hugo are you using (`hugo version`)?