Hugo: Shortcode output wrapped in tags under certain conditions

Created on 29 Nov 2015 · 58Comments · Source: gohugoio/hugo

Previous related issue 'Shortcode output wrapped in  tags' #1148 which was closed with 004fcdd.

Consistent incorrect wrapping of shortcode output under certain conditions

Reported in discuss forum here.

Salient points:

1) If there is just one shortcode, it doesn't get wrapped in  tags

2) If there are multiple shortcodes, the first doesn't get wrapped in  tags (probably related to above)

3) In order to ensure no shortcodes are wrapped in  tags, the final shortcode must be followed by a number of distinct  tag wrapped lines proportionate to the number of shortcodes in the document.

By that I mean if you have say 6 shortcodes one after the other, you need 16 lines following them in order for none to be wrapped in  tags. If you have 5 shortcodes one after the other you need 12 lines.

4) 'Lines following shortcode' don't have to come at the end of the document, if you have a series of shortcodes it seems the number of interceeding  tag wrapped lines are cumulatively ranked towards each consecutive shortcode so if you had 30 lines interspaced between shortcodes throughout the document, as long as each shortcode has at least 5 proceeding lines each, they'll all be spared wrapping.

5) Related to point 4, the  tags that get wrapped around shortcodes incorrectly also count towards the number of  tag wrapped lines needed to follow preceeding shortcodes - so enough shortcodes in a row and you seem to be about to start unwrapping some of the 'first' few in the list.

6) Length of shortcode title attribute also affects whether output wrapped in  tags though this has not been investigated in detail

NOTE - I may have miss counted the exact number of lines in between shortcodes, my eyes were actually bleeding...

Keep

Source

josharcheruk

Most helpful comment

Hello there,

Any news about this bug ?

It's a big issue for us right know, which will prevent us to migrate to Hugo on our production server. Workarounds are too complex for users unfortunately :(

Thank you !

carab on 23 Nov 2017

👍8

All 58 comments

As another participant in this discussion on discuss.gohugo.io, I can second that I see the above behaviour. I'm also willing to help continue debugging this.

hnarayanan on 30 Nov 2015

Am a new user... really appreciate the project, is incredibly elegant. Would really like to see bug fixed :)

tburt on 3 Dec 2015

See 1ce184b7f1df10ad8815dfe891e053eb04f099ea

These test cases are deliberately made green, even if they (probably) should not.

Test case 1-3:

The group of a, b, c shortcodes gets wrapped in a P-tag. This kind of makes sense. Is this correct?
b,c i wrapped, d is not. Inconsistent, but what is correct?
A weird one. This is @PuffinBlue s variant with 5 shortcoes and 12 lines ...

bep on 9 Jan 2016

We ran into this as well. We also had the problem, that a markdown list of shortcodes inserted a wrong closing p-tag after an image from within the shortcode.

Would it eventually make sense introduce a param or something like that, that prevents hugo from wrapping shortcode in p-tags?

AnBauer on 13 Jan 2016

@AnBauer Hugo doesn't add any p-tags, we actually try to remove them ... This is coming from Blackfriday (the Markdown engine) ... so a flag wouldn't help ...

bep on 13 Jan 2016

You meant p-tags, didn't you? OK, I thought "shortcode output gets wrapped in a P-tag" is a thing, hugo does. Thanks for clarification!

AnBauer on 13 Jan 2016

You meant p-tags, didn't you?

Yes.

bep on 13 Jan 2016

I also have this issue, although its behaviour is better on Hugo 0.15 but does happen when multiple shortcodes are used on the same page (as already mentioned above). Would love to see Hugo fix the Blackfriday output that adds those

's is that's possible.

Jos512 on 24 Jan 2016

First, a workaround:

Create a shortcode wrap.html:

{{ .Inner }}

Then wrap your shortcode sections in that:

{{< wrap >}}
{{< s >}}
{{< s >}}

{{< s >}}
{{< /wrap >}}

To be even more protective, wrap it in a div:

<div>
{{< wrap >}}
{{< s >}}
{{< s >}}

{{< s >}}
{{< /wrap >}}
</div>

I think Hugo should do two things:

Provide an internal wrapper shortcode (name?)
Provide a way to signal that a shortcode is "block level" and not needed in the Mardkdown evaluation (this is currently links; the ref and relref internal shortcodes). This will add a temporary div to the shortcode placeholders when sent to Blackfriday.
1. The "block level" will be the default.

/cc @halostatue @spf13

bep on 24 Jan 2016

👍2

Currently, i have the same problem and i for me the workaround does not work properly. Is there any progress from the hugo-team?

ahausamm on 9 Feb 2016

@ahausamm what is the problem with my workaround?

bep on 9 Feb 2016

@bep For me, it works only with the

before the shortcodes, the {{ wrap }} does not change anything. But i don't want to have some HTML in the markdown, do you have an example of your working workaround?

{{< go_table >}}
    {{< go_table_row >}}
        {{% go_table_cell %}}
## Cell 1|1
        {{% /go_table_cell %}}
        {{% go_table_cell %}}
## Cell 2|1
        {{% /go_table_cell %}}
        {{% go_table_cell %}}
## Cell 3|1
        {{% /go_table_cell %}}
    {{< /go_table_row >}}
    {{< go_table_row >}}
        {{% go_table_cell %}}
## Cell 1|2
        {{% /go_table_cell %}}
        {{% go_table_cell %}}
## Cell 2|2
        {{% /go_table_cell %}}
        {{% go_table_cell %}}
## Cell 3|2
        {{% /go_table_cell %}}
    {{< /go_table_row >}}
{{< /go_table >}}

{{< go_table >}}
    {{< go_table_row >}}
        {{% go_table_cell %}}
## Cell 1|1
        {{% /go_table_cell %}}
        {{% go_table_cell %}}
## Cell 2|1
        {{% /go_table_cell %}}
        {{% go_table_cell %}}
## Cell 3|1
        {{% /go_table_cell %}}
    {{< /go_table_row >}}
    {{< go_table_row >}}
        {{% go_table_cell %}}
## Cell 1|2
        {{% /go_table_cell %}}
        {{% go_table_cell %}}
## Cell 2|2
        {{% /go_table_cell %}}
        {{% go_table_cell %}}
## Cell 3|2
        {{% /go_table_cell %}}
    {{< /go_table_row >}}
{{< /go_table >}}

gives me this output (first table seems to be okay, second has the

around:

<table>
    <tr>
        <td><h2 id="cell-1-1:d680e8a854a7cbad6d490c445cba2eba">Cell 1|1</h2>
</td>
        <td><h2 id="cell-2-1:d680e8a854a7cbad6d490c445cba2eba">Cell 2|1</h2>
</td>
        <td><h2 id="cell-3-1:d680e8a854a7cbad6d490c445cba2eba">Cell 3|1</h2>
</td>
    </tr>
    <tr>
        <td><h2 id="cell-1-2:d680e8a854a7cbad6d490c445cba2eba">Cell 1|2</h2>
</td>
        <td><h2 id="cell-2-2:d680e8a854a7cbad6d490c445cba2eba">Cell 2|2</h2>
</td>
        <td><h2 id="cell-3-2:d680e8a854a7cbad6d490c445cba2eba">Cell 3|2</h2>
</td>
    </tr>
</table>

<p><table>
    <tr>
        <td><h2 id="cell-1-1:d680e8a854a7cbad6d490c445cba2eba">Cell 1|1</h2>
</td>
        <td><h2 id="cell-2-1:d680e8a854a7cbad6d490c445cba2eba">Cell 2|1</h2>
</td>
        <td><h2 id="cell-3-1:d680e8a854a7cbad6d490c445cba2eba">Cell 3|1</h2>
</td>
    </tr>
    <tr>
        <td><h2 id="cell-1-2:d680e8a854a7cbad6d490c445cba2eba">Cell 1|2</h2>
</td>
        <td><h2 id="cell-2-2:d680e8a854a7cbad6d490c445cba2eba">Cell 2|2</h2>
</td>
        <td><h2 id="cell-3-2:d680e8a854a7cbad6d490c445cba2eba">Cell 3|2</h2>
</td>
    </tr>
</table></p>

Any ideas?

ahausamm on 10 Feb 2016

@ahausamm use plain HTML tables and the {{< -- your use case was not my intented target for my "workaround" ... and It looks like a hack.

bep on 10 Feb 2016

@bep okay, thanks for your answer. For me, plain HTML isn't the right solution (for the moment) because i want to have some custom section with custom templates. So i will try another way or simply wait on the next update of hugo.

ahausamm on 10 Feb 2016

the workaround from @bep came unfortunately too late for my project, because we already have tons of mds with tons shortcodes in it. what I did was creating a grunt-task, which simply removes all wrong p-tags with a regex :-)

AnBauer on 10 Feb 2016

In my case a temporary workaround is to wrap each shortcode in a block-element for example a div

<div>
{{< shortcode >}}
</div>

FYI

maxxscho on 19 Feb 2016

I think, the workaround of @maxxscho would be the best workaround for this problem. I had another nice discussion about this problem in the gohugo-forum: https://discuss.gohugo.io/t/single-page-with-nested-content/2645

ahausamm on 22 Feb 2016

@maxxscho: nice idea. Unfortunately I have multiple shortcodes already consisting of block elements. And I didn't want html-code in the md files or needless nesting.
These are the patterns for the replace-job btw. (after htmlmin ran):

patterns: [
    {
        match: /<p><(div|section|md-content|h3|ul)(.*?)>/g,
        replacement: '<$1$2>'
    },
    {
        match: /<.(div|section|md-content|h3|ul)><.p>/g,
        replacement: '</$1>'
    }
]

AnBauer on 22 Feb 2016

As I think I said before, these "workarounds" should be included in the Hugo core. If we have to add a div to trick Blackfriday, we will remove it afterwords.

I will do it ... once I get some spare time.

bep on 22 Feb 2016

👍3

@AnBauer: nice one. Can you show me your gulp task, with handles the replacement? :)
Thx!

maxxscho on 22 Feb 2016

@maxxscho i think he uses gulp-replace-task (https://www.npmjs.com/package/gulp-replace-task). @AnBauer please correct me if i'm wrong.

ahausamm on 24 Feb 2016

@AnBauer actually im wondering how i can use gulp with hugo server -w to create valid html code and replace the "shortcoddmistakes"

ahausamm on 24 Feb 2016

I'm using grunt for that. But it should be nearly the same with gulp. Important is to htmlmin before replacing, so all line breaks are deleted. The config for grunt-replace looks like:

replace: {
    dist: {
        options: {
            patterns: [
                {
                    match: /<p><(div|section|md-content|h3|ul)(.*?)>/g,
                    replacement: '<$1$2>'
                },
                {
                    match: /<.(div|section|md-content|h3|ul)><.p>/g,
                    replacement: '</$1>'
                }
            ]
        }
    ,
        files: [
            {
                cwd: '<%= dist %>/',
                expand: true,
                flatten: false,
                filter: 'isFile',
                src: ['**/*.html'],
                dest: '<%= dist %>/'
            }
        ]
    }
}

The elements in the regex depend on the project, of course. E.g. I also use angular material, so I needed md-content.

@ahausamm: I have a custom "hugo" - task. When this one ran follows htmlmin and then replacing.

AnBauer on 25 Feb 2016

This issue has been automatically marked as stale because it has not been commented on for at least four months.

The resources of the Hugo team are limited, and so we are asking for your help.

If this is a bug and you can still reproduce this error on the master branch, please reply with all of the information you have about it in order to keep the issue open.

If this is a feature request, and you feel that it is still valuable, please open a proposal at https://discuss.gohugo.io/.

This issue will automatically be closed in four months if no further activity occurs. Thank you for all your contributions.

bep on 28 Feb 2017

Note/Update: This issue is marked as stale, and I may have said something earlier about "opening a thread on the discussion forum". Please don't.

If this is a bug and you can still reproduce this error on the latest release or the master branch, please reply with all of the information you have about it in order to keep the issue open.

If this is a feature request, and you feel that it is still relevant and valuable, please tell us why.

bep on 1 Mar 2017

Thanks for Hugo,

I'm still a Hugo n00b, but I can confirm that I have this problem, about the p-wrapping.

Hugo version: Hugo Static Site Generator v0.20-DEV linux/amd64 BuildDate: 2017-03-13T09:34:48+01:00
Installed using: go get -v github.com/spf13/hugo

_shortcodes/aaa.html:_

<div class="aaa">
    This is the aaa shortcode
</div>

_shortcodes/aab.html:_

<div class="aab">
    This is the aab shortcode.
    {{ .Inner }}
</div>

page.md variant 1:

+++
+++

{{% aaa %}}
{{% aab %}}
My Markdown content goes here
{{% /aab %}}

Output 1:

<p><div class="aaa">
    This is the aaa shortcode
</div>

<div class="aab">
    This is the aab shortcode.
    <p>My Markdown content goes here</p>

</div>
</p>

page.md variant 2:

+++
+++

{{% aaa %}}

{{% aab %}}
My Markdown content goes here
{{% /aab %}}

Output 2:

<div class="aaa">
    This is the aaa shortcode
</div>


<p><div class="aab">
    This is the aab shortcode.
    <p>My Markdown content goes here</p>

</div>
</p>

As you can see by inserting newlines I can move the P wrapping around, but I can't get rid of it in a sane manner.

Again, thanks for working on Hugo, it's awesome.

bashlund on 13 Mar 2017

I can also confirm that this issue still happens on Hugo 0.19. Like Thomas mentioned above, the issue in my case also happens when 'irregular' text is used inside a shortcode. With that, I mean that this text without line endings gives no problem:

{{% note %}}Example text.{{% /note %}}

Renders as:

<div class="note">
    <p>Example text.</p>
</div>

Going beyond regular text in the shortcode introduces the additional  tags.

{{% note %}}Example text.
And another line.{{% /note %}}

Renders the invalid HTML (according to the w3.org validator) of:

<div class="note">
    <p><p>Example text.
And another line.</p>
</p>
</div>

Likewise, the shortcode use like shown below also generates the invalid  tags:

{{% note %}}Example text.
![Example image](/assets/example.png){{% /note %}}

<div class="note">
    <p><p>Example text.
<img src="http://localhost:1313/assets/example.png" alt="Example image" /></p>
</p>
</div>

From my understanding, the  tag issue seems to happen whenever there's a new line (Enter) used in the shortcode, because I have no trouble with  tags when I include a long paragraph of text inside the shortcode.

Hugo Static Site Generator v0.19 windows/amd64 BuildDate: 2017-02-27T12:36:30+01:00
GOOS="windows"
GOARCH="amd64"
GOVERSION="go1.8"

Jos512 on 15 Mar 2017

I can also confirm that I am getting this issue. The content from my _index.md file is as follows:

title:          "Freelance software engineer"
description:    "Hello. I am a freelance Full-stack software engineer originally from Ireland and based in Berlin, Germany. This is my online portfolio which details my skills, experience and some of the projects I have worked on."
identifier:     "home"
---

## Skills
- Development of mobile applications, both cross platform and native.
- Effective communication skills; training, presentations, documentation and delegation.
- Practice and advocate the highest development standards.
- In–depth understanding of technologies across multiple domains.
- Strong analytical thinking and problem solving skills.
- Life–long learner with strong interest in continuous professional development.

## Languages
{{% taxonomies items="languages" %}}

## Frameworks
{{% taxonomies items="frameworks" %}}

## Infrastructure
{{% taxonomies items="infrastructure" %}}

## Tools
{{% taxonomies items="tools" %}}

Note: Calling the taxonomies shortcode using < as well as % doesn't make a difference to the outcome of the issue.

I have created a shortcode for taxonomies as follows:

{ $which    := .Get "items" }}
{{ $items   := where .Site.Pages "Section" $which }}
<ul class="taxonomies-list">
    {{ range sort $items ".Params.proficiency" "desc" }}
        {{ if .Params.proficiency }}
            <li>
                {{ .Render "card" }}
            </li>
        {{ end }}
    {{ end }}
</ul>

I am getting the stray p tags in the generated content as follows:

<h2 id="frameworks">Frameworks</h2>

<p> <!-- This should not be here -->

<ul class="taxonomies-list">    
    <li>
        <a class="term-card" href="/frameworks/mongoose" title="Mongoose" data-proficiency="4">
            Mongoose
        </a>
    </li>
    <!-- Other list items go in here -->
</ul>

</p> <!-- This should not be here -->

Note: The first call to the taxonomies shortcode does not raise this issue - only subsequent calls.

I have checked the content itself and the shortcode to see if there are any stray carriage returns. tabs or spaces that shouldn't be there.

I am using Hugo 0.20.0 and the source for my project can be seen at https://github.com/matfin/mattfinucane.com.

The live site where you can see the generated p tags is at https://mattfinucane.com

matfin on 16 May 2017

Found by coincidence: on my project it worked to add

[empty line]
</p>

right before the {{< shortcode >}} Hugo stopped generating p-tags in the shortcode after inserting that.

This is a bug and not feature request.

[Hugo Static Site Generator v0.24.1 linux/arm BuildDate: 2017-06-24T10:27:49+02:00]

TMBBerlin on 3 Jul 2017

😕1

Any idea when this will be fixed?
is there a functional workaround? All workarounds mentioned above does not work here.

Hugo Static Site Generator v0.24.1 darwin/amd64 BuildDate: 2017-06-29T11:20:08+02:00

Jehu on 30 Jul 2017

Workaround:

{{% grid class="row" %}}
{{% grid class="col-sm-4" %}}
foo
{{% /grid %}}
{{% /grid %}}

<!-- bar -->

{{% grid class="row" %}}
{{% grid class="col-sm-4" %}}
baz
{{% /grid %}}
{{% /grid %}}

was p-wrapped.

With:

</p>
{{% grid class="row" %}}
{{% grid class="col-sm-4" %}}
foo
{{% /grid %}}
{{% /grid %}}

<!-- bar -->

{{% grid class="row" %}}
{{% grid class="col-sm-4" %}}
baz
{{% /grid %}}
{{% /grid %}}

the second block of grid was p-wrapped.

</p>
{{% grid class="row" %}}
{{% grid class="col-sm-4" %}}
foo
{{% /grid %}}
{{% /grid %}}
<!-- bar -->
{{% grid class="row" %}}
{{% grid class="col-sm-4" %}}
baz
{{% /grid %}}
{{% /grid %}}
<p>

solved the pb. Note that there is no more empty lines between the shortcodes (important).

ldidry on 3 Oct 2017

Hello there,

Any news about this bug ?

It's a big issue for us right know, which will prevent us to migrate to Hugo on our production server. Workarounds are too complex for users unfortunately :(

Thank you !

carab on 23 Nov 2017

👍8

A similar situation:

columns.html:

 <div class="splitbox"><div class="left">
 {{ .Inner }}
 </div><div style="clear:both"></div></div>

column.html:

 </div><div class="right">

Then, when using this:

{{% columns %}}
On the *left*
{{< column >}}
On the *right*
{{% /columns %}}

There is an extra, completely empty  that gets added only on the second column, making the final result really ugly.

henryiii on 4 Dec 2017

I put @henryiii's comment in a test. This currently is green, but I think you'll spot the issue:

func TestFail(t *testing.T) {
    siteConfig := ""

    th, h := newTestSitesFromConfig(
        t,
        afero.NewMemMapFs(),
        siteConfig,
        "layouts/shortcodes/columns.html",
        `<div class="splitbox"><div class="left">
 {{ .Inner }}
 </div><div style="clear:both"></div></div>`,
        "layouts/shortcodes/column.html",
        `</div><div class="right">`,
        "layouts/_default/single.html",
        "{{ .Content }}",
    )

    fs := th.Fs

    writeSource(t, fs, "content/shortcode_test.md", `+++
title="test"
+++
{{% columns %}}
On the *left*
{{< column >}}
On the *right*
{{% /columns %}}`)

    err := h.Build(BuildCfg{})

    require.NoError(t, err)

    outputfile := "public/shortcode_test/index.html"
    file, err := th.Fs.Destination.Open(outputfile)

    buf := new(bytes.Buffer)
    buf.ReadFrom(file)
    contents := buf.String()

    fmt.Println(contents)


    th.assertFileContent(outputfile,
        `<div class="splitbox"><div class="left">
 <p>On the <em>left</em>
</div><div class="right">
On the <em>right</em></p>

 </div><div style="clear:both"></div></div>`,
    )
}

While I tried to make it work, I realized, Hugo or Blackfriday insert a  tag before it runs the shortcodes. Just remove the virtual shortcode files and you see:

<p>{{%% columns %%}}
On the <em>left</em>
{{&lt; column &gt;}}
On the <em>right</em>
{{%% /columns %%}}</p>

rmetzler on 15 Dec 2017

👍3

I hope this error get fixed sometime. Hugo can do all kinds of cool stuff, but it still adds unnecessary HTML code that breaks theme formatting.

In my case that means I have to rewrite my content to not trip into this bug, and that shouldn't be the case with a flexible website generator as Hugo.

Jos512 on 10 Feb 2018

@Jos512 I found that running HTML Tidy on the Hugo output can fix (and strip) most of the superfluous  tags.

DominikDeak on 11 Feb 2018

👍1

I use HTML Tidy to validate the output of any markup before publishing. So in case of any error or warning I'm able to cancel publishing to avoid buggy markup goes outside. So when I used Hugo's shortcodes the first time I got the following HTML Tidy warnings:

Warning: inserting implicit <p>
Warning: trimming empty <p>

After checking my files again and again I updated to Hugo's current master. As nothing helped I checked the issues and found this one. So it's kind of ironic to hear that HTML Tidy could be used as a workaround correcting/modifying the markup as I prefer to eliminate the cause.

atomota on 7 Mar 2018

👍1

I believe I'm encountering this issue too.

I have a simple shortcode called "test.html":

<!-- Test shortcode. -->

<span>{{ .Inner }}</span>
````

I try to use it inside a paragraph in my Markdown files:

Heading

{{% test %}}Some text.{{% /test %}}{{% test %}}Some more text.{{% /test %}}{{% test %}}And yet more text.{{% /test %}}

This outputs as:

Heading

Some text.

Some more text.

And yet more text.

```

This is not only inconsistent (the first span doesn't have an extra  tag, whereas the others do), but is also invalid, illegal HTML.

This makes shortcodes very frustrating to use, or even entirely unusable.

I tried the {{% wrap %}} solution but it makes no difference in my case. Using {{< ... >}} isn't an option for me as I need the content to be parsed as Markdown, for e.g. **bold text**.

If this bug is proving too difficult to fix (it's been open since 2015), and if HTML Tidy is able to resolve it, then perhaps it might be worth considering integrating HTML Tidy directly into Hugo? Outputting invalid HTML seems like quite a major issue to me.

ChrisTucker2000 on 15 Mar 2018

Another example (related to the one above), and I don't think this is one that HTML Tidy would be able to fix.

Another shortcode template called "test2.html":

<!-- Test shortcode 2. -->

<div>{{ .Inner }}</div>

Markdown file:

# Heading

{{% test2 %}}
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.{{% test %}}Some text.{{% /test %}} Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat.
{{% /test2 %}}

This outputs as:
```

Heading

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.

Some text. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat.

```

What I intended and expected was one paragraph containing a bit of text in a span tag. Instead what I got was two separate paragraphs. This breaks the page.

ChrisTucker2000 on 15 Mar 2018

I think I see what's going on here (in my case at least).

<!-- Test shortcode. -->

<span>{{ .Inner }}</span>

Hugo seems to see the line breaks as a trigger to create new paragraphs. If I uglify my shortcode and compress it down to the following...

<!-- Test shortcode. --><span>{{ .Inner }}</span>

...then suddenly it works fine and doesn't break my paragraphs. However, this makes my files very ugly and hard to maintain, especially when they're more complex than this simple example.

I think this is a bug in Hugo, and that Hugo needs to ignore whitespace-only blocks when deciding if there are multiple paragraphs or not. Basically, you should have multiple paragraphs if you have two or more lines of meaningful text separated by whitespace. If you have one line of meaningful text, with any amount of whitespace before or after it, then it should be only one paragraph.

ChrisTucker2000 on 15 Mar 2018

👍2

@ChrisTucker2000 thank you for the head up! And you're correct in your hypothesis, by removing the extra white lines the HTML generation become correct. I can reproduce it with my set-up, and I suspect it's partially related to #1778 too (usually, ToCs are plagued by random  tags).

I think it is possible to fix the issue by disabling specific blackfriday's features during {{% wrap %}} expansion for shortcodes, but it will require the re-write of several Hugo's components (and consequently breaking things for users relying to the broken mechanism). Let's see what @bep has to say.

Dr-Terrible on 15 Mar 2018

Hugo seems to see the line breaks as a trigger to create new paragraphs. If I uglify my shortcode and compress it down to the following...

That’s Markdown, not Hugo, no? If you want to use inline shortcodes, I would recommend using the <> rather than the {{}}, but this is starting to get forum-ish in nature...

rdwatters on 15 Mar 2018

Maybe it's the Markdown engine. When I say "Hugo", I really just mean "Hugo and all of its internals and dependencies".

I'm a newcomer to this issue and haven't been following it until now, but I did search around before and I came across old issues where a blocks/inline option was being discussed for the Markdown processor, and apparently in the end it was set up so that Hugo removes generated paragraphs from the Markdown output if the Markdown is one line. Perhaps I'm mistaken, but that's what I was referring to when I said "Hugo seems to see the line breaks as a trigger..." and "Hugo needs to ignore whitespace-only blocks when deciding...". I might be misunderstanding things though.

If by <> you mean {{< tags instead of {{%, that resolves the issue but then introduces a new issue in that Markdown no longer functions in the shortcode. Even inline shortcodes can need Markdown support.

ChrisTucker2000 on 15 Mar 2018

I see @ChrisTucker2000 @Jos512 . It was only a question since I haven't been involved in Hugo much for the past half year :smile:

If by <> you mean {{< tags instead of {{%, that resolves the issue but then introduces a new issue in that Markdown no longer functions in the shortcode. Even inline shortcodes can need Markdown support.

Maybe something like this for a param in the {{<>}} shortcode: .Get 1 | markdownify (positional) or .Get "mynamedparam" | markdownify (named). Sorry I can't be of more help.

rdwatters on 18 Mar 2018

Here's another simple way to reproduce this issue (using hugo v0.42.2):

layouts/shortcodes/bug.html:

content/posts/bug.md:

{{< bug >}}

{{< bug >}}

Resulting html:

<p>xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
</p>

In this case, any of the following prevents the  tag from being inserted:

Removing a single character from the shortcode, reducing it from 81 characters to 80 characters
Using the shortcode in the post any number of times other than twice
Wrapping each shortcode in the post with a <div>

tdryer on 8 Jul 2018

👍3

This bug is still present in 0.46 and none of the above workarounds seem to work for me. I am using sections in my shortcodes and placing them in p makes the markup invalid - which definitely is a showstopper. Does anyone have a pointer to the regex that is supposed to insert these p's?

danbruen on 15 Aug 2018

👍1

Complex designs are always messed up because of this.
I have to leave HUGO in favour of another static site builder.

Jehu on 15 Aug 2018

👍1

What a shame... but still, ancient template toolkit (TT2) never had problems like this. Might as well migrate back again.

danbruen on 15 Aug 2018

FYI: mmark shows the same behaviour. If it really is a bug in blackfriday and not hugo itself, it has been in there for a long time, as mmark started off as a fork of blackfriday and has not seen much development in the last couple of years.

danbruen on 15 Aug 2018

And, for the record: Same problem with using pandoc instead of blackfriday. At this point I am pretty sure the problem has its roots in hugo itself.

danbruen on 15 Aug 2018

I was happy to see this bug fixed in 0.47. This corrected several cases of  elements being incorrectly generated in our web site. Sadly, after switching to 0.47, this problem is still occurring in one specific case.

To repro, grab https://github.com/istio.github.io at commit be7c374f297a712fb6e5cd1be98b656a3b52732f and build the site with Hugo 0.47. This is where the error occurs:

./public/docs/examples/multicluster/iks-icp/index.html
- 7563:6: ERROR: Unexpected end tag : p (line 7563)

geeknoid on 18 Aug 2018

The problem with our content is that there was an extra period after a closing shortcode:

{{< text >}}
some preformatted text
{{< /text >}}.

where text is a shortcode that produces a pre/code block. The presence of this period caused a p block to be wrapped around the pre/code sequence:

<p>
<pre><code>some preformatted text
</code</pre>
</p>

which is invalid HTML. Removing the period in the markdown fixed the problem.

geeknoid on 19 Aug 2018

👍1

In some cases reported above and in my own case, Hugo wraps the second shortcode with p. For example, if there is a markdown like the following,

{{< first-shortcode />}}

{{< second-shortcode />}}

{{< third-shortcode />}}

the markdown will be converted to the following:

The output from first-shortcode

<p>The output from second-shortcode</p>

The output from third-shortcode

A simple workaround for this case is (1) define an empty shortcode like this:

<!--
  This shortcode is used as a workaround for a Hugo's bug.

  [Issue #1642] Shortcode output wrapped in <p> tags under certain conditions
    https://github.com/gohugoio/hugo/issues/1642
-->{{ with .Inner }}{{ end -}}

and (2) put the shortcode as the second shortcode.

{{< first-shortcode />}}

{{< empty "Workaround for a Hugo's bug. Don't remove this." />}}

{{< second-shortcode />}}

{{< third-shortcode />}}

TakahikoKawasaki on 9 Sep 2018

😕1

Following the trail of issues and discussions of shortcodes and  tags, I wind up here. I am not certain that this is the same issue, but I get an empty paragraph generated after a shortcode (such as highlight), _but only with certain themes_. I made a simple test site and the issue does not come up if I use the "simple-hugo-theme" theme. It looks like a problem somewhere in the templates of my custom theme, and I have not traced the root of it. Just a heads-up to anyone searching for this issue like me to check the theme.

clutterstack on 30 Jan 2019

@clutterstack. With closed issues, you're likely to get very little (if any) feedback. I would recommend taking your issue to the forums at https://discourse.gohugo.io and get feedback directly from other Hugo users.

rdwatters on 1 Feb 2019

@clutterstack pls. see this discussion: https://discourse.gohugo.io/t/solved-empty-p-before-and-after-plain-html-in-markdown-files/5559

You must handle your shortcodes like block elements.

christophlieck on 20 Feb 2019

👍1

@rdwatters @christophlieck Thanks for your comments -- in fact, I hadn't expected any feedback! I'm sure I've done something simple wrong in my own theme templates.

I commented because my comment would been helpful to me, had I seen one like it in my prior searches. It took me some time to discover that my problem was not with my shortcode or the way I was using it in markdown files.

clutterstack on 20 Feb 2019

I just came across this when looking for a solution to remove

elements from around the {{ .Content }} when it is generated. it seem hugo or blackfriday assumes you need these. I am using hugo to generate a site.

I have BODY element wrapped in my markdown file at the start and finish in each md file. not really using hugo for mark down but for keeping html code in every file so it will generate a site with header and footers in every file.

To remove those annoying PARAGRAPH elements for a generated site i find a sed command on linux in my hugo project. This will remove the paragraph elements. I have to do this every time i generate the site. Here are the commands I use.

find public -name "*.html" -exec sed -i 's/<body>/<body>/' {} \;
find public -name "*.html" -exec sed -i 's/<\/body><\/p>/<\/body>/' {} \;