Hugo: Some documents work with asciidoctor but not with asciidoc

[anthonyfok]

As @nickoe reported in Issue #470:

2015-06-25:

I just tried to add an foo.asciidoc file with a frontmatter and some random chars in the content, but it will not render. The only thing I see is the title (filename) of the post. Is there any way to get more debugging? I do have a2x in my path, if that is what the temporal solution of asciidoc is using.
ERROR: 2015/06/25 exit status 1

2015-07-06:

I just tried to use master, and I can render some asciidoc, but it will not upport the asciidoc include stuff and images seem to be wrong (not rendering), but I have not debugged those. This against 9b3d0cf5bf669603fa017e7054fe0ef8343c9e1d.

2015-09-16, at the closing of #470:

Except that using asciidoc does not work, although asciidoctor works.

Currently (Hugo v0.14 and v0.15-DEV as of today) Hugo first tries to call asciidoctor, and if that fails, tries asciidoc, i.e. asciidoctor is preferred.

@nickoe, if it is not too much trouble for you, could you please post an example where Hugo works with asciidoctor but fails with asciidoc? That way, we can decide if we can tweak the command-line options we use to call asciidoc, or whether to drop asciidoc altogether and use asciidoctor all the time.

Many thanks for your help!

[nickoe]

I just tested with the example website that is available somewhere I don't exactly rember now. The one used in the hugo introduction documentation. To reproduce, just uninstall asciidoctor from your system. You will see that for example the posts has no body text.

So the important thing is to make hugo use asciidoc, which I did bu uninstalling asciidoctor. You should be able to reproduce that easily.

[ingorichter]

I had the same issue, when I tried to use asciidoc for my blog. I couldn't get it to work, since asciidoctor wasn't installed on my machine and the fallback asciidoc, produced an error.
Perhaps it might already be helpful to give a better error message in the case that asciidoc or asciidoctor has issues processing the content. That would have helped me for sure.
I could make the change and provide a PR for that.

[jzacsh]

(btw, @anthonyfok thanks for implementing asciidoc support :)

tl;dr I _think_ it's known that asciidoc and asciidoctor are slightly
different _(ie: aside from hugo, i've also experienced that each have errors
where the other does not)_ - the hugo bug is that the error seems to be
swallowed[1]

---

Below I'm just injecting some debug printlns to see exactly what's given to asciidoc stdin, then manually copy/pasting the output so that I can run asciidoc myself. You can see asciidoc outputs an error message when I run it myself, and this is never shown by hugo[1]. How hugo swallows the error message, I'm unsure - I have to run right now, otherwise I'd look into what the jww package is, and try to fix it...

Versions

• hugo version

  Hugo Static Site Generator v0.17-DEV BuildDate: 2016-07-22T17:02:12-04:00

• asciidoc: 8.6.9
• my hugo content
  If it helps, you can git clone https://github.com/jzacsh/jzacsh-www
  and checkout my hugo site exactly as I'm seeing it now while running the below
  steps, at b4b9f75cf348

Reproducing

So I made the following changes to https://github.com/spf13/hugo/blob/91b61b976d8d/helpers/content.go#L472-L479

$ vi $GOPATH/src/github.com/spf13/hugo/helpers/content.go +:478
# make below changes...

diff --git a/helpers/content.go b/helpers/content.go
index e955b00..d5ba2ec 100644
--- a/helpers/content.go
+++ b/helpers/content.go
@@ -475,6 +475,9 @@ func getAsciidocContent(content []byte) []byte {
        var out bytes.Buffer
        cmd.Stdout = &out
        if err := cmd.Run(); err != nil {
+               jww.ERROR.Println("START dumping stdin what was passed to ", path)
+               jww.ERROR.Println(string(cleanContent[:len(cleanContent)]))
+               jww.ERROR.Println("END finished dumping stdin passed to ", path)
                jww.ERROR.Println(err)
        }

Running hugo -v with said diff:

$ go install github.com/spf13/hugo && hugo -v
INFO: 2016/07/22 17:01:30 hugo.go:467: Using config file: /home/jzacsh/media/src/jzacsh-www/config.toml
... snipped ...
Started building site
INFO: 2016/07/22 17:01:30 content.go:472: Rendering with /usr/bin/asciidoc ...
INFO: 2016/07/22 17:01:30 content.go:472: Rendering with /usr/bin/asciidoc ...
ERROR: 2016/07/22 17:01:31 content.go:478: START dumping stdin what was passed to  /usr/bin/asciidoc
ERROR: 2016/07/22 17:01:31 content.go:479: :inkscape: https://inkscape.org
:nerdsvg: https://content.j.zac.sh/art/20100519_001_nerd-friend.svg
:burrito: https://content.j.zac.sh/art/burrito-riding-bicycle.jpg

Below are some scanned drawings from a time when I filled multiple new pages
per-day in a sketchbook. Some drawings here _may_ also be vector drawings,
created in {inkscape}[inkscape] with a wacom bamboo tablet ({nerdsvg}[eg "nerd
friend"]).

Lastly, I've started doodling a bit on the go, in a https://doodle.j.zac.sh[**doodle**
mobile-app I created] ({burrito}[eg "burrito riding bicycle"]).

== What's Shown Here
Work shown below is a randomly loaded subset of my scanned work _(until I find
time to create something nice small screens)_. In the meantime, feel free to
link:art-index.html[checkout the full listing] of what's scanned.

{-{-HUGOSHORTCODE-1-}-}

ERROR: 2016/07/22 17:01:31 content.go:480: END finished dumping stdin passed to  /usr/bin/asciidoc
ERROR: 2016/07/22 17:01:31 content.go:481: exit status 1
ERROR: 2016/07/22 17:01:31 content.go:478: START dumping stdin what was passed to  /usr/bin/asciidoc
ERROR: 2016/07/22 17:01:31 content.go:479: {-{-HUGOSHORTCODE-1-}-}

ERROR: 2016/07/22 17:01:31 content.go:480: END finished dumping stdin passed to  /usr/bin/asciidoc
ERROR: 2016/07/22 17:01:31 content.go:481: exit status 1
... snipped ...
0 draft content
0 future content
0 expired content
2 pages created
0 non-page files copied
0 paginator pages created
0 categories created
0 tags created
in 257 ms

Copy pasted the above ^ debug output into ./foo.adoc

:inkscape: https://inkscape.org
:nerdsvg: https://content.j.zac.sh/art/20100519_001_nerd-friend.svg
:burrito: https://content.j.zac.sh/art/burrito-riding-bicycle.jpg

Below are some scanned drawings from a time when I filled multiple new pages
per-day in a sketchbook. Some drawings here _may_ also be vector drawings,
created in {inkscape}[inkscape] with a wacom bamboo tablet ({nerdsvg}[eg "nerd
friend"]).

Lastly, I've started doodling a bit on the go, in a https://doodle.j.zac.sh[**doodle**
mobile-app I created] ({burrito}[eg "burrito riding bicycle"]).

== What's Shown Here
Work shown below is a randomly loaded subset of my scanned work _(until I find
time to create something nice small screens)_. In the meantime, feel free to
link:art-index.html[checkout the full listing] of what's scanned.

{-{-HUGOSHORTCODE-1-}-}

BUG: You can see there's a clear error message about "safe mode":

$ asciidoc --no-header-footer --safe  - < foo.adoc
asciidoc: ERROR: <stdin>: line 4: unsafe: ifeval invalid
asciidoc: FAILED: <stdin>: line 4: ifeval invalid safe document

FWIW, without safe mode, I see no errors:

$ asciidoc --no-header-footer - < foo.adoc
<div class="paragraph"><p>Below are some scanned drawings from a time when I filled multiple new pages
per-day in a sketchbook. Some drawings here <em>may</em> also be vector drawings,
created in <a href="https://inkscape.org">inkscape</a> with a wacom bamboo tablet (<a href="https://content.j.zac.sh/art/20100519_001_nerd-friend.svg">eg "nerd
friend"</a>).</p></div>
<div class="paragraph"><p>Lastly, I&#8217;ve started doodling a bit on the go, in a <a href="https://doodle.j.zac.sh"><strong>doodle</strong>
mobile-app I created</a> (<a href="https://content.j.zac.sh/art/burrito-riding-bicycle.jpg">eg "burrito riding bicycle"</a>).</p></div>
<div class="sect1">
<h2 id="_what_8217_s_shown_here">What&#8217;s Shown Here</h2>
<div class="sectionbody">
<div class="paragraph"><p>Work shown below is a randomly loaded subset of my scanned work <em>(until I find
time to create something nice small screens)</em>. In the meantime, feel free to
<a href="art-index.html">checkout the full listing</a> of what&#8217;s scanned.</p></div>
<div class="paragraph"><p>{-{-HUGOSHORTCODE-1-}-}</p></div>
</div>
</div>

---

[1] demonstration showing hugo -v doesn't pass the full error message onto user _(right after Started building site there should be one more ERROR line - see above debugging for more)_:

$ hugo -v
INFO: 2016/07/22 16:42:26 hugo.go:467: Using config file:
...snipped...

Started building site
INFO: 2016/07/22 16:42:26 content.go:472: Rendering with /usr/bin/asciidoc ...
INFO: 2016/07/22 16:42:26 content.go:472: Rendering with /usr/bin/asciidoc ...
ERROR: 2016/07/22 16:42:26 content.go:478: exit status 1
ERROR: 2016/07/22 16:42:26 content.go:478: exit status 1
... snipped ...
0 draft content
0 future content
0 expired content
2 pages created
0 non-page files copied
0 paginator pages created
0 tags created
0 categories created
in 204 ms

[jzacsh]

_(Note: I went to the trouble of trying to figure it out, because I kind of prefer asciidoc over asciidoctor)_

[essen]

Currently I modify asciidoc to remove the if safe() condition and the exception thrown when it's in safe mode. I don't really need -safe personally, but I didn't see where I could configure hugo to stop using it.

[bep]

This issue has been automatically marked as stale because it has not been commented on for at least four months.

The resources of the Hugo team are limited, and so we are asking for your help.

If this is a bug and you can still reproduce this error on the master branch, please reply with all of the information you have about it in order to keep the issue open.

If this is a feature request, and you feel that it is still valuable, please open a proposal at https://discuss.gohugo.io/.

This issue will automatically be closed in four months if no further activity occurs. Thank you for all your contributions.

[bep]

Note/Update: This issue is marked as stale, and I may have said something earlier about "opening a thread on the discussion forum". Please don't.

If this is a bug and you can still reproduce this error on the latest release or the master branch, please reply with all of the information you have about it in order to keep the issue open.

If this is a feature request, and you feel that it is still relevant and valuable, please tell us why.

[dobesv]

I think it would be good to remove the "--safe" option when running asciidoc, because asciidoc cannot generate html in safe mode.

[anthonyfok]

See also https://bugs.debian.org/907478
Bug#907478: hugo: FTBFS if asciidoc is installed but not asciidoctor

To decide: whether to remove the --safe option when running asciidoc, or to remove asciidoc (Python 2 implementation) support altogether and support solely asciidoctor.

Links about asciidoc's obsolescence in the near future:

1. https://github.com/asciidoc/asciidoc

NOTE: This implementation is written in Python 2, which EOLs in Jan 2020. AsciiDoc development is being continued under @asciidoctor. http://asciidoc.org/

1. https://lists.debian.org/debian-devel/2018/10/msg00123.html
   Debian's discussion on whether "Asciidoc transition to the python3 implementation or just EOL"

[RMStoica-zivver]

https://github.com/gohugoio/hugo/issues/1437#issuecomment-243086241 has a great point in that the parameters being passed to external helpers should be chosen by the user, as opposed to hardcoded.

For a bit of context: I'm trying to use .adoc files with hugo, and passing the --safe option to either asciidoc or asciidoctor makes them ignore the include:: directive.

for more context, others are reduced to ugly hacks such as: http://ratfactor.com/hugo-adoc-html5s/

[anoff]

I would support more customization options for the external helpers.
How about the option to define the filetype & corresponding sys call incl. parameters entirely via the CLI?

e.g. hugo --ext "adoc=asciidoctor -r asciidoctor-diagram" to also include the diagram or other libraries