Hugo: highlight syntax for functional languages is ugly, but in native pygments is ok

[kwladyka]

Example of code:

(def counter (atom 0))
(def foo (atom 1))
(defn slow-inc [n]
  (swap! counter inc)
  (Thread/sleep 200)
  (inc n))

(pmap
  (fn [_]
    (swap! foo slow-inc))
  (range 100))

@counter
@foo

Wiht https://highlightjs.org/ and darkula looks like this:

With http://pygments.org/ and for example perldoc

while in hugo with perldoc like this:

with pygmentsUseClassic: "true" and pygmentsUseClassic: "false"

Why there are so huge differences? hugo make highlighting really bad. How to fix it?

[kwladyka]

Generated Hugo HTML:

<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-clojure" data-lang="clojure">(<span style="color:#66d9ef">def </span>counter (atom <span style="color:#ae81ff">0</span>))
(<span style="color:#66d9ef">def </span>foo (atom <span style="color:#ae81ff">1</span>))
(<span style="color:#66d9ef">defn </span>slow-inc [n]
  (swap! counter inc)
  (Thread/sleep <span style="color:#ae81ff">200</span>)
  (inc n))

(pmap
  (<span style="color:#66d9ef">fn </span>[_]
    (swap! foo slow-inc))
  (range <span style="color:#ae81ff">100</span>))

<span style="color:#f92672">@</span>counter
<span style="color:#f92672">@</span>foo
=&gt; <span style="color:#ae81ff">1602</span>
=&gt; <span style="color:#ae81ff">101</span></code></pre></div>

As I understand it HTML is wrong.

Compare it to http://pygments.org/:

<div class="hlcode">
<div class="syntax"><pre><span></span><span class="p">(</span><span class="k">def </span><span class="nv">counter</span> <span class="p">(</span><span class="nf">atom</span> <span class="mi">0</span><span class="p">))</span>
<span class="p">(</span><span class="k">def </span><span class="nv">foo</span> <span class="p">(</span><span class="nf">atom</span> <span class="mi">1</span><span class="p">))</span>
<span class="p">(</span><span class="kd">defn </span><span class="nv">slow-inc</span> <span class="p">[</span><span class="nv">n</span><span class="p">]</span>
  <span class="p">(</span><span class="nf">swap!</span> <span class="nv">counter</span> <span class="nv">inc</span><span class="p">)</span>
  <span class="p">(</span><span class="nf">Thread/sleep</span> <span class="mi">200</span><span class="p">)</span>
  <span class="p">(</span><span class="nb">inc </span><span class="nv">n</span><span class="p">))</span>

<span class="p">(</span><span class="nf">pmap</span>
  <span class="p">(</span><span class="k">fn </span><span class="p">[</span><span class="nv">_</span><span class="p">]</span>
    <span class="p">(</span><span class="nf">swap!</span> <span class="nv">foo</span> <span class="nv">slow-inc</span><span class="p">))</span>
  <span class="p">(</span><span class="nb">range </span><span class="mi">100</span><span class="p">))</span>

<span class="o">@</span><span class="nv">counter</span>
<span class="o">@</span><span class="nv">foo</span>
</pre></div>

</div>

Word like atom in (def counter (atom 0)) are in span, but not in Hugo.

[kwladyka]

https://discourse.gohugo.io/t/bug-highlight-syntax-for-functional-languages-is-wrong-but-in-native-pygments-is-ok/19569

Please also see topic here

[moorereason]

Hugo's built-in highlighting uses Chroma. I don't speak clojure, but I'm assuming this is an issue with the Chroma clojure lexer. Please raise an issue upstream in the Chroma project.

[alecthomas]

As stated in the linked issue, I can't replicate this with the chroma command-line tool:

$ cat t.clj
(def counter (atom 0))
(def foo (atom 1))
(defn slow-inc [n]
  (swap! counter inc)
  (Thread/sleep 200)
  (inc n))

(pmap
  (fn [_]
    (swap! foo slow-inc))
  (range 100))

@counter
@foo
$ chroma --html-only --html ./t.clj
<pre class="chroma"><span class="p">(</span><span class="k">def </span><span class="nv">counter</span> <span class="p">(</span><span class="nf">atom</span> <span class="mi">0</span><span class="p">))</span>
<span class="p">(</span><span class="k">def </span><span class="nv">foo</span> <span class="p">(</span><span class="nf">atom</span> <span class="mi">1</span><span class="p">))</span>
<span class="p">(</span><span class="kd">defn </span><span class="nv">slow-inc</span> <span class="p">[</span><span class="nv">n</span><span class="p">]</span>
  <span class="p">(</span><span class="nf">swap!</span> <span class="nv">counter</span> <span class="nv">inc</span><span class="p">)</span>
  <span class="p">(</span><span class="nf">Thread/sleep</span> <span class="mi">200</span><span class="p">)</span>
  <span class="p">(</span><span class="nb">inc </span><span class="nv">n</span><span class="p">))</span>

<span class="p">(</span><span class="nf">pmap</span>
  <span class="p">(</span><span class="k">fn </span><span class="p">[</span><span class="nv">_</span><span class="p">]</span>
    <span class="p">(</span><span class="nf">swap!</span> <span class="nv">foo</span> <span class="nv">slow-inc</span><span class="p">))</span>
  <span class="p">(</span><span class="nb">range </span><span class="mi">100</span><span class="p">))</span>

<span class="o">@</span><span class="nv">counter</span>
<span class="o">@</span><span class="nv">foo</span>
</pre>

This HTML looks identical to Pygments and nothing like the HTML from Hugo.

[kwladyka]

@moorereason Are you sure it is Chroma issue?
From what @alecthomas pasted it looks like Chroma works well.
Should we remove upstream label?

[kwladyka]

Ok guys it is my last issue moving blog to hugo. Any hints what else I can do? Or it is just a Hugo bug and I can do nothing.
@moorereason any suggestion?

[moorereason]

@kwladyka, to work around this issue, you need to force the use of CSS classes instead of inline styles with the pygmentsUseClasses option.

Should we remove upstream label?

No. I have updated the upstream issue with a reproduction of the issue outside of Hugo.

[alecthomas]

Reading back, I honestly can't even tell what the issue is here.

The original complaint is:

with pygmentsUseClassic: "true" and pygmentsUseClassic: "false"
...
Why there are so huge differences? hugo make highlighting really bad. How to fix it?

It seems like the complaint is just that Hugo's syntax highlighting is "ugly", using either Chroma or Pygments. Given that Chroma this Clojure fragment identically to Pygments:

(from the newly created Chroma Playground)

I don't know what else to say.

[kwladyka]

@alecthomas Not sure what you mean.

Here I described WHAT is wrong:
https://github.com/gohugoio/hugo/issues/6068#issue-462288805

Here I described WHY it is wrong:
https://github.com/gohugoio/hugo/issues/6068#issuecomment-510023145

Here is deeper WHY by @moorereason :
https://github.com/alecthomas/chroma/issues/259#issuecomment-511537882

[kwladyka]

to work around this issue, you need to force the use of CSS classes instead of inline styles with the pygmentsUseClasses option.

@moorereason I have right HTML, but still wrong highlight. I mean words are wrapped like they should (like in pygments) this time, but highlight is still wrong.

More details:
https://github.com/alecthomas/chroma/issues/259#issuecomment-511736512

I can't be sure, but it looks like it is Chroma Clojure syntax issue.

[alecthomas]

Is the image I just posted right or wrong?

[kwladyka]

@alecthomas it is the right one

[alecthomas]

Okay so that is generated directly by Chroma (try it yourself on the Chroma Playground) exactly as Pygments does. This suggests that if Hugo does not look correct it is either a) misconfigured or b) has a bug. Either way, this is outside my area of expertise.

[kwladyka]

I can confirm https://swapoff.org/chroma/playground/ show good highlight.
@alecthomas thank you for your time

So back again to Hugo. Oh man, so confusing. Does Hugo use the latest ver. of Chroma? To be honest I can't provide more useful details. We need Hugo expert to figure out this.

[moorereason]

@kwladyka,
It appears to me that Hugo and Chroma are indeed working correctly. I incorrectly conflated two items in this discussion that sent me off in the wrong direction. I'm able to make everything work with the following config.toml settings:

pygmentsCodeFences: true
pygmentsStyle: "perldoc"

@alecthomas,
Sorry for the noise, but thank you for being patient with us as we worked through this.

[kwladyka]

@moorereason hmm it doesn't work for me. I am doing this like that from the beginning.

What is your hugo version?

hugo version
Hugo Static Site Generator v0.55.6/extended darwin/amd64 BuildDate: unknown

I have the same syntax issue with the latest jojomi/hugo docker image.

Can you paste screen / HTML of what you get? Maybe you think it is correct, but it is not? Is it exactly the same like https://swapoff.org/chroma/playground/ ?

[moorereason]

You're correct, @kwladyka. (And I keep being wrong in this thread!)

This chroma bug was fixed upstream in https://github.com/alecthomas/chroma/commit/ea14dd8660c608784452d795ad4f0f2373a4aba8. We need to update the chroma dependency to v0.6.4.

Cc: @bep

[moorereason]

Fixed in 95b1d3013b4717f8b02093a99d1d0c4a6a1ca929.

[kwladyka]

@moorereason Do you know when will be release with this fix?